Questions tagged [newlines]

Use for questions about representation and manipulation of the end-of-line marker in text files, in particular when editing the same file on different operating systems where the representation may vary.

Filter by
Sorted by
Tagged with
0 votes
3 answers
140 views

How to convert CR to CRLF line endings on a file?

An ASCII text file, with CR line terminators, (Mac style) needs to be converted to have CRLF line terminators (DOS/Windows style). Search for CR not followed by LF, and replace that CR with a CRLF. So ...
Pro Backup's user avatar
  • 4,904
0 votes
2 answers
190 views

How to add an empty line after lines starting with a timestamp

The log viewing experience is so bad in my current company. I want to add new line to the logs that I've trimmed for a specific time duration. I want a new line after every date-time. The date-time ...
achhainsan's user avatar
0 votes
1 answer
136 views

"man cat" mentions "LFD." What is "LFD?"

man cat says: -v, --show-nonprinting use ^ and M- notation, except for LFD and TAB I checked info cat, and it uses the same acronym: ‘-v’ ‘--show-nonprinting’ Display ...
Daniel Kaplan's user avatar
2 votes
3 answers
2k views

How to deal with CRLF, CR line endings

I have a Perl script that parses data sent to me from a bunch of school districts. I'm adding a new school and have run into a problem I've never faced before. When I do $line = <INPUT>, it ...
Chanel's user avatar
  • 99
2 votes
5 answers
179 views

How to remove newline characters inside paragraphs with sed or awk

I was wanting to know how to remove newline characters in paragraphs in this book and others for use in a kindle. The desired effect is to have each block that is separated by a blank line to be ...
Seth Buchanan's user avatar
10 votes
3 answers
2k views

Add a CR before every LF

I have a text file being processed on a Windows machine. There is a need to remove trailing tab characters prior to using bcp utility to load the data from the file into a database table. The ...
knot22's user avatar
  • 271
0 votes
1 answer
45 views

maintain appended logs in separate file despite tail command termination

I have tomcat catalina.log file that keeps getting updated. I wish to maintain its past 500 lines + any appended logs into a new file catalina.log_new for the next 2 hours. To start off I run nohup ...
Ashar's user avatar
  • 491
0 votes
1 answer
60 views

How to find and manipulate files that contain newline in filename?

follow up on How to find files that contain newline in filename? , I need to do some manipulation on the result, for simplicity lets suppose I need to chown them. Tried following, but does not work: # ...
Nick's user avatar
  • 125
-1 votes
1 answer
228 views

Newline character '\n' (return key) in *nix!

While studying Linux, I noticed that the newline character is used not only to separate different lines - that is to set the beginning and the end of a line - but also is used to give some input to ...
Angelo Giannuzzi's user avatar
0 votes
1 answer
119 views

grep searching end of line pattern from DOS files

Like Detecting pattern at the end of a line with grep, but for DOS files of CRLF line terminators. Like in https://unix.stackexchange.com/a/462633/374303, one way is to use dos2unix but I don't have ...
xpt's user avatar
  • 1,488
2 votes
2 answers
236 views

Do executable shebang scripts always require LF line endings?

On Stack Overflow a comment to Can PowerShell Core handle ps1 files with CRLF line endings in Linux environments? asserted that executable files that use a shebang must use only the LF line ending, ...
Garret Wilson's user avatar
0 votes
1 answer
94 views

GNU sed: append both BEFORE name start and AFTER name end (output is: "$ ls -xN")

Please could someone check if my GNU sed script is the nice way to do it or if it's some sort of bad / wrong hack :p $ ls -xN 1 2 3 4 $ ls -xN | sed -E "s/(.{1})$/\1\.log/" | sed -E "s/...
T. Caio's user avatar
  • 129
0 votes
1 answer
207 views

MacOS append char at end of line with sed

I'm on MacOS where I want to enclose field names by "" in the header. Reading the file with sed, I use this command sed -i.bkp -e '1s/,/","/g' -e '1s/.*/"&/g' -e '1s/.*/&...
user538064's user avatar
1 vote
1 answer
621 views

How do I search for a string and then insert a newline at the beginning, for all lines in file?

I want to search an entire file for the string --- when it appears at the beginning of a line. I then want to insert a newline before each instance of that string. In my .vimrc file, I added this ...
Bastette's user avatar
1 vote
2 answers
587 views

How to prevent grep --color from botching the output when a carriage return from Windows line ends is matched?

I have a file with some lines likes x y and when I run grep -E "x$" filename.txt it does not match anything. As vi shows ^M in the end of most lines, I guess the problem is mixed Dos and ...
allo's user avatar
  • 936
0 votes
1 answer
180 views

How to make Zsh interpret \n in parameter expansion

I’m using Zsh’s parameter expansion flags to split a string into an array. This works: string="one two" array=("${(f)string}") echo "${array[1]}" # Returns: # one This ...
user137369's user avatar
2 votes
3 answers
3k views

Stuck at removing a trailing newline from a string variable

Sometimes simplest things take your time in shell scripting like crazy. Content='me \n you \n him \n' echo $Content > Names.txt When I open the Names.txt it has two empty lines at the end. I want ...
Saeed Neamati's user avatar
31 votes
2 answers
3k views

When using cat to create file, I have to type ctrl+d twice to finish the input, is this expected? [duplicate]

I just learned a trick to create a new file with the cat command. By my testing, if the last line is not followed by a newline, I have to type ctrl+d twice to finish the input, as demonstrated below. ...
Fajela Tajkiya's user avatar
-2 votes
2 answers
574 views

Why does echo in Bash modify the content of a independent variable?

I have the following code (minimal example). #!/bin/bash ...
kaiya's user avatar
  • 222
4 votes
7 answers
562 views

awk: deal with newline separation in wrong place

I have some data that looks like this: abc 123 456 789 def 111 222 333 ghi 999 888 777 666 i.e. the records are separated by multiple newlines but in the wrong place. What I want is to get it like ...
dain's user avatar
  • 145
1 vote
0 answers
59 views

Bash prompt with a 'non' normal character not wrapping new line with long commands

So, when typing in long commands, on most of the terminals I use except for UXTERM, the bash prompt will overwrite the text on the same line and won't go to a new line. This is my bash prompt in the ...
a e's user avatar
  • 31
1 vote
2 answers
701 views

sed replacing a string with itself and a new line

I have the following string multiple times in a html file <br> <h4 id=... Now, I know sed might not be perfect for html editing, but could tell me how can I replace br> <h4 id with br&...
radui's user avatar
  • 111
1 vote
1 answer
619 views

When assigning a command output to a variable, the new lines are replaced by white spaces [duplicate]

I have a file and the idea is to grep the content following a pattern ("create mode") and to store this content in a bash variable in order to use it then in a curl statement. I have the ...
nicg's user avatar
  • 81
2 votes
1 answer
109 views

Can ZFS snapshot names contain newlines? If yes, how to parse the output of zfs list -t snapshot?

The following question relates to ZFS on Linux (ZoL) / OpenZFS, as it is provided e.g. in Debian Buster and Debian Bullseye. As the title says, I'd like to know whether ZFS snapshot names ...
Binarus's user avatar
  • 3,320
0 votes
2 answers
220 views

Use a modifier with find command's regex?

I'm trying to use the find command's regex to match newlines using the dot character. This usually is accomplished via the s modifier; however, different regex engines have different formats and I ...
Harry Muscle's user avatar
  • 2,497
2 votes
1 answer
213 views

Adding a command to break long lines into shorter ones to an ed script

I frequently have long lines in my ed document which I would like to split into separate lines of max length. I found this command which achieves this: fold -s -w80 file Split very long lines I can ...
Edman's user avatar
  • 492
3 votes
2 answers
2k views

Mapfile not removing trailing newline

Sample Data: Tab separated tsv file a.1.58 fadado/CSV https://github.com/fadado/CSV a.1.63 jehiah/json2csv https://github.com/jehiah/json2csv a.1.80 stedolan/jq https://github.com/stedolan/jq/...
Porcupine's user avatar
  • 1,844
3 votes
7 answers
3k views

How to remove \n in a string

I have a tab delimited text file where the 23rd column with string has \n in it which causes it to break to the next line. I opened the text file in vi and enabled white space characters and I can see ...
ntropi's user avatar
  • 37
2 votes
1 answer
3k views

Why am I seeing % as the line end when running cat file cmd [duplicate]

This really confused me, when I do a cat cmd, I always see something like % at the end of a file, but if I open the file on editor or sublime, it is gone. So how to make sure my file is in a correct ...
Tiger's user avatar
  • 347
0 votes
1 answer
383 views

create two new lines using paste command in bash

I'd like to paste three files into a single file with two spaces between each file using the paste command in linux bash: paste -d '\n\n' file1.txt file2.txt file3.txt > output.txt However, there ...
X3nion's user avatar
  • 33
5 votes
2 answers
1k views

Redirect ssh output to file changes line endings

First of all, if I do a simple ssh [email protected] "cat backup.tar" > backup.tar this is not happening! So, there is something else converting it which I don't know. I have the ...
Enak's user avatar
  • 153
0 votes
2 answers
51 views

How to type like this?

I was reading Shell Programming in Unix, Linux and OS X, 4th Edition. In P.112, it showed an example like this $ lines=one' > 'two $ echo "$lines" one two $ lines=ones\ > two $ echo &...
Jacky's user avatar
  • 161
0 votes
5 answers
609 views

How to break sed commands to nested parts in a Bash environment?

I work with a CentOS Bash environment and I have this long one liner sed command which I, as someone who is not a professional sysadmin and works with sed perhaps once in two years, find a bit "...
timesharer's user avatar
2 votes
1 answer
582 views

Rename file with contents of file

So I've downloaded a bunch of text files and want to rename these files to the title that is contained within in those files. The output of head sn-800.txt is: GIBSON RESEARCH CORPORATION ...
Fabby's user avatar
  • 5,374
0 votes
1 answer
101 views

echo $’hi\nhi’ > /etc/list.list not working

When I use echo $’hi\nhi’ > /etc/list.list in an executable script text file I create comprising multiple commands on Kali Linux 2020.4 Live USB it: Includes the $ in the files text while ...
markbunting's user avatar
0 votes
2 answers
312 views

Converting multiline to single line in a file looses double quotes

Here is my sample file cat test.txt "IND WEB", "Speed Web (Internal webserver)", "Web Bill", I tried the below two solutions to convert multiline to single line however ...
Ashar's user avatar
  • 491
13 votes
9 answers
5k views

Concatenate files placing an empty line between them

I have a bunch of files with the same extension (let's say .txt) and I would like to concatenate them. I am using cat *.txt > concat.txt, but I would like to add a new line between each file so to ...
Gigiux's user avatar
  • 507
0 votes
1 answer
2k views

Bash for loop over ls splits line at " (" character

Currently I'm writing a bash script that is supposed to loop over some files in a directory in order of the creation date. To do so I use (with real code instead of the echo): for i in $(ls -tr) do ...
Someone's user avatar
  • 193
10 votes
2 answers
3k views

Why are end-of-file and end-of-input-signal treated differently by sha256sum?

I was trying to compute sha256 for a simple string, namely "abc". I found out that using sha256sum utility like this: sha256sum file_with_string gives results identical to: sha256sum # ...
mdx's user avatar
  • 203
2 votes
0 answers
427 views

Redirecting only whole lines from serial port to netcat

I would like to redirect data from two serial ports over ethernet connection to single port. I've managed to send data, but as serial port data is slow, messages might get corrupted as these does not ...
jaker's user avatar
  • 23
4 votes
2 answers
763 views

how to ask cat (and maybe grep ?) not to take into account a new line when it's inside double quotes?

I have a large csv file that starts this way : codeRegion,nomEPCI,codeDepartement,nomCommune,populationTotale,titre,objet,site_web,libelle_objet_social1,libelle_objet_social2,siret,numero_waldec,...
Marc Le Bihan's user avatar
0 votes
1 answer
13k views

Issue passing newline `\n` character to curl command

My Jenkins multiline parameter is read in a variable SERVER_IP which is passed to Unix curl command as below: The curl command triggers a webpage(Jenkins Build) and passes the parameters to the ...
Ashar's user avatar
  • 491
0 votes
2 answers
1k views

Is a text file bigger on Windows than on Linux because of carriage return?

Let's say I have the same ASCII text file on Linux and Windows: one two three The two files will have \n and \r\n respectively as EOL character on the two OSes. Does this mean that the file on Linux ...
Enlico's user avatar
  • 1,525
12 votes
2 answers
2k views

extra \x0D appended when echo \0x0A

I am trying to send some hex values to a tty port /dev/ttyS2. When I use echo or printf \x0A it will print an extra \x0D before the \x0A. i=1 a=$(printf "\xAA\xEE\x0A\x%02x" $i) echo -ne &...
dmSherazi's user avatar
  • 247
0 votes
3 answers
443 views

How do I print line numbers but reset line counter at empty line?

I have a file.txt containing: this is the first second line not last line fourth but first second in list seventh in file seventh with nl Normally I would just cat and pipe | it into nl like so: $&...
ntruter42's user avatar
  • 187
2 votes
2 answers
338 views

Is a NEWLINE character at line end required by LSB (Linux Standard Base)?

Both POSIX and SUS (Single UNIX Specification) define a line as A sequence of zero or more non-<newline> characters plus a terminating <newline> character. Many distributions are more ...
Mario Nette's user avatar
2 votes
3 answers
285 views

How can I render newline escape sequences in xargs?

The following command prints 3 lines: echo -e 'a\nb\nc' result: a b c How can I achieve the above result when passing from xargs? echo 'a\nb\nc' | xargs -I{} echo -e {} Current result: ...
Marinos An's user avatar
0 votes
2 answers
479 views

how to replace with whitespace?

For reasons explained here. I wanted to replace a period and whitespace (e.g. .\t or .\n) with the period plus a linebreak plus the whitespace (i.e. change the example strings to .\n\t or .\n\n, resp....
tinlyx's user avatar
  • 646
3 votes
2 answers
4k views

How can I write an `eval` command containing a new line into one line?

Since I need to set and use alias in the same line in bash, I would like to use Hauke Laging's workaround: eval 'alias df5=df df5 -h' (I took df and df5 for the sake of the example. I am aware that ...
Franck Dernoncourt's user avatar
1 vote
1 answer
401 views

sed bash script is unexpectedly replacing character with space instead of newline [duplicate]

Using bash command syntax: echo "The*quick*brown*fox"|sed s/*/\\n/g I get the desired output: The quick brown fox However in script: IN_TXT="The*quick*brown*fox" OUT_TXT=$(echo $IN_TXT|sed s/*/\\...
SPB's user avatar
  • 13

1
2 3 4 5
7