Skip to main content

Questions tagged [newlines]

Use for questions about representation and manipulation of the end-of-line marker in text files, in particular when editing the same file on different operating systems where the representation may vary.

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Why do newline characters get lost when concatenating strings ending with newline characters? [duplicate]

Scenario: $ cat xxx.txt | od -c 0000000 x \n x \n 0000004 $ value="" # 1st iteration $ value+="$(<xxx.txt)" ; echo "$value" | od -c 0000000 x \n x \n ...
pmor's user avatar
  • 619
0 votes
1 answer
52 views

Redirect command output to file (append) and then append a newline character

I'm running the following one-liner: cat input.txt | xargs -I{} bash -c './script.sh "{}"' >> output.txt The script works and does exactly what I want it to do. The only issue is that ...
dlyk1988's user avatar
  • 103
0 votes
3 answers
734 views

How to convert CR to CRLF line endings on a file?

An ASCII text file, with CR line terminators, (Mac style) needs to be converted to have CRLF line terminators (DOS/Windows style). Search for CR not followed by LF, and replace that CR with a CRLF. So ...
Pro Backup's user avatar
  • 5,014
0 votes
2 answers
199 views

How to add an empty line after lines starting with a timestamp

The log viewing experience is so bad in my current company. I want to add new line to the logs that I've trimmed for a specific time duration. I want a new line after every date-time. The date-time ...
achhainsan's user avatar
0 votes
1 answer
249 views

"man cat" mentions "LFD." What is "LFD?"

man cat says: -v, --show-nonprinting use ^ and M- notation, except for LFD and TAB I checked info cat, and it uses the same acronym: ‘-v’ ‘--show-nonprinting’ Display ...
Daniel Kaplan's user avatar
2 votes
3 answers
4k views

How to deal with CRLF, CR line endings

I have a Perl script that parses data sent to me from a bunch of school districts. I'm adding a new school and have run into a problem I've never faced before. When I do $line = <INPUT>, it ...
Chanel's user avatar
  • 99
2 votes
5 answers
236 views

How to remove newline characters inside paragraphs with sed or awk

I was wanting to know how to remove newline characters in paragraphs in this book and others for use in a kindle. The desired effect is to have each block that is separated by a blank line to be ...
Seth Buchanan's user avatar
10 votes
3 answers
2k views

Add a CR before every LF

I have a text file being processed on a Windows machine. There is a need to remove trailing tab characters prior to using bcp utility to load the data from the file into a database table. The ...
knot22's user avatar
  • 271
0 votes
1 answer
47 views

maintain appended logs in separate file despite tail command termination

I have tomcat catalina.log file that keeps getting updated. I wish to maintain its past 500 lines + any appended logs into a new file catalina.log_new for the next 2 hours. To start off I run nohup ...
Ashar's user avatar
  • 511
0 votes
1 answer
73 views

How to find and manipulate files that contain newline in filename?

follow up on How to find files that contain newline in filename? , I need to do some manipulation on the result, for simplicity lets suppose I need to chown them. Tried following, but does not work: # ...
Nick's user avatar
  • 125
-1 votes
1 answer
305 views

Newline character '\n' (return key) in *nix!

While studying Linux, I noticed that the newline character is used not only to separate different lines - that is to set the beginning and the end of a line - but also is used to give some input to ...
Angelo Giannuzzi's user avatar
0 votes
1 answer
136 views

grep searching end of line pattern from DOS files

Like Detecting pattern at the end of a line with grep, but for DOS files of CRLF line terminators. Like in https://unix.stackexchange.com/a/462633/374303, one way is to use dos2unix but I don't have ...
xpt's user avatar
  • 1,644
2 votes
2 answers
296 views

Do executable shebang scripts always require LF line endings?

On Stack Overflow a comment to Can PowerShell Core handle ps1 files with CRLF line endings in Linux environments? asserted that executable files that use a shebang must use only the LF line ending, ...
Garret Wilson's user avatar
0 votes
1 answer
108 views

GNU sed: append both BEFORE name start and AFTER name end (output is: "$ ls -xN")

Please could someone check if my GNU sed script is the nice way to do it or if it's some sort of bad / wrong hack :p $ ls -xN 1 2 3 4 $ ls -xN | sed -E "s/(.{1})$/\1\.log/" | sed -E "s/...
T. Caio's user avatar
  • 129
0 votes
1 answer
231 views

MacOS append char at end of line with sed

I'm on MacOS where I want to enclose field names by "" in the header. Reading the file with sed, I use this command sed -i.bkp -e '1s/,/","/g' -e '1s/.*/"&/g' -e '1s/.*/&...
user538064's user avatar
1 vote
1 answer
757 views

How do I search for a string and then insert a newline at the beginning, for all lines in file?

I want to search an entire file for the string --- when it appears at the beginning of a line. I then want to insert a newline before each instance of that string. In my .vimrc file, I added this ...
Bastette's user avatar
1 vote
2 answers
725 views

How to prevent grep --color from botching the output when a carriage return from Windows line ends is matched?

I have a file with some lines likes x y and when I run grep -E "x$" filename.txt it does not match anything. As vi shows ^M in the end of most lines, I guess the problem is mixed Dos and ...
allo's user avatar
  • 946
0 votes
1 answer
204 views

How to make Zsh interpret \n in parameter expansion

I’m using Zsh’s parameter expansion flags to split a string into an array. This works: string="one two" array=("${(f)string}") echo "${array[1]}" # Returns: # one This ...
user137369's user avatar
2 votes
3 answers
3k views

Stuck at removing a trailing newline from a string variable

Sometimes simplest things take your time in shell scripting like crazy. Content='me \n you \n him \n' echo $Content > Names.txt When I open the Names.txt it has two empty lines at the end. I want ...
Saeed Neamati's user avatar
31 votes
2 answers
4k views

When using cat to create file, I have to type ctrl+d twice to finish the input, is this expected? [duplicate]

I just learned a trick to create a new file with the cat command. By my testing, if the last line is not followed by a newline, I have to type ctrl+d twice to finish the input, as demonstrated below. ...
Fajela Tajkiya's user avatar
-2 votes
2 answers
682 views

Why does echo in Bash modify the content of a independent variable?

I have the following code (minimal example). #!/bin/bash ...
kaiya's user avatar
  • 222
4 votes
7 answers
567 views

awk: deal with newline separation in wrong place

I have some data that looks like this: abc 123 456 789 def 111 222 333 ghi 999 888 777 666 i.e. the records are separated by multiple newlines but in the wrong place. What I want is to get it like ...
dain's user avatar
  • 145
1 vote
0 answers
60 views

Bash prompt with a 'non' normal character not wrapping new line with long commands

So, when typing in long commands, on most of the terminals I use except for UXTERM, the bash prompt will overwrite the text on the same line and won't go to a new line. This is my bash prompt in the ...
a e's user avatar
  • 31
1 vote
2 answers
765 views

sed replacing a string with itself and a new line

I have the following string multiple times in a html file <br> <h4 id=... Now, I know sed might not be perfect for html editing, but could tell me how can I replace br> <h4 id with br&...
radui's user avatar
  • 111
1 vote
1 answer
698 views

When assigning a command output to a variable, the new lines are replaced by white spaces [duplicate]

I have a file and the idea is to grep the content following a pattern ("create mode") and to store this content in a bash variable in order to use it then in a curl statement. I have the ...
nicg's user avatar
  • 81
2 votes
1 answer
123 views

Can ZFS snapshot names contain newlines? If yes, how to parse the output of zfs list -t snapshot?

The following question relates to ZFS on Linux (ZoL) / OpenZFS, as it is provided e.g. in Debian Buster and Debian Bullseye. As the title says, I'd like to know whether ZFS snapshot names ...
Binarus's user avatar
  • 3,420
0 votes
2 answers
235 views

Use a modifier with find command's regex?

I'm trying to use the find command's regex to match newlines using the dot character. This usually is accomplished via the s modifier; however, different regex engines have different formats and I ...
Harry Muscle's user avatar
  • 2,547
2 votes
1 answer
228 views

Adding a command to break long lines into shorter ones to an ed script

I frequently have long lines in my ed document which I would like to split into separate lines of max length. I found this command which achieves this: fold -s -w80 file Split very long lines I can ...
Edman's user avatar
  • 512
3 votes
2 answers
2k views

Mapfile not removing trailing newline

Sample Data: Tab separated tsv file a.1.58 fadado/CSV https://github.com/fadado/CSV a.1.63 jehiah/json2csv https://github.com/jehiah/json2csv a.1.80 stedolan/jq https://github.com/stedolan/jq/...
Porcupine's user avatar
  • 2,056
3 votes
7 answers
3k views

How to remove \n in a string

I have a tab delimited text file where the 23rd column with string has \n in it which causes it to break to the next line. I opened the text file in vi and enabled white space characters and I can see ...
ntropi's user avatar
  • 37
2 votes
1 answer
3k views

Why am I seeing % as the line end when running cat file cmd [duplicate]

This really confused me, when I do a cat cmd, I always see something like % at the end of a file, but if I open the file on editor or sublime, it is gone. So how to make sure my file is in a correct ...
Tiger's user avatar
  • 347
0 votes
1 answer
410 views

create two new lines using paste command in bash

I'd like to paste three files into a single file with two spaces between each file using the paste command in linux bash: paste -d '\n\n' file1.txt file2.txt file3.txt > output.txt However, there ...
X3nion's user avatar
  • 33
5 votes
2 answers
2k views

Redirect ssh output to file changes line endings

First of all, if I do a simple ssh [email protected] "cat backup.tar" > backup.tar this is not happening! So, there is something else converting it which I don't know. I have the ...
Enak's user avatar
  • 153
0 votes
2 answers
52 views

How to type like this?

I was reading Shell Programming in Unix, Linux and OS X, 4th Edition. In P.112, it showed an example like this $ lines=one' > 'two $ echo "$lines" one two $ lines=ones\ > two $ echo &...
Jacky's user avatar
  • 161
0 votes
5 answers
663 views

How to break sed commands to nested parts in a Bash environment?

I work with a CentOS Bash environment and I have this long one liner sed command which I, as someone who is not a professional sysadmin and works with sed perhaps once in two years, find a bit "...
timesharer's user avatar
2 votes
1 answer
605 views

Rename file with contents of file

So I've downloaded a bunch of text files and want to rename these files to the title that is contained within in those files. The output of head sn-800.txt is: GIBSON RESEARCH CORPORATION ...
Fabby's user avatar
  • 5,404
0 votes
1 answer
105 views

echo $’hi\nhi’ > /etc/list.list not working

When I use echo $’hi\nhi’ > /etc/list.list in an executable script text file I create comprising multiple commands on Kali Linux 2020.4 Live USB it: Includes the $ in the files text while ...
markbunting's user avatar
0 votes
2 answers
359 views

Converting multiline to single line in a file looses double quotes

Here is my sample file cat test.txt "IND WEB", "Speed Web (Internal webserver)", "Web Bill", I tried the below two solutions to convert multiline to single line however ...
Ashar's user avatar
  • 511
13 votes
9 answers
5k views

Concatenate files placing an empty line between them

I have a bunch of files with the same extension (let's say .txt) and I would like to concatenate them. I am using cat *.txt > concat.txt, but I would like to add a new line between each file so to ...
Gigiux's user avatar
  • 527
0 votes
1 answer
3k views

Bash for loop over ls splits line at " (" character

Currently I'm writing a bash script that is supposed to loop over some files in a directory in order of the creation date. To do so I use (with real code instead of the echo): for i in $(ls -tr) do ...
Someone's user avatar
  • 193
10 votes
2 answers
3k views

Why are end-of-file and end-of-input-signal treated differently by sha256sum?

I was trying to compute sha256 for a simple string, namely "abc". I found out that using sha256sum utility like this: sha256sum file_with_string gives results identical to: sha256sum # ...
mdx's user avatar
  • 203
2 votes
0 answers
459 views

Redirecting only whole lines from serial port to netcat

I would like to redirect data from two serial ports over ethernet connection to single port. I've managed to send data, but as serial port data is slow, messages might get corrupted as these does not ...
jaker's user avatar
  • 23
4 votes
2 answers
767 views

how to ask cat (and maybe grep ?) not to take into account a new line when it's inside double quotes?

I have a large csv file that starts this way : codeRegion,nomEPCI,codeDepartement,nomCommune,populationTotale,titre,objet,site_web,libelle_objet_social1,libelle_objet_social2,siret,numero_waldec,...
Marc Le Bihan's user avatar
0 votes
1 answer
14k views

Issue passing newline `\n` character to curl command

My Jenkins multiline parameter is read in a variable SERVER_IP which is passed to Unix curl command as below: The curl command triggers a webpage(Jenkins Build) and passes the parameters to the ...
Ashar's user avatar
  • 511
0 votes
2 answers
1k views

Is a text file bigger on Windows than on Linux because of carriage return?

Let's say I have the same ASCII text file on Linux and Windows: one two three The two files will have \n and \r\n respectively as EOL character on the two OSes. Does this mean that the file on Linux ...
Enlico's user avatar
  • 1,677
12 votes
2 answers
2k views

extra \x0D appended when echo \0x0A

I am trying to send some hex values to a tty port /dev/ttyS2. When I use echo or printf \x0A it will print an extra \x0D before the \x0A. i=1 a=$(printf "\xAA\xEE\x0A\x%02x" $i) echo -ne &...
dmSherazi's user avatar
  • 247
0 votes
3 answers
461 views

How do I print line numbers but reset line counter at empty line?

I have a file.txt containing: this is the first second line not last line fourth but first second in list seventh in file seventh with nl Normally I would just cat and pipe | it into nl like so: $&...
ntruter42's user avatar
  • 187
2 votes
2 answers
359 views

Is a NEWLINE character at line end required by LSB (Linux Standard Base)?

Both POSIX and SUS (Single UNIX Specification) define a line as A sequence of zero or more non-<newline> characters plus a terminating <newline> character. Many distributions are more ...
Mario Nette's user avatar
2 votes
3 answers
301 views

How can I render newline escape sequences in xargs?

The following command prints 3 lines: echo -e 'a\nb\nc' result: a b c How can I achieve the above result when passing from xargs? echo 'a\nb\nc' | xargs -I{} echo -e {} Current result: ...
Marinos An's user avatar
0 votes
2 answers
489 views

how to replace with whitespace?

For reasons explained here. I wanted to replace a period and whitespace (e.g. .\t or .\n) with the period plus a linebreak plus the whitespace (i.e. change the example strings to .\n\t or .\n\n, resp....
tinlyx's user avatar
  • 830

1
2 3 4 5
7