The tag has no usage guidance.

learn more… | top users | synonyms

89
votes
14answers
103k views

How to add a newline to the end of a file?

Using version control systems I get annoyed at the noise when the diff says No newline at end of file. So I was wondering: How to add a newline at the end of a file to get rid of those messages?
72
votes
5answers
25k views

What's the point in adding a new line to the end of a file?

Some compilers (especially C or C++ ones) give you warnings about: No new line at end of file I thought this would be a C-programmers-only problem, but github displays a message in the commit view: ...
19
votes
6answers
13k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
17
votes
7answers
26k views

How to test whether a file uses CRLF or LF without modifying it?

I need to periodically run a command that ensures that some text files are kept in Linux mode. Unfortunately dos2unix always modifies the file, which would mess file's and folder's timestamps and ...
15
votes
3answers
11k views

Finding all “Non-Binary” files

Is it possible to use the find command to find all the "non-binary" files in a directory? Here's the problem I'm trying to solve. I've received an archive of files from a windows user. This archive ...
14
votes
2answers
2k views

Why doesn't the Enter key send EOL?

Unix / Linux EOL is LF, linefeed, ASCII 10, escape sequence \n. Here's a Python snippet to get exactly one keypress: import sys, tty, termios fd = sys.stdin.fileno() old_settings = ...
14
votes
8answers
7k views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
10
votes
5answers
15k views

How to put a newline special character into a file using the echo command and redirection operator?

I would like to create a file by using the echo command and the redirection operator, the file should be made of a few lines. I tried to include a newline by "\n" inside the string: echo "first ...
10
votes
6answers
42k views

Can sed replace new line characters?

Is there an issue with sed and new line character? I have a file test.txt with the following contents aaaaa bbbbb ccccc ddddd The following does not work: sed -r -i 's/\n/,/g' test.txt ...
10
votes
1answer
2k views

Why ZSH ends a line with a highlighted percent symbol?

I've noticed this on occasion with a variety of applications. I've often thought it was because the output was cancelled early (ctrl+c, for example) or something similar, and zsh is filling in a new ...
10
votes
2answers
36k views

Removing all spaces, tabs, newlines, etc from a variable?

This is the error I am getting and it's failing because of a variable whose value is supposed to be 2 (I am getting this using a select * from tabel). I am getting spaces in that variable. + 0 != ...
8
votes
6answers
508 views

Force newlines with cat wildcard printing

I want to use cat with wildcards in bash in order to print multiple small files (every file is one sentence) to standard output. However, the separate file contents are not separated by a newline, ...
8
votes
1answer
18k views

No such file or directory but I can see it!

I'm trying to run a python script, on a headless Raspberry PI using winSCP and get the following error message: Command '"./areadetect_movie_21.py"' failed with return code 127 and error message ...
8
votes
4answers
32k views

How to add a carriage return before every newline?

I have a file that only uses \n for new lines, but I need it to have \r\n for each new line. How can I do this? For example, I solved it in Vim using :%s/\n/\r\n/g, but I would like to use a script ...
7
votes
4answers
4k views

Replace a string containing newline characters

With the bash shell, in a file with rows like the following ones first "line" <second>line and so on I would like to replace one or more occurrences of "line"\n<second> with other ...
7
votes
4answers
10k views

Replace \n by a newline in sed portably

I'm so confused with GNU sed, POSIX sed, and BSD sed's myriad of options. How do I replace literal \n with the newline character using these three sed types?
7
votes
2answers
893 views

read -a array -d '\n' < foo, exit code 1

If I try to execute read -a fooArr -d '\n' < bar the exit code is 1 -- even though it accomplishes what I want it to; put each line of bar in an element of the array fooArr (using bash 4.2.37). ...
7
votes
1answer
802 views

shebang line not working with cr-lf

Why are the shebang parts of the following elementary scripts not working: $ cat hello.sh #! /bin/sh echo Hello $ ./hello.sh bash: ./hello.sh: /bin/sh^M: bad interpreter: No such file or directory $ ...
7
votes
3answers
1k views

Remove trailing new line from 'clipboard' to prevent execution on the terminal?

Is there some way to run whatever you copy to the 'clipboard' through some sort of filter? Ideally to strip out the trailing newline from something you copy, so that it doesn't auto-run in the ...
7
votes
1answer
443 views

Why do *nix filesystems support file names with newlines? [duplicate]

According to this page: File names in Linux can contain any characters other than (1) a forward slash ( / ), which is reserved for use as the name of the root directory (i.e., the directory that ...
6
votes
4answers
1k views

Add a newline into a filename with `mv`

It's a serious question. I test some awk scripts and I need files with a newline in their names. Is it possible to add a newline into a filename with mv? I now, I can do this with touch: touch ...
6
votes
2answers
2k views

sed: read whole file into pattern space without failing on single-line input

Reading a whole file into pattern space is useful for substituting newlines, &c. and there are many instances advising the following: sed ':a;N;$!ba; [commands...]' However, it fails if the ...
6
votes
4answers
2k views

How can I delete a trailing newline in bash?

I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline: $ printf "one\ntwo\n" | ...
6
votes
5answers
8k views

Why do newline characters get lost when using command substitution?

I have a text file named links.txt which looks like this link1 link2 link3 I want to loop through this file line by line and perform an operation on every line. I know I can do this using while ...
6
votes
5answers
1k views

Remove backslash + newline sequences

We are having a file in Linux which contains one line per record, but problem comes when the line contains some new line characters. In this case, the backslash is appended at the end of line and the ...
6
votes
2answers
3k views

How to search for text in a file ignoring newlines?

I'd like to search for text that may be split over several lines in a file. A grep that would ignore line breaks and return the matching span of lines. e.g. I would be searching for is an example ...
6
votes
1answer
59 views

How can I make `file` output line break type and encoding for all file types?

I often find myself in the need to check the encoding and the line break type of a file. The file utility seems to be a perfect fit. Unfortunately it doesn't output the desired information for all ...
5
votes
5answers
2k views

How to replace a word with new line

I have a text file with following data and each row ends with |END|. T|somthing|something|END|T|something2|something2|END| I am tryig to replace |END| with \n new line with sed. sed ...
5
votes
4answers
6k views

concatenate two files without adding a newline

If I nano two files, one of which reads 'this' without me entering a newline, and one of which reads 'is' without me entering a newline, I want to be able to then cat the two files together into ...
5
votes
5answers
344 views

Automatic transformation of newlines in shell variable assignment

This came out of one of my comments to this question regarding the use of bc in shell scripting. bc puts line breaks in large numbers, e.g.: > num=$(echo 6^6^3 | bc) > echo $num ...
5
votes
3answers
720 views

How can I replace a newline with sed?

I've got text with whitespace (including a newline) that I need to replace. I'd like to use sed, but AIX doesn't seem to support the answers to sed newline questions I found. The version of bash I'm ...
5
votes
3answers
5k views

Collapsing a stream of multiple lines into one line

I have a file containing a a large number of lines, each of which contains a bunch of numbers which are separated by spaces. I process this data in a pipe in some way, and then I want to collapse the ...
5
votes
2answers
121 views

How to insert newlines into PATTERN when using fgrep / grep -F / grep --fixed-strings

When using fgrep, the man page says it will Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched. In bash, what's the correct way to insert ...
5
votes
3answers
952 views

How do I capture stdin to a variable without stripping any trailing newlines?

In a shell script... How do I capture stdin to a variable without stripping any trailing newlines? Right now I have tried: var=`cat` var=`tee` var=$(tee) In all cases $var will not have the ...
5
votes
1answer
484 views

Cannot cat or grep contents of file

I'm on Debian Jessie. When I open a new textfile with mousepad and save it, then modify it with vim and save it again, I cannot print or grep the contents of it anymore. If I do cat file, nothing is ...
5
votes
1answer
165 views

array[@] output all messed up?

I've got this code: Unix+=("Stock List") while read line; do result=$(wget -O - -o /dev/null "http://finance.yahoo.com/d/quotes.csv?s=$line&f=sl1&e=.csv" | tr ',' ' ' | tr '"' ' ') ...
5
votes
2answers
5k views

Carriage return vs newline in vim

In vim, when I opened a file, I wanted to add two empty lines at the end of each current line. I used the following switch: %s/$/\n\n/ Apparently, this does not work, but what does work is: ...
5
votes
2answers
204 views

Elegant way to prevent command substitution from removing trailing newline

I'm customizing my zsh PROMPT and calling a function that may or may not echo a string based on the state of an environment variable: function my_info { [[ -n "$ENV_VAR"]] && echo "Some ...
4
votes
3answers
21k views

Why can't find -regex match a newline?

Why does this fail? touch "$(printf "a\nb")"; find . -regex './.\n.' I also tried these, none of which work: find . -regextype posix-extended -regex '.\n.' find . -regextype posix-awk -regex ...
4
votes
3answers
2k views

sed fails to remove newline character

I've been using sed for quite some time but here is a quirk I came around with, which I am not able to resolve. Let me explain my problem with the actual case. Scene#1 printf "ls" | xclip ...
4
votes
5answers
751 views

Trying to add a newline to the paste command

Here is the weak attempt at a paste command trying to include a newline: paste -d -s tmp1 tmp2 \n tmp3 \n tmp4 tmp5 tmp6 > tmp7 Basically I have several lines in each tmp and I want the ...
4
votes
2answers
355 views

\n prints an n instead of a newline when using sed in MaxOSX

It's a general problem when I use sed but \n works when I use it outside of sed. This is one example of what I get with sed. I have the file myfile with the following content: 1 2 3 4 5 When I use ...
4
votes
2answers
3k views

Character count in Unix wc command

When I give wc command on a file with contents given below, it gives the number of characters as 30. Does it include the end of file character? Since including the space and newline there are only 29 ...
4
votes
3answers
6k views

How to bulk convert all the file in a file system branch between Unix and Windows line break format?

Everybody knows :-) that in Windows plain text files lines are terminated with CR+LF, and in Unix&Linux - with LF only. How can I quickly convert all my source code files from one format to ...
4
votes
1answer
151 views

bash prevent printf from interrupting another printf

I have the following test script. #!/bin/bash function foo { printf "Test line break: $1\nafter line break\n\n" } for VARIABLE in {1..30} do foo $VARIABLE & done wait Now I executed the ...
4
votes
1answer
129 views

Gedit displays text, but cat tells a different story

I am trying to sort a dictionary from Grady Ward's Moby and have encountered a bit of a problem. When I open it in gedit, I get: abaca×N abaciscus×N abacist×N aback×v abacli×p Abaco×N abactinal×A ...
4
votes
3answers
1k views

How to skip file in sed if it contains regex?

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed: find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+ As ...
4
votes
2answers
103 views

Verify line endings from a text created by perl script

I am interested in seeing the contents of a text file in hex mode. I have tried :%!xxd and the problem is that I can not really understand which hex code represents each letter e.g. when you use an ...
3
votes
4answers
16k views

Remove ^M character from log files

Remove ^M character from log files. In my script I redirect output of my program to a log file. The output of my log file contains some ^M (newline) characters. I need to remove them while running ...
3
votes
2answers
2k views

Shell script works when saved with nano but not when saved with Notepad++

I've been working on a bash script all day and have been testing on the machine I will be using it on. When I copy the script across from Notepad++ into a new file with nano editor inside SSH and ...