The tag has no wiki summary.

learn more… | top users | synonyms

48
votes
7answers
48k views

How to add a newline to the end of a file?

Using version control systems I get annoyed at the noise when the diff says No newline at end of file. So I was wondering: How to add a newline at the end of a file to get rid off those messages?
41
votes
4answers
8k views

What's the point in adding a new line to the end of a file?

Some compilers (especially C or C++ ones) give you warnings about: No new line at end of file I thought this would be a C-programmers-only problem, but github displays a message in the commit view: ...
13
votes
8answers
2k views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
9
votes
2answers
3k views

Finding all “Non-Binary” files

Is it possible to use the find command to find all the "non-binary" files in a directory? Here's the problem I'm trying to solve. I've getting an archive of files from a windows user. This archive ...
7
votes
1answer
220 views

Why do *nix filesystems support file names with newlines? [duplicate]

According to this page: File names in Linux can contain any characters other than (1) a forward slash ( / ), which is reserved for use as the name of the root directory (i.e., the directory that ...
6
votes
5answers
189 views

Remove backslash + newline sequences

We are having a file in Linux which contains one line per record, but problem comes when the line contains some new line characters. In this case, the backslash is appended at the end of line and the ...
6
votes
4answers
185 views

Replace \n by a newline in sed portably

I'm so confused with GNU sed, POSIX sed, and BSD sed's myriad of options. How do I replace literal \n with the newline character using these three sed types?
6
votes
2answers
17k views

Removing all spaces, tabs, newlines, etc from a variable?

This is the error I am getting and it's failing because of a variable whose value is supposed to be 2 (I am getting this using a select * from tabel). I am getting spaces in that variable. + 0 != ...
6
votes
1answer
223 views

shebang line not working with cr-lf

Why are the shebang parts of the following elementary scripts not working: $ cat hello.sh #! /bin/sh echo Hello $ ./hello.sh bash: ./hello.sh: /bin/sh^M: bad interpreter: No such file or directory $ ...
5
votes
5answers
212 views

Automatic transformation of newlines in shell variable assignment

This came out of one of my comments to this question regarding the use of bc in shell scripting. bc puts line breaks in large numbers, e.g.: > num=$(echo 6^6^3 | bc) > echo $num ...
5
votes
4answers
524 views

How to add a carriage return before every newline?

I have a file that only uses \n for new lines, but I need it to have \r\n for each new line. How can I do this? For example, I solved it in Vim using :%s/\n/\r\n/g, but I would like to use a script ...
5
votes
3answers
3k views

Collapsing a stream of multiple lines into one line

I have a file containing a a large number of lines, each of which contains a bunch of numbers which are separated by spaces. I process this data in a pipe in some way, and then I want to collapse the ...
5
votes
1answer
183 views

Cannot cat or grep contents of file

I'm on Debian Jessie. When I open a new textfile with mousepad and save it, then modify it with vim and save it again, I cannot print or grep the contents of it anymore. If I do cat file, nothing is ...
5
votes
1answer
132 views

array[@] output all messed up?

I've got this code: Unix+=("Stock List") while read line; do result=$(wget -O - -o /dev/null "http://finance.yahoo.com/d/quotes.csv?s=$line&f=sl1&e=.csv" | tr ',' ' ' | tr '"' ' ') ...
5
votes
3answers
364 views

Remove trailing new line from 'clipboard' to prevent execution on the terminal?

Is there some way to run whatever you copy to the 'clipboard' through some sort of filter? Ideally to strip out the trailing newline from something you copy, so that it doesn't auto-run in the ...
5
votes
2answers
766 views

Carriage return vs newline in vim

In vim, when I opened a file, I wanted to add two empty lines at the end of each current line. I used the following switch: %s/$/\n\n/ Apparently, this does not work, but what does work is: ...
4
votes
4answers
3k views

concatenate two files without adding a newline

If I nano two files, one of which reads 'this' without me entering a newline, and one of which reads 'is' without me entering a newline, I want to be able to then cat the two files together into ...
4
votes
3answers
251 views

How can I delete a trailing newline in bash?

I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline: $ printf "one\ntwo\n" | ...
4
votes
5answers
115 views

Trying to add a newline to the paste command

Here is the weak attempt at a paste command trying to include a newline: paste -d -s tmp1 tmp2 \n tmp3 \n tmp4 tmp5 tmp6 > tmp7 Basically I have several lines in each tmp and I want the ...
4
votes
2answers
182 views

\n prints an n instead of a newline when using sed in MaxOSX

It's a general problem when I use sed but \n works when I use it outside of sed. This is one example of what I get with sed. I have the file myfile with the following content: 1 2 3 4 5 When I use ...
4
votes
2answers
217 views

read -a array -d '\n' < foo, exit code 1

If I try to execute read -a fooArr -d '\n' < bar the exit code is 1 -- even though it accomplishes what I want it to; put each line of bar in an element of the array fooArr (using bash 4.2.37). ...
4
votes
2answers
10k views

Can sed replace new line characters?

Is there an issue with sed and new line character? I have a file test.txt with the following contents aaaaa bbbbb ccccc ddddd The following does not work: sed -r -i 's/\n/,/g' test.txt ...
4
votes
2answers
1k views

Character count in Unix wc command

When I give wc command on a file with contents given below, it gives the number of characters as 30. Does it include the end of file character? Since including the space and newline there are only 29 ...
4
votes
3answers
124 views

How do I capture stdin to a variable without stripping any trailing newlines?

In a shell script... How do I capture stdin to a variable without stripping any trailing newlines? Right now I have tried: var=`cat` var=`tee` var=$(tee) In all cases $var will not have the ...
4
votes
3answers
546 views

How to skip file in sed if it contains regex?

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed: find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+ As ...
4
votes
2answers
62 views

Verify line endings from a text created by perl script

I am interested in seeing the contents of a text file in hex mode. I have tried :%!xxd and the problem is that I can not really understand which hex code represents each letter e.g. when you use an ...
3
votes
3answers
5k views

Why can't find -regex match a newline?

Why does this fail? touch "$(printf "a\nb")"; find . -regex './.\n.' I also tried these, none of which work: find . -regextype posix-extended -regex '.\n.' find . -regextype posix-awk -regex ...
3
votes
3answers
3k views

Remove ^M character from log files

Remove ^M character from log files. In my script I redirect output of my program to a log file. The output of my log file contains some ^M (newline) characters. I need to remove them while running ...
3
votes
3answers
192 views

Split a returned block of text into lines in terminal?

When I give commands like echo $PATH or which program, I often get a very large block of text consisting of may directories. This is fairly useless, if I am for example, looking to see if there is or ...
3
votes
3answers
389 views

Convert newline-delimited list with spaces into arguments [duplicate]

I'm trying to write a script that I can run from the context menu in Caja (a clone of Nautilus). Its purpose is to pass the files that are selected in the file manager to a given program (for example, ...
3
votes
1answer
202 views

Is the last new line on fstab important?

What happens if the last line of fstab is not terminated by a newline? Why do people get a warning when that last line is not terminated by a newline?
3
votes
1answer
573 views

Does input redirection (<) append a newline character?

I was originally of the impression that $ ./myprog moo[CTRL-D] is exactly the same as $ echo moo > cow $ ./myprog < cow But I found that myprog always counts one more \n in the second ...
3
votes
3answers
2k views

How to bulk convert all the file in a file system branch between Unix and Windows line break format?

Everybody knows :-) that in Windows plain text files lines are terminated with CR+LF, and in Unix&Linux - with LF only. How can I quickly convert all my source code files from one format to ...
2
votes
2answers
580 views

Shell script works when saved with nano but not when saved with Notepad++

I've been working on a bash script all day and have been testing on the machine I will be using it on. When I copy the script across from Notepad++ into a new file with nano editor inside SSH and ...
2
votes
4answers
3k views

How to detect dos format files in git bash

Git Bash is a nice bash shell you get in Windows as part of the installation of Git. It comes with other typical unix tools bundled inside, such as grep, sed, awk, perl. It doesn't have the file ...
2
votes
2answers
514 views

view file containing ISO-8859 text, CRLF line terminators, and escape sequences

I have stumbled upon a strange text file in an archive of an old dos game downloaded from some abandonware site. The text file is named WHY_NOT.TXT. I assume the file is some sort of .NFO file from ...
2
votes
2answers
251 views

Removing consecutive newlines with ed

Input file: 1 line1\ 2 line2\ 3 line3\ 4 \ 5 line4\ 6 \ 7 \ 8 line5\ Desired output: 1 line1 2 line2 3 line3 4 5 line4 6 7 line5 Is it possible with POSIX ed? Removing the trailing backslashes ...
2
votes
1answer
37 views

Why does combining `less` and `script` disrupt line endings?

This bash snippet will print out the numbers 1 to 10: $ for i in $(seq 1 10); do echo $i; done 1 2 3 4 5 6 7 8 9 10 When I try to use this with script and less like so: $ script -q -c 'for i in ...
2
votes
1answer
161 views

Insert newlines with vim

How to insert newlines in vim? For example in a file containing test-classes; classes; xmlunit-1.3.jar I am looking for the vim equivalent of sed command sed -i 's/; /\n/g' classpath to obtain ...
2
votes
2answers
712 views

Why does wc -l tell me that this non-empty file has 0 lines?

xclip/xsel are utilities that allow you to pipe clipboard output to the shell on Ubuntu. Unfortunately, they seem to fail in combination with the wc command. Try this. Open a new file and type: ...
2
votes
2answers
218 views

Directories are listed twice

I have something odd. I creates some directories using a very simple script : ... create all directories ( running under a user that has the correct rights ) e.g : mkdir -p ...
2
votes
1answer
253 views

How to `tee` dropping all CR lines only in the file but not stdout?

When using e.g. rsync, I'd like to tee its output such that I can both view the current progress live, and have a log for later. Using just piping | tee logfile the logfile will contain many lines à ...
2
votes
1answer
58 views

add newlines in .cshrc command output

I'm trying to set up a "dynamic" motd for users when they log in on FreeBSD 10. I've got it set up so that it uses .cshrc and has a few commands to output some info. However one of those prints as a ...
2
votes
5answers
5k views

Carriage return issue

I have this in a bash script DAY2="20130605"<Cr> echo "This is yesterday date:"$DAY2"end" Why is the output the following? It seems as though there is a carriage return in DAY2 but where is ...
2
votes
3answers
332 views

How to read '\n' into variable with Bash's built-in command?

Somehow I'm not able to read the trailing \n sign into the REPLY variable. Under any circumstances I want to avoid a blank line that results from the \n being echoed by read but echo one in case of ...
2
votes
0answers
38 views

How can I make sed not append a newline character? [duplicate]

How can I make sed not append a newline character to an input stream that was missing one? If I can't, what's the quickest way to remove a newline character from a variable? A code sample was ...
1
vote
2answers
188 views

Unable to do head or tail for a file

I have a .CSV file which upon passing the file test_file.csv command gives the output as: test_file.csv: ISO-8859 English text, with CR line terminators When I am using cat, head or tail command on ...
1
vote
2answers
193 views

Why the inconsistency with using cat vs. echo piped to this sed command?

Inspired by this question: sed: N command does not read single line I wanted to understand why there appears to be an inconsistency between the following 2 commands: #1. echo This command produces ...
1
vote
1answer
35 views

Wierd output using paste with serial option

I have a testA.txt with contents as shown below [jiewmeng@JM textFiles]$ cat testA.txt The quick brown fox jumped over the lazy dog. paste normally works [jiewmeng@JM textFiles]$ paste testA.txt ...
1
vote
1answer
605 views

sed: N command does not read single line

Why sed's N command does not read single line? # single line without \n $ echo -en 'abc' | sed -n 'N;p' $ # I expected 'abc' # single line with \n $ echo -en 'abc\n' | sed -n 'N;p' $ # I expected ...