Linked Questions

268
votes
4answers
210k views

Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
188
votes
4answers
36k views

Why is using a shell loop to process text considered bad practice?

Is using a while loop to process text generally considered bad practice in POSIX shells? As Stéphane Chazelas pointed out, some of the reasons for not using shell loop are conceptual, reliability, ...
23
votes
3answers
8k views

Are files opened by processes loaded into RAM?

Commands, for instance sed, are programs and programs are codified logic inside a file and these files are somewhere on the hard disk. However when commands are being run, a copy of their files from ...
19
votes
7answers
20k views

`tail -f` until text is seen

I've got a CI server with a command-line interface that allows me to remotely kick-off a job (jenkins CI server and the jenkins-cli.jar tool). After I kick the job off I tail -f the log (sorry for ...
12
votes
6answers
2k views

How can I add random string for each line?

I'm trying to add random string for each line while running: awk '{print "name" "'$ran'" "-"$0}' 'myfile' Before that, the random string is generated: ran="$(tr -dc '[:alnum:]' </dev/urandom | ...
9
votes
6answers
7k views

Read a line-oriented file which may not end with a newline

I have a file named /tmp/urlFile where each line represents a url. I am trying to read from the file as follows: cat "/tmp/urlFile" | while read url do echo $url done If the last line doesn't ...
14
votes
3answers
5k views

How to read different lines of a file to different variables?

I would like to read different lines of a text file to different variables. For example input.txt: line1 foo foobar bar line2 bar line3 foo line4 foobar bar I want this result to be stored in ...
9
votes
4answers
2k views

Performance of loop vs expansion

Need expert suggestions on below comparison: Code Segment using loop: for file in `cat large_file_list` do gzip -d $file done Code segment using simple expansion: gzip -d `cat ...
4
votes
7answers
8k views

How to compare 2 strings in UNIX shell script? [duplicate]

I have a variable which stores a string, the output of a sed command. I want to execute a set of commands only if this string value matches either of the 2 other strings. I used the below code. #! /...
7
votes
3answers
2k views

What does “${line#*'Caused By'}” != “$line” mean in a shell script?

Can someone explain what this means in a shell script? while read -r line do if [ "${line#*'Caused By'}" != "$line" ]; then echo "Yes" fi done
16
votes
1answer
986 views

What is the meaning of “<&3” and “done < file11 3< file22”

I have the following script: while read lineA && read lineB <&3 do echo "$lineA" echo "$lineB" echo done < file11 3< file22 This code is working fine. But I don't ...
1
vote
4answers
2k views

Unix bash/ksh : Selection of first non space character from file from specific line

I have file file1.txt whose contents are as follows: Date List ----------- Quarter Date Year Date Month Date Now I want to read the non space elements from each row of file ...
7
votes
2answers
3k views

A while loop and an here-document - what happens when?

I have this while loop and here-document combo which I run in Bash 4.3.48(1) and I don't understand its logic at all. while read file; do source ~/unwe/"$file" done <<-EOF x.sh y.sh EOF ...
4
votes
5answers
7k views

Bash while loop read from colon-delimited list of paths using IFS

I am trying to write a bash function that behaves similarly to the where builtin in tcsh. In tcsh, where lists all the builtins, aliases, and the absolute paths to executables on the PATH with a given ...
5
votes
3answers
1k views

“echo $IFS” does not return the value of IFS

I have set the IFS to x, i.e IFS=x. Now if I check the value of IFS, then it appears to be empty if I do not use double-quotes: ~ $ echo $IFS | cat -e $ ~ $ echo "$IFS" | cat -e x$ ~ $ echo $HOME /...

15 30 50 per page