5

I have a file that contains locations of different files called locationfile.txt it contains:

/home/user/documents/file1
/home/user/file2
/home/user/file3

Each text files have about 10 lines and I want to output all files to another file called finalfile.

I figured out that if I do

cat locationfile.txt | while read line; do cat "$line"; done

This prints all files together so it becomes a ~30 line text file.

What my question is, how can I spit that file out put include the file locations inbetween the text like this:

/home/user/documents/file1
text
text
text
""
/home/user/file2
text
text
text
""
/home/user/file3
text
text
text
""
8

Before the cat do an echo or printf of the filename:

while read line; do printf '%s\n' "$line"; cat "$line"; done <locationfile.txt >finalfile

Or, more readable:

while read line; do
  printf '%s\n' "$line"
  cat "$line"
done <locationfile.txt >finalfile

Note that this requires all paths in locationfile.txt to have no \ in them.

Although having \ in path names is highly unusual, it would be safer to do

while read -r line; do
  printf '%s\n' "$line"
  cat "$line"
done <locationfile.txt >finalfile

... which doesn't have the same restrictions.

Adding a check to make sure the file actually exist too, and outputting a warning if it doesn't:

while read -r line; do
  if [[ -f "$line" ]]; then
    printf '%s\n' "$line"
    cat "$line"
  else
    printf 'Warning: "%s" does not exist\n' "$line" >&2
  fi
done <locationfile.txt >finalfile

"Does not exist" here means "is at least not a regular file".

If you want the "" there as well, as a sort of end-of-file-contents-marker, just put that out after cat:

while read -r line; do
  if [[ -f "$line" ]]; then
    printf '%s\n' "$line"
    cat "$line"
    echo '""'
  else
    printf 'Warning: "%s" does not exist\n' "$line" >&2
  fi
done <locationfile.txt >finalfile

And finally, give things self-documenting names:

while read -r file_path; do
  if [[ -f "$file_path" ]]; then
    printf '%s\n' "$file_path"
    cat "$file_path"
    echo '""'
  else
    printf 'Warning: "%s" does not exist\n' "$file_path" >&2
  fi
done <file_paths.txt >files_concat_with_paths.txt
  • 1
    while read line works with spaces in file names. And since here is no splitting of the inputl, you don't need to set IFS. Or am I missing something? – xhienne Jan 18 '17 at 16:55
  • @xhienne You're not missing anything, that is correct. Setting IFS to be empty also disables the removal of initial and trailing whitespace in the input, which I usually want to retain, but which may cause problems here. I will amend the answer. – Kusalananda Jan 18 '17 at 17:17
  • @xhienne, pretty sure multiple consecutive spaces will still be mishandled, but perhaps that's only on initial and trailing whitespace. – Wildcard Jan 19 '17 at 4:28
  • @Wildcard You are right, both of you, leading and trailing spaces are not stored in line. But multiple consecutive inner spaces are. And if you don't specify line, the whole line goes into $REPLY, including leading and trailing spaces. – xhienne Jan 19 '17 at 13:16
  • @xhienne ... with IFS=, yes. – Kusalananda Jan 19 '17 at 13:26
3
while read file; do
    ( echo "$file"; cat "$file"; echo '"""' ) >> /path/to/outputfile
done < /path/to/filelist
3

Here's an option if you have GNU awk, that avoids shell loops altogether:

xargs -d '\n' -a locationfile.txt gawk 'BEGINFILE{print FILENAME} 1' > newfile

or with GNU sed:

xargs -d '\n' -a locationfile.txt sed -s '1F' > newfile

If you don't care about the exact format of the filenames, then you can use this trick with head:

xargs -d '\n' -a locationfile.txt head -vn -0 > newfile

(the -n -0 tells head to output all lines except the first 0 i.e. all lines).

3

Another Perl approach:

$ perl -le 'do{print $_,`cat $_`,"\"\"";} for <>' locationfile.txt
text
text
text
""
text
text
text
""
text
text
text
""

That's using a bit of Perl magic and might need an explanation. The -l removes trailing newlines from each input line and adds a newline to each print call. The -e is how you give a script to perl.

Now, <> is "the contents of the input file split into an array". So, do {foo} for <> will run foo on each line of the input file. Inside that for loop, the special variable $_ is assigned to each input line. The `command` syntax will run command in the default shell (a system call) and return its output. So, cat $_ will cat each filename in locationfile.txt.

Altogether, print $_,`cat $_`,"\"\"" will print the file name (currently in $_), the result of the cat (the file's contents) and then "" (but we need to escape those, so \"\").

In other words, that script means "read each input line, print it, cat the file it contains and print a "" after each file.

2

You can just add echo "$line" to your loop:

cat locationfile.txt | while read line; do echo "$line" && cat "$line"; done
  • You could also add && echo '""' after cat to match user's desired output :) – Sergiy Kolodyazhnyy Jan 18 '17 at 18:09
  • This did exactly what I wanted!!! Now I will add in a check file as @Kusalananda said – user211145 Jan 18 '17 at 18:38
2

AWK approach via system() function. We make up the command string via sprintf and pass it on to system()

$ awk '{print $0; cmd=sprintf("cat \"%s\"",$0);system(cmd); print "\"\""}' filenames.txt                                 
/home/xieerqi/input1.txt
Hello, I am input file #1
And this is second line of the text
""
/home/xieerqi/input2.txt
And I am input file #2
Roses are red, violets are blue
AWK is awesome, Python is too
""
/etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.1.1
""

And Perl approach , but with opening file within the code:

$ perl -ne 'chomp; open($fh,$_) or die;print "$_\n";while($line = <$fh>){ print $line; print "\"\"\n" if eof}' filenames>
/home/xieerqi/input1.txt
Hello, I am input file #1
And this is second line of the text
""
/home/xieerqi/input2.txt
And I am input file #2
Roses are red, violets are blue
AWK is awesome, Python is too
""
/etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.1.1
""

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.