5

I'm trying to prefix and append blocks of text to a list of files. Here is where I am so far. The sticking point is the sed -i "1i \$prefix" "$file" && line. sed won't substitute the value of prefix in. I tried to follow various threads about this on unix.sx and a couple of other places, but just got a headache.

Can someone tell me how to fix this? If it is too complicated I'm Ok with using something else instead of sed.

This is closely related to How do I append text to the beginning and end of multiple text files in Bash?, which is where I got the code below from. The difference is that the question does not cover expanding a variable, and also in my case the strings in question are multiline.

filelist=(foo.tex bar.tex)

prefix='\documentclass[12pt]{article}                                                                                                                                       
\usepackage{myarticle}                                                                                                                                                      
%\xpretocmd{\opening}{\insertname}{}{}                                                                                                                                      
\begin{document}                                                                                                                                                            
%\insertname                                                                                                                                                                
\begin{verbatim}'

suffix='\end{verbatim}                                                                                                                                                      
\end{document}'                                                                                                                                                                 

for file in "${filelist[@]}"; do
  sed -i "1i \$prefix" "$file" &&
  echo "$suffix" >> "$file"
done
4
  • See this: unix.stackexchange.com/a/45208/4873 and delete that backslash in front of $prefix. What is it for?
    – angus
    Jan 24, 2014 at 21:32
  • @angus I'm guessing you have in mind something like for file in "${filelist[@]}"; do sed -i "1i $(echo -E "$prefix" | sed -e 's/\\/\\\\/g')" "$file" && echo "$suffix" >> "$file" done, but this gives an error. Jan 24, 2014 at 22:11
  • OK, I see why you had a backslash. It was part of the sed command; I missed it. You need to double it and add a new line after it: 1i\\ NEWLINE $(echo .... See the linked answer and write your sed line exactly as it is there, only add -i and change 5a by 1i and $text by $prefix. That's all.
    – angus
    Jan 25, 2014 at 1:04
  • I'll write it in an answer, it will be clearer.
    – angus
    Jan 25, 2014 at 1:07

4 Answers 4

2

How about (assuming your shell has process substitution):

for file in "${filelist[@]}"; do
    cat <(printf "%s" "$prefix") "$file" <(printf "%s" "$suffix") > "$file"_$$\ 
     && mv "$file"_$$ "$file"
done

or, better, in Perl (untested):

perl -MTie::File -e '
    @ARGV//=<STDIN>;
    chomp(@ARGV);
    for(@ARGV){
        tie @lines,"Tie::File",$_;
        @lines=($ENV{prefix} @lines $ENV{suffix})
    }' "${filelist[@]}"

For an alternative approach, if the prefix and suffix are fixed, see the following question (which also happens to be about (La)TeX :) ):

3
  • Hi Joseph. Thanks for the suggestions. The last one does indeed seem like a nice simple way to approach the problem. Jan 24, 2014 at 22:25
  • @FaheemMitha Great. Let us know how things work out...
    – Joseph R.
    Jan 24, 2014 at 22:30
  • I found you link to "extractig and copying parts..." more useful.
    – user55518
    Jan 24, 2014 at 22:43
2

With zsh:

zmodload zsh/mapfile
for i ($filelist) mapfile[$i]="$prefix
$mapfile[$i]$suffix
"

With ksh93 or bash (or zsh):

for file in "${filelist[@]}"; do
  {
    rm -- "$file" && {
      printf '%s\n' "$prefix"
      cat
      printf '%s\n' "$suffix"
    } > "$file"
  } < "$file"
done
1

Here are a few choices, none of which use sed. I find that in general, trying to get sed to play nice with external variables is rarely worth the effort.

Shell

for file in "${filelist[@]}"; do 
  tmp=$(mktemp); 
  printf "%s\n%s\n%s\n" "$prefix" $(cat "$file") "$suffix" > "$tmp" && 
  mv "$tmp" "$file"; 
done

Perl/Shell

for file in "${filelist[@]}"; do 
  tmp=$(mktemp); 
  perl -lpe 'BEGIN{print $ENV{prefix}}END{print $ENV{suffix}}' $file 
  mv "$tmp" "$file"; 
done

Pure Perl

This one has the advantage of not relying on external modules:

perl -le 'foreach (@ARGV){
           open($f,"+<","$_"); @a=<$f>; 
           print $f "$ENV{suffix}\n@a$ENV{prefix}"
          }' "${filelist[@]}"

NOTE:

The variables need to be exported in order for the Perl solutions to work, otherwise, they won't be available in the %ENV hash.

1

The case is similar to this: Appending a string containing escape character with sed

I suggest the same answer (adapted):

for file in "${filelist[@]}"; do
  sed -i "1i\\
$(echo -E "$prefix" | sed -e 's/\\/\\\\/g' | sed -e '$! s/$/\\/')" "$file" &&
  echo "$suffix" >> "$file"
done

The sed command i requires each line except the last to be terminated with a backslash (this was missing from the referred answer). Also, backslashes need to be doubled, so sed won't try to interpret them. The echo outputs the contents of the variable $prefix. Next in the pipeline, the sed command doubles the backslashes. The second sed command adds backslashes at the end of every line except the last.

EDIT How the sed program $! s/$/\\/ works:

First, the address $ means to execute the command that follows on the last line. $! negates that, so the command will be executed on all lines except the last.

Then, the s/// command does substitution. It will substitute the empty string at the end of the line, denoted by $, with a backslash. But the backslash is a special character, so we need to escape it with another backslash for sed to read it correctly.

6
  • Hi Angus, thanks for the helpful reply. Unfortunately, the backslashes at the ends of lines appears in the final output. Is it possible to suppress this? Also, could you reference some sed documentation that (a) i requires each line except the last the be terminated with a backslash and (b) that backslashes need to be doubled? Finally if you go into a little more detail on how sed -e '$! s/$/\\/' works, that would be helpful. Jan 25, 2014 at 12:26
  • (a) -> gnu.org/software/sed/manual/html_node/…, (b) -> on the same page, see documentation for a\ . (c) see edited answer.
    – angus
    Jan 25, 2014 at 19:10
  • Hi Angus. Thanks for the references. These could reasonably be in the question, I think. However, I don't see any change in the code (unless I'm missing something). How can one get rid of the extra backslashes? Jan 25, 2014 at 19:34
  • I can't reproduce the problem. There aren't extra backslashes in the output when I run the script exactly as written. Maybe you aren't using GNU sed?
    – angus
    Jan 25, 2014 at 19:49
  • sed --version GNU sed version 4.2.1. What version are you using? In any case, odd. I'll check again in case I made a mistake. Jan 25, 2014 at 19:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .