Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Replacing strings in files based on certain search criteria is a very common task. How can I

  • replace string foo with bar in all files in the current directory?
  • do the same recursively for sub directories?
  • replace only if the file name matches another string?
  • replace only if the string is found in a certain context?
  • replace if the string is on a certain line number?
  • replace multiple strings with the same replacement
  • replace multiple strings with different replacements
share|improve this question
1  
This is intended to be a canonical Q&A on this subject (see this meta discussion), please feel free to edit my answer below or add your own. –  terdon Feb 1 '14 at 17:08
    
I just want to replace foo with bar in a specific line. all the lines contains foo's.what to do? –  user2433165 Sep 4 '14 at 11:38
    
@user2433165 look at the last bullet point of solution number 3 in my answer below. Just change 4 to the line number you want. –  terdon Sep 4 '14 at 14:05
    
@mikeserv I don't see why not. My answer is basically sed, awk and perl all of which are available on the vast majority of systems. Note that it's a community wiki answer and is so precisely to encourage people to edit and improve it. If you have more portable solutions, please add them. –  terdon Sep 4 '14 at 18:23
    
@mikeserv cool. Please post an answer, or edit mine, giving alternate solutions then. That's the point of the canonical Q&As. Not that there necessarily be a single all inclusive answer, but that it be a single all inclusive thread. Oh, and -i is not GNU, it is also BSD and maybe more. It's just not POSIX. –  terdon Sep 4 '14 at 18:54

2 Answers 2

up vote 118 down vote accepted

1. Replacing all occurrences of one string with another in all files in the current directory:

These are for cases where you know that the directory contains only regular files and that you want to process all non-hidden files. If that is not the case, use the approaches in 2.

All sed solutions in this answer assume GNU sed. If using FreeBSD or OS/X, replace -i with -i ''. Also note that the use of the -i switch with any version of sed has certain filesystem security implications and is unadvisable in any script which you plan to distribute in any way.

  • Non recursive, files in this directory only:

    sed -i -- 's/foo/bar/g' *
    perl -Ti -pe 's/foo/bar/g' ./* 
    

    (the perl one will fail for file names ending in | or space)).

  • Recursive, regular files (including hidden ones) in this and all subdirectories

    find . -type f -exec sed -i 's/foo/bar/g' {} +
    

    If you are using zsh:

    sed -i -- 's/foo/bar/g' **/*(D.)
    

    (may fail if the list is too big, see zargs to work around).

    If you are using bash, bash having no support for glob qualifiers, you can't check for regular files:

    shopt -s globstar
    shopt -s dotglob
    

    Then:

    sed -i -- 's/foo/bar/g' **/*
    

2. Replace only if the file name matches another string / has a specific extension / is of a certain type etc:

  • Non-recursive, files in this directory only:

    sed -i -- 's/foo/bar/g' *baz*    ## all files whose name contains baz
    sed -i -- 's/foo/bar/g' *.baz    ## files ending in .baz
    
  • Recursive, regular files in this and all subdirectories

    find . -type f -name "*baz*" -exec sed -i 's/foo/bar/g' {} +
    

    If you are using bash:

    shopt -s globstar
    shopt -s dotglob
    

    Then:

    sed -i -- 's/foo/bar/g' **/*baz*
    sed -i -- 's/foo/bar/g' **/*.baz
    

    If you are using zsh:

    sed -i -- 's/foo/bar/g' **/*baz*(D.)
    sed -i -- 's/foo/bar/g' **/*.baz(D.)
    

    The -- serves to tell sed that no more flags will be given in the command line. This is useful to protect against file names starting with -.

  • If a file is of a certain type, for example, executable (see man find for more options):

    find . -type f -executable -exec sed -i 's/foo/bar/g' {} +
    

    zsh:

    sed -i -- 's/foo/bar/g' **/*(D*)
    

3. Replace only if the string is found in a certain context

  • Replace foo with bar only there is a baz later on the same line:

    sed -i -e:1 -e's/foo\(.*baz\)/bar\1/;t1' file
    

    In sed, using \( \) saves whatever is in the parentheses and you can then access it with \1. There are many variations of this theme, to learn more about such regular expressions, see here. We need to repeat the operation for all foo occurrences, which is done with the t conditional branching.

  • The same thing might be done in a single global replacement, without resorting to test loops, with a little preparation:

    sed 'y/ /\n/;s/foo/& /g
         s/... \([^ ]*baz\)/bar\1/g
         s/ //g;y/\n/ /' <in >out
    

    That temporarily displaces all <spaces> on a line \newlines (which are guaranteed not to occur in pattern-space otherwise) so that the <spaces> can be used as delimiters for the foo...baz split fields and later restores the spaces when through.

  • Replace foo with bar only if foo is found on the 3d column (field) of the input file (assuming whitespace-separated fields):

    gawk -i inplace 'gsub(/foo/,"baz",$3)' file
    

    (need gawk 4.1.0 or newer).

  • For a different field just use $N where N is the number of the field of interest. For a different field separator (: in this example) use:

    gawk -i inplace -F':' 'gsub(/foo/,"baz",$3)' file
    

    Another solution using perl:

    perl -i -ane '$F[2]=~s/foo/baz/g; $" = " "; print "@F\n"' foo 
    

    NOTE: both the awk and perl solutions will print space separated fields even if the input file had tabs. For a different field use $F[N-1] where N is the field umber you want and for a different field separator use (the $"=":" sets the output field separator to :):

    perl -i -F':' -ane '$F[2]=~s/foo/baz/g; $"=":";print "@F"' foo 
    
  • Replace foo with bar only on the 4th line:

    sed -i '4s/foo/bar/g' file
    gawk -i inplace 'NR==4{gsub(/foo/,"baz")};1' file
    perl -i -pe 's/foo/bar/g if $.==4' file
    

4. Multiple replace operations: replace with different strings

  • You can combine sed commands:

    sed -i 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
    
  • or Perl commands

    perl -i -pe 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
    
  • If you have a large number of patterns, it is easier to save your patterns and their replacements in a sed script file:

    #! /usr/bin/sed -i
    s/foo/bar/g
    s/baz/zab/g
    
  • Or, if you have too many pattern pairs for the above to be feasible, you can read pattern pairs from a file (two space separated patterns, $pattern and $replacement, per line):

    while read -r pattern replacement; do   
        sed -i "s/$pattern/$replacement/" file
    done < patterns.txt
    
  • That will be quite slow for long lists of patterns and large data files so you might want to read the patterns and create a sed script from them instead. The following assumes a <space> delimiter separates a list of MATCH<space>REPLACE pairs occurring one-per-line in the file patterns.txt :

    sed 's| *\([^ ]*\) *\([^ ]*\).*|s/\1/\2/g|' <patterns.txt |
    sed -f- ./editfile >outfile
    

    The above format is largely arbitrary and, for example, doesn't allow for a <space> in either of MATCH or REPLACE. The method is very general though: basically, if you can create an output stream which looks like a sed script, then you can source that stream as a sed script by specifying sed's script file as -stdin.

  • You can combine and concatenate multiple scripts in similar fashion:

    SOME_PIPELINE |
    sed -e'#some expression script'  \
        -f./script_file -f-          \
        -e'#more inline expressions' \
    ./actual_edit_file >./outfile
    

    A POSIX sed will concatenate all scripts into one in the order they appear on the command-line. None of these need end in a \newline.

  • grep can work the same way:

    sed -e'#generate a pattern list' <in |
    grep -f- ./grepped_file
    
  • When working with fixed-strings as patterns, it is good practice to escape regular expression metacharacters. You can do this rather easily:

    sed 's/[]$&^*\./[]/\\&/g
         s| *\([^ ]*\) *\([^ ]*\).*|s/\1/\2/g|
    ' <patterns.txt |
    sed -f- ./editfile >outfile
    

5. Multiple replace operations: replace multiple patterns with the same string

  • Replace any of foo, bar or baz with foobar

    sed -Ei 's/foo|bar|baz/foobar/g' file
    
  • or

    perl -i -pe 's/foo|bar|baz/foobar/g' file
    
share|improve this answer
1  
A version with ed should be added because sed -i is not supported on lots of sed version. but I never manage to use ed correctly –  Kiwy Feb 7 '14 at 12:52
1  
@somethingSomething, the sed command does not rename! It reads every line of every input file and replaces the first occurrence of foo with bar on each line. Of course that will take ages for 13GB of files! –  terdon Nov 2 '14 at 14:13
4  
On Mac OS X it must be sed -i '' 's/foo/bar/' file, as -i expects an extension string for the backup file ('' therefore means no backup file will be created) –  adius Jan 2 at 19:17
1  
@StéphaneChazelas thanks for the edit, it did indeed fix several things. However, please don't remove information that is relevant to bash. Not everyone uses zsh. By all means add zsh info but there is no reason to remove the bash stuff. Also, I know that using the shell for text processing is not ideal but there are cases where it is needed. I edited in a better version of my original script that will create a sed script instead of actually using the shell loop to parse. This can be useful if you have several hundred pairs of patterns for example. –  terdon Jan 16 at 15:10
1  
@terdon, your bash one is incorrect. bash before 4.3 will follow symlinks when descending. Also bash has no equivalent for the (.) globbing qualifier so can't be used here. (you're missing some -- as well). The for loop is incorrect (missing -r) and means making several passes in the files and adds no benefit over a sed script. –  Stéphane Chazelas Jan 16 at 15:16

"You could also use find and sed, but I find that this little line of perl works nicely.

perl -pi -w -e 's/search/replace/g;' *.php
  • -e means execute the following line of code.
  • -i means edit in-place
  • -w write warnings
  • -p loop over the input file, printing each line after the script is applied to it.

" (Extracted from http://www.liamdelahunty.com/tips/linux_search_and_replace_multiple_files.php)

My best results come from using perl and grep (to ensure that file have the search expression )

perl -pi -w -e 's/search/replace/g;' $( grep -rl 'search' )
share|improve this answer

protected by Community yesterday

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.