sed in conjuction with xargs and grep not working

Question

I'm trying to change a string in multiple text files (configurations migrated to other user). I used this command:

grep -iIlr "/home/user/.local" | xargs sed -i 's/\/home\/user\/\.local/~\/\.local/g'

also tried this:

grep -iIlr "/home/user/.local" | xargs -d '\n' sed -i "s/\/home\/user\/\.local/~\/\.local/g"

They don't work. Errors is produced by sed, the output of the command is:

sed: can't read dir/file1: No such file or directory
sed: can't read dir/file2: No such file or directory
...
sed: can't read dir/file99: No such file or directory

The error is produced for every file (99 in total). The dir/file1..99 returned is valid, it's the proper path and filename for the files.

grep alone produces the proper list of files. No special characters on file names, I think not spaces also (not fully certain about this). Isn't xargs passing a proper list?

OS: RHEL 8.10 fully updated,
GNU grep v 3.1,
GNU sed 4.5,
xargs (GNU findutils) 4.6.0

---

Adding the output (sample last lines) of grep -iIlr "/home/user/.local" command.

pwd, ~/.local/pipx

venvs/pipx/bin/pipx
venvs/pipx/bin/activate.fish
venvs/pipx/bin/activate-global-python-argcomplete
venvs/pipx/bin/activate.csh
venvs/pipx/pyvenv.cfg
venvs/pipx/lib/python3.11/site-packages/pipx_shared.pth
shared/bin/pip3.11
shared/bin/activate
shared/bin/pip3
shared/bin/pip
shared/bin/wheel
shared/bin/activate.fish
shared/bin/activate.csh
shared/pyvenv.cfg

Answer 1

After my brain almost melted,

The problem was this:

$ alias grep
alias grep='grep --color=yes'

So grep output included some escape sequences for colour which xargs + sed could not parse. My eyes just saw nice colours :)

\grep or unalias grep solved the problem.

Answer 2

xargs by default expects an input in a very specific format where arguments are meant to be separated by whitespace characters (the list of which depends on the xargs implementation and the locale though in the case of GNU xargs that's limited to space, tab and newline) and where single quotes double quotes and backslash can be used to escape those (though in its own unique way).

That's definitely not the format of the list of files output by grep -rl.

With -d '\n' (a GNU extension), the format changes to being newline delimited with no way to escape newline. That happens to be the format output by grep -rl, but that still doesn't work for file names containing newline characters. The output of grep -rl is simply not post-processable reliably for arbitrary file names.

GNU grep (and you're already using a few GNUisms) has a -Z/--null option to output the list of files for -l in a post-processable fashion which you can combine with xargs -r0 (also GNU extensions, but more portable than -d):

grep -rlFIZ /home/user/.local . |
  xargs -r0 sed -Ei 's|/home/user(/\.local)\>|~\1|g'

I've also removed the -i option to grep as file paths are case sensitive on most Unix-like systems including RHEL and you're not using the i flag to sed's s anyway, added the -F for fixed-string match (to avoid having to escape the . regexp operator). Added \> for word boundary so it doesn't match on the /home/user/.local inside /home/user/.locale for instance (though it will still match inside /home/user/.local-tmp), explicitly specified which directory to recursively look into (for clarity and compatibility with older versions of GNU grep) and added some capturing, recalled with \1.

Your problem has been explained by you having a grep='grep --color=yes' alias instead of grep='grep --color=auto' or grep='grep --color' which means grep outputs colouring escape sequences even when its output doesn't go directly to a terminal.

There are other possible explanations.

Files in ~/.local are written by many applications, files are being added and remove at will by them; in between grep finding files and sed processing them, some of those files could very well have gone.

Also beware sed -i doesn't really edit files in-place, it replaces them with an edited copy. If some applications currently have the files you're editing opened, they will still carry on seeing the original (now deleted) copy and any change they're making to them from now on will be lost.

sed -i also breaks hard links and symlinks (though GNU grep -r will not look into symlinks).

Replacing /home/user with ~ is also often not a valid thing to do. Tilde expansion is something done by shells (most shells these days), and a few other applications, but certainly not all. Outside of those, ~/.local/file is the path of the file file inside the .local subdir of a subdir of the current working directory called ~ (like after mkdir -p '~/.local' && touch '~/.local/file'). So replacing /home/user/.local with ~/.local is likely to break many applications.

Also beware that GNU grep -I to skip binary files uses heuristics and is not foolproof (either way, can give false positives and false negatives). Some text-based formats like PHP data serialisation format will be broken if you replace a string with another one of a different length.