using sed with variables defined in 2 arrays

Question

I have followed the very many posts on this site regarding the subject but I am still clearly doing something wrong...

My goal is to define values in two different arrays, then use sed to search a string of text defined in the first array with that of the second array.

Code below:

#!/bin/bash

# define variables
defaultdirs=( Templates Documents Music Pictures Videos )
customdirs=( custom\/templates custom\/documents custom\/music custom\/pictures custom\/videos )

# replace text strings
for index in ${!defaultdirs[*]}
    do
    echo ${defaultdirs[$index]} will become ${customdirs[$index]}
    sed -i 's/${defaultdirs[$index]}/${customdirs[$index]}/g' ~/Desktop/scripts/test_replace.txt
done

echo outputs the right strings, but sed is not be getting the right information as the text file remains unchanged.

Thoughts?

For reference, this is the contents of test_replace.txt

# This file is written by xdg-user-dirs-update
# If you want to change or add directories, just edit the line you're
# interested in. All local changes will be retained on the next run.
# Format is XDG_xxx_DIR="$HOME/yyy", where yyy is a shell-escaped
# homedir-relative path, or XDG_xxx_DIR="/yyy", where /yyy is an
# absolute path. No other format is supported.
# 
XDG_DESKTOP_DIR="$HOME/Desktop"
XDG_DOWNLOAD_DIR="$HOME/Downloads"
XDG_TEMPLATES_DIR="$HOME/Templates"
XDG_PUBLICSHARE_DIR="$HOME/Public"
XDG_DOCUMENTS_DIR="$HOME/Documents"
XDG_MUSIC_DIR="$HOME/Music"
XDG_PICTURES_DIR="$HOME/Pictures"
XDG_VIDEOS_DIR="$HOME/Videos"

Answer 1

First problem: single quoted strings don't have variables in them expanded.

Second problem: As-is, the slashes in your replacement will break the s///g command after fixing the first problem. Use a different delimiter for s.

Third (lesser) problem: You're running sed multiple times over the same file, which isn't very efficient, and the -i option for in-place editing is non-standard and different implementations that do provide it act differently (The common case people run into issues with is that the GNU version doesn't require a argument, but the Mac OS version does). When wanting to edit a file and save changes, it's usually better to use a dedicated file editor like ed or ex.

Doing it all with one ed invocation instead:

#!/bin/bash

# define variables
defaultdirs=(Templates Documents Music Pictures Videos)
customdirs=(custom/templates custom/documents custom/music custom/pictures custom/videos)

# replace text strings
(for index in ${!defaultdirs[*]}; do
     echo "${defaultdirs[$index]} will become ${customdirs[$index]}" >&2
     echo "g/${defaultdirs[$index]}/s|${defaultdirs[$index]}|${customdirs[$index]}|g"
 done;
 echo w) | ed -s test_replace.txt

---

Alternative that sends the X will become Y messages to standard output instead of standard error, and runs ed in a coprocess, with individual commands redirected to its input instead of using a pipeline:

#!/bin/bash

# define variables
defaultdirs=(Templates Documents Music Pictures Videos)
customdirs=(custom/templates custom/documents custom/music custom/pictures custom/videos)

coproc ED { ed -s test_replace.txt; } 2>/dev/null

# replace text strings
for index in ${!defaultdirs[*]}; do
     echo "${defaultdirs[$index]} will become ${customdirs[$index]}"
     echo "g/${defaultdirs[$index]}/s|${defaultdirs[$index]}|${customdirs[$index]}|g" >&${ED[1]}
done
printf '%s\n' w q >&${ED[1]}
wait $ED_PID

Answer 2

You can collect all the substitute commands into one input to sed as well:

for index in ${!defaultdirs[*]}
  do   echo "s#${defaultdirs[$index]}#${customdirs[$index]}#g"
  done | sed -f- ~/Desktop/scripts/test_replace.txt

Answer 3

The issue, as Shawn has already pointed out, is that you create a sed script with a syntax error. The syntax error comes from the fact that you try to use / in an s command that uses that same character as a delimiter.

You try to counteract this by escaping the / in the strings in the customdirs array, but you would need \\/ to actually insert an escaping \ in the string.

Instead, here's a different approach:

find_strings=( Templates Documents Music Pictures Videos )
replace_strings=( custom/templates custom/documents custom/music custom/pictures custom/videos )

set -- "${find_strings[@]}"

sed_stmts=( )
for replace_string in "${replace_strings[@]}"; do
   # sed_stmts+=( -e 's,\("$HOME/\)'"$1"'",\1'"$replace_string"'",' )

    # simpler, but less precise:
    #    sed_stmts+=( -e "s,$1,$replace_string," )
    # alternatively:
    #    sed_stmts+=( -e "s/${1//\//\\/}/${replace_string//\//\\/}/" )

    shift
done

sed "${sed_stmts[@]}" test_replace.txt >new-test_replace.txt

I've also taken the liberty to make the substitutions a bit more reliable by also matching the "$HOME/ prefix string and the final ".

This will end up calling sed like so:

sed -e 's,\("$HOME/\)Templates",\1custom/templates",' -e 's,\("$HOME/\)Documents",\1custom/documents",' -e 's,\("$HOME/\)Music",\1custom/music",' -e 's,\("$HOME/\)Pictures",\1custom/pictures",' -e 's,\("$HOME/\)Videos",\1custom/videos",' test_replace.txt

and it does this by building up a set of sed statements in the array sed_stmts which are then used in a single call to sed.

The pairing up of the two sets of strings in the two arrays is done by assigning one of the arrays to the list of positional parameters using set, and then iterating over the other array. In each iteration, shift is used to shift off the front-most element of the list of positional parameters.