3

I have a list of filenames that I want to create soft links to (call it filenames.txt). When using

cat filenames.txt | awk 'system("ln -s "$0" ~/directory\ with\ spaces/subdirectory/")`

it seems that it chokes up when encountering anything (either a filename or the destination directory) with a space.

How do I create these soft links without having to first rename everything to ensure there are no spaces?

  • Are the items in filenames.txt only filenames or full paths ? – Sergiy Kolodyazhnyy Jan 15 '17 at 7:30
  • Only the names (the files are all in the same directory, so it should be trivial to prepend the full path as a prefix if that makes other things easier) – cjm Jan 15 '17 at 7:32
  • OK. I'll write some thing up, will post in like 10 - 15 mins – Sergiy Kolodyazhnyy Jan 15 '17 at 7:33
  • 1
    By the way, I fixed awk for you. There was some trickery that's needed to be done via sprintf() – Sergiy Kolodyazhnyy Jan 15 '17 at 10:18
2

NOTE: All examples presented here are to be run from the directory where files and the list of filenames are located. For instance, if they are in /mnt/Hard\ Drive/some\ files/ folder, please ensure the filenames.txt is stored there as well. When executing the commands from shell, please cd /mnt/Hard\ Drive/some\ files/ and then run the examples.

AWK

AWK's system call can be used with items that contains spaces, but it's slightly complicated. Your original code had a few syntax errors and useless cat there. Regardless of that fact, proper way would be for command to be build with sprintf() into variable first, and pass that variable to system(). Like so:

$ ls dir\ with\ spaces/                                                                                                  
$ awk '{cmd=sprintf("fpath=$(realpath -e \"%s\" );%s %s \"$fpath\" \"%s\"",$0,"ln","-s","dir with spaces/");system(cmd) }' filenames.txt                   
$ ls dir\ with\ spaces/                                                                                                  
ascii.txt@  disksinfo.txt@  input.txt@  process info.txt@

BASH ( or any Bourne-like shell )

The path of lesser resistance, would be via small shell script given below:

#!/bin/bash
# uncomment set -x for debugging
#set -x 

# Preferably, the directory should be full path
dir="dir with spaces/"

while IFS= read -r line
do
    if [ "x$line" != "x"  ] && [ -e "$line" ];
    then
        fpath=$( realpath -e "$line" )
        ln -s "$fpath" "$dir"
    fi
done < filenames.txt

Script in action:

$ ls -l dir\ with\ spaces/                                                                                        
total 0
$ cat filenames.txt                                                                                               
input.txt
ascii.txt
disksinfo.txt
process info.txt
$ ./makelinks.sh                                                                                                  
$ ls -l dir\ with\ spaces/                                                                                        
total 0
lrwxrwxrwx 1 xieerqi xieerqi  9 1月  15 00:47 ascii.txt -> ascii.txt
lrwxrwxrwx 1 xieerqi xieerqi 13 1月  15 00:47 disksinfo.txt -> disksinfo.txt
lrwxrwxrwx 1 xieerqi xieerqi  9 1月  15 00:47 input.txt -> input.txt
lrwxrwxrwx 1 xieerqi xieerqi 16 1月  15 00:47 process info.txt -> process info.txt

The way it works is simple: while IFS= read -r line; do . . . done < filenames.txt reads filenames.txt line by line, each time line going into line variable. We check if that line isn't empty and if the file exists ( this script is to be executed from the same directory where original files and the file-list are located ). If both conditions are true, we make links.

Note that you should ensure the file ends with a newline - if there's a last line that doesn't end with newline character ( which does happen ) , that final line will be skipped, thus there won't be any link made for that file.

Although not ideal and not without quirks, this is a working solution for cases where awk's system() is not available ( which is probably rare nowadays ).

xargs

A slightly simpler shell approach would be via xargs and bash -c '' sh , where we read filenames.txt , again, line by line with -L1 flag, and use each line as arguments to bash. Using $@ we take whole array of command-line arguments ( $1,$2,$3 . . . etc.) and turn in into string variable, which is then passed on to ln. The sh at the end is to set $0 variable. It's arguable whether it's necessary to do, but that's not the topic of this question, hence let's skip it:)

$ xargs -I {} -L1 bash -c 'file="$@";fpath=$(realpath -e "$file"); ln -s "$fpath"  ./dir\ with\ spaces/"$file" ' sh < filenames.txt                      
$ ls dir\ with\ spaces/                                                                                                  
ascii.txt@  disksinfo.txt@  input.txt@  process info.txt@

Python

$ ls dir\ with\ spaces/ 
$ python -c "import os,sys;fl=[f.strip() for f in sys.stdin];map(lambda x: os.symlink(os.path.realpath(x),'./dir with spaces/' + x),fl)" < filenames.txt
$ ls dir\ with\ spaces/                                                                                                  
ascii.txt@  disksinfo.txt@  input.txt@  process info.txt@

The way this works is simple - we redirect filenames.txt into python's stdin, read in each line into list fl, and then run os.symlink() for each item in list. Lengthy one-liner, but works.

Perl

A shorter version is achieved via Perl:

$ perl -lane 'use Cwd "realpath";symlink(realpath($_),"./dir with spaces/" . $_)' < filenames.txt                                                     
$ ls dir\ with\ spaces/                                                                                                  
ascii.txt@  disksinfo.txt@  input.txt@  process info.txt@
  • @cjm Did you quote $sourcedir variable like "$sourcedir" ? I'll look into other solutions, will see if I can fix those up. I'm using Ubuntu Linux, so I'm on different system, but in theory that should still work – Sergiy Kolodyazhnyy Jan 16 '17 at 4:59
  • @cjm OK, I think I found the issue. ln needs to know full path of the original file. I've added os.path.realpath() function call to python, tested it out - it resolves properly. Without it, it makes symlink to itself. I'll see what I can do about perl one – Sergiy Kolodyazhnyy Jan 16 '17 at 5:03
  • @cjm I know, I've seen your other comments. My test file also has a filename with spaces in it, so all examples I'm posting should be ok with filenames that contain spaces. It's the full path of the file that we need to worry about – Sergiy Kolodyazhnyy Jan 16 '17 at 5:07
  • Let us continue this discussion in chat. – cjm Jan 16 '17 at 5:10
2

For this kind of task, don't use a scripting language which calls an external command with system() or a similar function. It presents the same dangers and pitfalls as Bourne Shell's eval.

xargs allows you to do it safely and efficiently:

xargs -d '\n' ln -s -t ~/directory\ with\ spaces/subdirectory/ < filenames.txt

What it does is build one command by adding the content of filenames.txt at the end of the command you have given as parameter. Then it executes it. With the example above, the command it would build would look like:

ln -s -t directory file1 file2 ... fileN

What's interesting is that you don't have to quote anything in filenames.txt, and you don't have to use nested quoting for the directory.

  • I tried that and it complained xargs: unterminated quote – cjm Jan 15 '17 at 19:13
  • 1
    xargs isn't that "safe": only use plain xargs if you your file names don't contain whitespace or \'" otherwise see How to make xargs handle spaces and special chars from cat? – don_crissti Jan 15 '17 at 20:52
  • The filenames in the input file often do have spaces in them. – cjm Jan 16 '17 at 4:23
0

In the string literal, use a double backslash to get a single backslash in the string that is passed to the shell, which causes the shell to interpret the next character literally:

awk 'system("… ~/directory\\ with\\ spaces/subdirectory/")'

To quote the input for use in a shell snippet, one method is to add a backslash before every character. Portable awk doesn't make this easy (GNU awk has extensions that do). Alternatively, put a single quote around each element, and replace all single quotes in the element by '\''. Awk makes this solution easy: just use gsub. If your awk snippet is in a shell script, you can use a backslash-octal escape to put a single quote in an awk string literal.

<filenames.txt awk '{
    gsub(/\047/, /\047\\\047\047/, $0);
    system("ln -s \047" $0 "\047 ~/directory\\ with\\ spaces/subdirectory/");
}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.