3

Can I use bash variable substitution to extract a piece of a variable based on a delimeter? I'm trying to get the immediate directory name of a filename (in this case, foo).

$ filename=./foo/bar/baz.xml

I know I could do something like

echo $filename | cut -d '/' -f 2

or

echo $filename | awk -F '/' '{print $2}'

but it's getting slow to fork awk/cut for multiple filenames.

I did a little profiling of the various solutions, using my real files:

echo | cut:

real    2m56.805s
user    0m37.009s
sys     1m26.067s

echo | awk:

real    2m56.282s
user    0m38.157s
sys     1m31.016s

@steeldriver's variable substitution/shell parameter expansion:

real    0m0.660s
user    0m0.421s
sys     0m0.235s

@jai_s's IFS-wrangling:

real    1m26.243s
user    0m13.751s
sys     0m28.969s

Both suggestions were a huge improvement over my existing ideas, but the variable substitution is fastest because it doesn't require forking any new processes.

  • 1
    Send all the filenames to one invocation of awk and it will be significantly faster than any solution in pure bash – 123 Jan 7 '16 at 14:04
  • Can you not use an array, do them all at once then put it in a new array ? – 123 Jan 7 '16 at 14:46
10

You can remove the shortest leading substring that matches */

tmp="${filename#*/}"

and then remove the longest trailing substring that matches /*

echo "${tmp%%/*}"
4
    echo $f
    a/b/c

    $ (IFS='/';set $f; echo $1)
     a

    $ (IFS='/';set $f; echo $2)
     b

    $ (IFS='/';set $f; echo $3)
     c

with wild card it seems to work with double or single quotes -

    f="a?b?c"
     $(IFS="?"; set $f; echo $1)
     a
    echo $f
    a*b*c
    (IFS="*"; set $f; echo $1)
    a

yes, you'll have to unset the IFS back to default

    unset IFS
  • Ooh, I like that. – bonh Jan 7 '16 at 14:45
  • This is usually my preferred method as well, but bear in mind that Bash only supports $1 through $9 using this syntax. For 10th and later arguments, the ${10} form must be used. – James Sneeringer Jan 7 '16 at 17:55
  • 1
    Doesn't work when $f contains wildcards. And you need to restore IFS afterwards (or do this in a command substitution, to get the value of a field, and that strips off trailing newlines). – Gilles Jan 7 '16 at 23:34
  • The example works in isolation (inside Git bash on Windows), but when I pipe from the find command I get this error: echo: write error: Bad address. – bonh Jan 25 '16 at 17:54
  • Okay, looks like I have to unset IFS every time. – bonh Jan 25 '16 at 17:57
1

Feed the list to awk to speed it up:

awk -F '/' '{print $2}' < <(find /usr)
awk -F '/' '{print $2}' < inputfile

Demonstration:

time awk -F '/' '{print $2; SUM++} END {print "number of directories found: " SUM}' < <(find /usr -type d)
usr
usr
.
.
number of directories found: 16748

real    0m8.910s
user    0m0.050s
sys     0m0.050s
0

Why don't you use the "dirname" command, instead of all this awk/sed/cut stuff?

filename=./foo/bar/baz.xml
dirname $filename

Yields:

./foo/bar
  • In this case I was looking for the immediate directory, not the full directory path. – bonh Dec 12 '17 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.