8

I'm trying to write a bash script where part of a directory is a long, unknown string. I know the first part of the directory name. How do I combine the known string and the wildcard in the path? As of now, I have:

ID=$1;

IMP="[email protected]:/data/"$ID"/EER_DATA/$ID"*"/ImpostorScores.txt"

but this passes a literal *.

7
  • bash does this by default e.g. for FOO=/*/passwd. Is the path local (so an immediate glob expansion can happen) or is it remote?
    – thrig
    May 19, 2016 at 22:56
  • Globbing doesn't understand hosts. This will look for a directory named '[email protected]:` in the current directory and so on. The glob won't find a match and thefore remain as an asterisk. What's would you do with the $IMP variable if if did expand across hosts? May 19, 2016 at 22:56
  • I'm sorry, I don't understand the question.
    – Adam_G
    May 19, 2016 at 22:58
  • In regards to the first question, it's a remote path. Ultimately I'll be using this in an scp command
    – Adam_G
    May 19, 2016 at 22:59
  • 1
    Globbing a remote path would require either that the filesystem be exported (NFS, FUSE-over-SSH, etc) or that a SSH is done over to the system and a glob run there.
    – thrig
    May 19, 2016 at 23:42

3 Answers 3

2

There are several options,

  1. Use NFS or FUSE-over-SSH or something to expose the remote filesystem locally, then let bash apply the default FOO=/*/passwd glob on that exported filesystem path. (ZSH has a ${~spec} glob substitution parameter expansion, otherwise see your shell's manual.)
  2. SSH over to the remote system, and do the glob there. ssh host 'echo /*/passwd'
  3. Pass the escaped (or otherwise defanged via backslash or single quotes) glob through to the remote system, scp host:/\*/passwd .

Some of these options are prone to race conditions should the filename change between when the glob is done and any subsequent steps involving that (possibly changed) filename.

2
  • I'm not familiar with 1. For #2, can you provide more detail? I understand how to login to my server via ssh, but I've never used it in a shell script on my local machine. For 3, I've tried escaping and using single quotes, but it's still passing "*"
    – Adam_G
    May 20, 2016 at 0:37
  • @thrig #2 is resolve the full path first then assign it to a variable. dest=$(ssh host 'echo /data/$ID/EER_DATA/$ID*'). Then use that variable, scp filename host:$dest/filename
    – vjangus
    May 20, 2016 at 1:51
0

You can use tar over ssh,

tar cf - filename | ssh [email protected] "cd /data/$ID/EER_DATA/$ID*; tar xf -"

Make sure you have setup ssh-agent so it won't prompt you with the password. I don't think you can use wildcard dir with scp, not even working with a regular cp.

I've tested this on bash and zsh.

1
  • I'm trying to copy a file, not a directory. This returns the error "Not a directory". Which part should I edit?
    – Adam_G
    May 20, 2016 at 19:31
0

The right side of a "Variable Assignment" is considered quoted (no splitting or globing):

LESS=+'/A variable may be assigned' man bash

A variable may be assigned to by a statement of the form:
name=[value]
Word splitting is not performed, ... . Pathname expansion is not performed.

Therefore:

$ var=*
$ echo "$var"
*

To get an asterisk to expand, you need to use it un-quoted:

var="$(echo *)"                ### Expand to the list of files on pwd.

In your case:

$ ID=one
$ IMP="$( echo "[email protected]:/data/$ID/EER_DATA/"*"/ImpostorScores.txt")"
$ echo "$IMP"
[email protected]:/data/one/EER_DATA/something/ImpostorScores.txt

That something will be expanded if the whole path match, that is, a tree like this actually exists:

$ tree [email protected]\:/
[email protected]:/
└── data
    └── one
        └── EER_DATA
            └── something
                └── ImpostorScores.txt

Note: I removed the second $ID from your source, easy to re-place.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .