1

I am trying to copy files from one location to another and given below are some sample ones:

aaa_bbb_ccc_ddd_cost_code_20140330.gz
aaa_bbb_ccc_ddd_revenue_zone_20140329.gz
aaa_bbb_ccc_ddd_benefit_extract_20140330.csv.gz
aaa_bbb_ccc_ddd_profit_zone_20150509.csv.gz
aaa_bbb_ccc_ddd_loss_zone_20140330.csv
aaa_bbb_ccc_ddd_username.csv.gz

from the above list, the files that should be copied should be in the following format:

aaa_bbb_ccc_ddd_cost[or]revenue[or]benefit[or]profit[or]loss_yyyymmdd.csv.gz

which means that the files

aaa_bbb_ccc_ddd_loss_zone_20140330.csv
aaa_bbb_ccc_ddd_username.csv.gz

shouldn't be copied.

Also, I need to assign it to a variable and I am trying something like this but it doesn't seem to work:

FILENAME="egrep 'aaa_bbb_ccc_ddd_(cost|revenue|benefit|profit)_code_[0-9]{8}.csv.gz'"

The reason I am trying to assign this to a variable is because I need to use it later in the code for something like this:

SOURCE_DIR="/temp"
DESTN_DIR="/output"
FILENAME=`egrep 'aaa_bbb_ccc_ddd_(cost|revenue|benefit|profit)_code_[0-9]{8}.csv.gz'`
echo "FILENAME is:" $FILENAME
for SAMPLE_FILE in $(ls "$SOURCE_DIR/$FILENAME")
do
cp $SAMPLE_FILE $DESTN_DIR
done

Is there an alternate way in which this can be achieved?

5 Answers 5

5

Use find and its -exec option (here using GNU find for it's -regex predicate):

find . -regextype posix-egrep -regex '.*/aaa_bbb_ccc_ddd_(cost|revenue|benefit|profit|loss)_[[:alpha:]]+_[0-9]+\.csv\.gz' -exec mv {} "$DESTN_DIR" \;

Note:

  • find . tells find to look for files starting in the current directory.

  • By default, GNU find uses emacs-style regular expressions. I prefer -regextype posix-egrep but you may switch to any of the supported styles with which you are familiar.

  • A regular expression is used to select the files: -regex '.*/aaa_bbb_ccc_ddd_(cost|revenue|benefit|profit|loss)_[[:alpha:]]+_[0-9]+\.csv\.gz'. This allows for the standard prefix of aaa_bbb_ccc_ddd_ followed by one of the words specified by (cost|revenue|benefit|profit|loss), followed by another unspecified word, _[[:alpha:]]+, followed by a date, _[0-9]+, followed by the desired extension of .csv.gz. You may want to fine tune this.

  • Any such files found will be moved to the target directory via -exec mv {} "$DESTN_DIR" \;. When find finds a matching file, it executes this command replacing {} with the file name. This will work even if the file names have spaces, newlines, or other difficult characters.

Using the default (emacs) style of regular expression

The default style of regex for GNU find requires some escaping of the grouping and alternation operators:

find . -regex '.*/aaa_bbb_ccc_ddd_\(cost\|revenue\|benefit\|profit\|loss\)_[[:alpha:]]+_[0-9]+\.csv\.gz' -exec echo mv {} targetdir \;

Mac OSX

The Mac OSX version of find (man page here) supports -regex but not -regextype. I wouldn't be surprised, though, if its regex syntax required some subtle changes.

IBM AIX 5

The man page for the IBM AIX version of find is here. It is does not support -regex.

4
  • John, I get the error -->find: 0652-017 -regextype is not a valid option. I am using this on Korn Shell
    – user68112
    May 21, 2014 at 7:20
  • @user68112 I have added a version that doesn't use regextype. Are you on Linux(GNU) or OSX(BSD)? I tested both the commands above with GNU find version 4.4.2 (linux). GNU and OSX/BSD, unfortunately, often have minor but very annoying incompatibilities.
    – John1024
    May 21, 2014 at 8:07
  • I am on AIX 5.3 and running on Korn Shell but sadly the same issue still persists even with regex
    – user68112
    May 21, 2014 at 10:05
  • @user68112 I found the man page for AIX find here and, as you have found, it does not support either -regexttype or -regex. I have updated the answer to note that.
    – John1024
    May 21, 2014 at 18:46
3

With zsh:

setopt extendedglob
source_dir="/temp"
destn_dir="/output"
pattern='aaa_bbb_ccc_ddd_(cost|revenue|benefit|profit)_code_[0-9](#c8).csv.gz'
print -r "pattern is: $pattern"
cp -- $source_dir/$~pattern $destn_dir

ksh93 patterns can express that with

aaa_bbb_ccc_ddd_@(cost|revenue|benefit|profit)_code_{8}(\d).csv.gz

And ksh88 with:

aaa_bbb_ccc_ddd_@(cost|revenue|benefit|profit)_code_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].csv.gz

However the extended globbing operators don't work inside variables so as to avoid breaking POSIX compatibility:

echo @(a)

is wrong syntax as per POSIX, so ksh can make it a new glob operator. However:

x='@(a)'
echo $x

is completely specified per POSIX and is meant to output @(a) (with the default value of IFS), not a if there's a file called a in the current directory.

So you'd need to resort to using eval which can be tricky to get right

pattern='aaa_bbb_ccc_ddd_@(cost|revenue|benefit|profit)_code_{8}([0-9]).csv.gz'
print -r "pattern is: $pattern"
eval 'cp -- "$source_dir"/'"$pattern"' "$destn_dir"'
3
  • what is cp -- .. ?
    – Baba
    May 21, 2014 at 9:04
  • 2
    @Babyy, That's cp $file_or_option $dest or cp -- $file $dest. -- marks the end of options. Generally, when passing variable arguments to command, you need to mark the end of the options with -- so that the arguments are not taken as options by mistake. (note that you can leave out quotes like that with zsh only, with other shells it's cp -- "$file" "$dest", and even with zsh, that doesn't harm to use quotes here) May 21, 2014 at 9:08
  • @Stephane, Is there an equivalent option for ksh as well?
    – user68112
    May 22, 2014 at 5:26
1

That can be done in one line:

find /temp -maxdepth 1 -type f | \
 grep -P 'aaa_bbb_ccc_ddd_(cost|revenue|benefit|profit)_.*[0-9]{8}' | \
 xargs cp -t /output
  • find lists the contents of the folder without subfolders.
  • grep your filenames
  • and copy (cp) them of the target directory (-t)

You just have to ajust your regex, because in your question it was not 100% clear. Some files have .csv.gz, some .csv and some .gz.

1
set -- ./aaa_bbb_ccc_ddd_[!ul]*
cp "$@" -t $location2
var=$*
1

Any modern shell supports what you want directly, not with the basic glob syntax but like this:

cp aaa_bbb_ccc_ddd_{cost,revenue,benefit,profit,loss}_[0-9]*.csv.gz destination_dir

This expands into five arguments, each of which is a glob of the form ..._keyword_<digits>...

So, to answer your second question, here's how to assign each one to a variable in turn:

for FNAME in aaa_bbb_ccc_ddd_{cost,revenue,benefit,profit,loss}_[0-9]*.csv.gz
do
    echo $FNAME
    if [ -e $FNAME ]
    then
        cp $FNAME <destination>
    fi
done

The existence check (if [ -e $FNAME ]) is because if any of the five globs doesn't match anything, the glob will be retained as-is and you'll get an error message.

2
  • with the for loop option above, it says: A file or directory in the path name does not exist. It probably is considering it as a file and not as a pattern
    – user68112
    May 22, 2014 at 3:54
  • Oh right, that's indeed what it is: If you use a glob and it doesn't match anything, it stays in its glob form. Does it echo the star right before the error? You can add a if [ -e $FNAME ] then ... to suppress the error.
    – alexis
    May 22, 2014 at 12:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .