3

I would like to delete everything from a folder except csv files

I am trying with a bash script and I am getting this error:

syntax error near unexpected token `('

This is my script :

 PATH=/tmp/

 run_spark_local
 rm -v !($PATH*.csv)

 cp -r $PATH /data/logs/

I have also tried

rm -v !("$PATH*.csv")
  • What is run_spark_local? – Jesse_b May 10 at 15:06
  • 1
    Turn on extglob- shopt -s extglob. – fd0 May 10 at 15:11
  • 2
    Clearing out /tmp could have adverse consequences on other programs running on the system. – Stephen Kitt May 10 at 15:29
  • @Jesse_b a function to run my spark app – MobZSPARK May 10 at 16:06
  • 2
    This is why you don't use ALLCAPS variable names in your code. – glenn jackman May 10 at 18:53
4

You should avoid setting the PATH variable. This is used by your shell to find valid commands, setting it to /tmp/ is going to prevent the script from being able to find the rm and cp commands altogether.

You can accomplish what you want with the following find command:

find /tmp -not -name '*.csv' -not -path /tmp -exec rm -vr {} \;

Note: this will delete any subdirectories under /tmp as well. If you do not want this you must change to:

find /tmp -not -name '*.csv' -type f -exec rm -v {} \;

Another note: This will still recurse into the subdirectories and delete the files in them. If you do not want this you can use the maxdepth argument:

find /tmp -not -name '*.csv' -maxdepth 1 -type f -exec rm -v {} \;

Extra note: I would never run a find ... -exec command that you find online without verifying it will do what you need it to do first. You should run:

find /tmp -not -name '*.csv' -not -path /tmp

And verify it is finding only the files you want before adding the -exec rm -vr {} \; bit.

  • find /tmp -not -name '.csv' -not -path /tmp -exec rm -vr {} \; This Worked well for me Thank you What if i want to do the same thing but in the hdfs . does this work ? hdfs dfs find /tmp -not -name '.csv' -not -path /tmp -exec rm -vr {} \; – MobZSPARK May 10 at 15:47
6

Please do not set $PATH it is a environment variable.

For BASH, so long as the extglob shell option is enabled (that is the default for a lot of linux distributions), it is just:

rm !(*.csv)

With a folder path it will be something like...

rm yourfolder/!(*.csv)

If you think extglob is not enabled in your environment, just do this:

shopt -s extglob
4

Instead of deleting everything else in /tmp, I’d recommend only copying the files you’re interested in:

cp /tmp/*.csv /data/logs/

or even

mv /tmp/*.csv /data/logs/

if you don’t need to keep them in /tmp.

This assumes that all the files you’re interested in are directly in /tmp; your use of rm suggests that they are.

Since /tmp is a shared temporary directory, it could contain other files which other running processes expect to find there, and deleting everything in /tmp apart from the CSV files could have adverse consequences. As others have mentioned, you shouldn’t change PATH either since your shell uses that to find the commands you’re using.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.