1

I have some 250 CSV source files in the incoming directory.

My task is to merge those CSV files and generate a file like Merge_. the Informatica Job will then pick the merge file as the source and work.

Now the merge part is taken care of. I want all the CSV files to be zipped and archived to another directory say /source/archive as soon as the merge file is generated.

Also, I want the Merge_ file to remain like that and untouched.

I am aware of how to zip and tar separately but not sure how to combine both and move to some other directory.

I am trying to use something like this but it's not working:

tar -czvf All_Files.tar.gz *.csv  /Source/Archive --remove-files --exclude Merge_file_1.csv, Merge_File_2.csv

Expecting it to tar and zip then move the file to /source/archive and remove the original files except the merge files

Improve this question
2
  • Both zip and tar have options to write to a specific file: zip --outfile somewhere or tar -cf somewhere, where somewhere is a full path that can be outside your workingdir. You could tar --to-stdout and redirect stdout where you need it. Or you could leave the zipping to tar tar czf somewhere Merge_*.
    – markgraf
    Commented Sep 16, 2019 at 7:05
  • Added the command i am trying but first off all, it's creating tar file in same directory and error like "tar: Removing leading `/' from member names" and i also want to exclude the two merge files from getting removed
    – user3901666
    Commented Sep 16, 2019 at 7:29

1 Answer 1

Reset to default
0

You need to do it in two commands, as --remove-files will ignore --exclude.

tar -czvf ../All_Files.tar.gz --exclude 'Merge_*' *.csv && \
find . -type f -name '*.csv' -not -name 'Merge_*' -delete

The message "tar: Removing leading '/' from member names" is NOT an error, but tar working as intended.

Please note that --exclude takes a pattern (e.g. 'Merge_*'), not a comma-separated list. You need to put it prior to the files you want to exclude from the archive, otherwise the Merge_-files will end up in your archive.

Improve this answer
3
  • can i run this together in the script as you have used && \. Btw what does && \ do?
    – user3901666
    Commented Sep 16, 2019 at 7:59
  • 1
    && will only run the command following it íf the previous command was successful. So if tar fails for some reason, your csv-files will not be deleted. The backslash tells the shell that the command is continued on the next line. It is not needed, but it makes the command easier to read.
    – markgraf
    Commented Sep 16, 2019 at 8:03
  • Thanks a lot mark !! Much Appreciated for your help. :)
    – user3901666
    Commented Sep 16, 2019 at 8:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .