I have a folder tree which contains multiple .zip folders (approx 600) each containing four files. I wish to extract just one file from each folder and copy it out to a separate folder. To complicate things, the file has the same name in each folder.Is there a script i can run to do this? Any help appreciated.

  • How do you recognize the correct file? By content, file size or date? – Alexander Aug 18 '15 at 9:50
  • it is by file name, the file has the same name in each folder. – Phil Aug 18 '15 at 10:25
  • But if the file has the same name in each folder, then how do you recognize the correct file, after extracting? By folder name? – Alexander Aug 18 '15 at 10:30
  • 1
    @alexander , each folder represents a date. eg: 20150301 and the file is a standard name such as log1.gz. I'm ultimately trying to amalgamate the contents of files for analyses. – Phil Aug 18 '15 at 10:34

You can try the following:

mkdir -p "DESTDIR"
for i in *.zip ; do
         unzip "$i" "THEFILE" -d "DESTDIR"
         mv "DESTDIR/THEFILE" "DESTDIR/$(basename "$i" .zip)_THEFILE"
done

Replace DESTDIR with the destination path. And THEFILE with the filename of the the file in each .zip file which you wan to extract. The files are renamed according to the filename of the zip file.

  • jofel, thanks for quick response. It may be my poor scripting knowledge but I can't get that to work. Here is an example of my folder structure: -> top_folder -> folder1 (...n) / file1.gz, file2.gz, file3.gz I want to extract file1 only from each folder and use it's contents in analyses later. – Phil Aug 18 '15 at 9:46
  • @Phil Do you have .zip or .gz or .tar.gz files? It seems that your question was not clear in this point. Please edit it. Make a short example with the current situation and the final situation you want. – jofel Aug 18 '15 at 9:56
  • @Phil: jofel's answer is correct for the question as posed. If you need something else, your question needs to be different. You do realize that "THEFILE" in his answer is a kind of variable that you are supposed to replace, such as with file1.gz in your case, right? – Warren Young Aug 18 '15 at 9:58
  • @jofel sorry for confusion. This is inhered data so I'm trying to make sense of it. The top folder seems to be an archive. Listing it in a terminal, it shows up as "FolderName.zip" When I navigate to the next level I have a list of folders and in each, there are a number of files with a .gz extension. It is for example "file1.gz" which I want to extract and output to another folder. thanks – Phil Aug 18 '15 at 10:23
  • @Phil: So are you saying you just just one Zip file, containing many *.gz files, collected into sub-folders? That is, FolderName.zipdir1/file1.gz, dir1/file2.gz, etc., then dir2/file1.gz, dir2/file2.gz, etc? Also, is this an actual zip file, or a directory named FolderName.zip? – Warren Young Aug 18 '15 at 10:34

Info-Zip's unzip program can recognize glob patterns in file names. Therefore, if you have a zip file with many files called file1.gz in different folders:

$ unzip FolderName.zip '*/file1.gz'
Archive:  FolderName.zip
 extracting: FolderName/b/c/file1.gz          
 extracting: FolderName/b/e/file1.gz          
 extracting: FolderName/b/f/file1.gz 

This despite the fact that FolderName.zip also contains files named things other than file1.gz. If you can match the desired file with a glob pattern, unzip can extract it.

I get the impression from your other commentary that you need to concatenate all of these extracted files. Something like this will do what you want, assuming the top-level folder containing the unzip output is called FolderName, and the target file (file1.gz) is at the same folder depth in all subfolders:

$ gzcat FolderName/*/*/file1.gz > one-big-uncompressed-log-file

That is, it skips over two subfolder levels to pull out the file1.gz files, then uncompresses and concatenates them with the common gzcat program, found most everywhere you find gzip support.

  • This looks like what I need, I'll not get back to it for a couple of hours but will let you know how I get on. Many thanks. Phil – Phil Aug 18 '15 at 11:18
  • Looks like I have not made this easy for myself. I'm trying to do this process on a SQLServer platform. The terminal command line does not recognise Unzip. I have installed the cyqwin terminal and it recognises gunzip but will not accept wildcards "*" so I'll need to spend some time getting a zip application installed and try again. Is there any you recommend to install or avoid? – Phil Aug 18 '15 at 13:50
  • @Phil: Re: "it recognises gunzip". gunzip is not the same thing as unzip. Re-run Cygwin's setup.exe and install unzip. – Warren Young Aug 18 '15 at 13:57
  • @Phil: Re: "will not accept wildcards" This is either the same confusion as above (gunzip vs unzip) or you are trying to run Cygwin commands through PowerShell or cmd.exe. Use the Cygwin Terminal, which runs Bash by default, which has the same quoting and wildcard processing behavior as a Linux box. – Warren Young Aug 18 '15 at 13:57
  • @Phil: Re: "any you recommend to install or avoid" Don't bother with anything that you can't install via Cygwin's setup.exe. Third-party native-Windows utilities will not play well with Cygwin. – Warren Young Aug 18 '15 at 13:57

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.