Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a config file which consists of list of URIs I want to download. For example,

  http://xyz.abc.com/Dir1/Dir3/sds.exe
  http://xyz.abc.com/Dir2/Dir4/jhjs.exe
  http://xyz.abc.com/Dir1/itr.exe

I want to read the config file and and copy each URL but at the same time create the same directory structure as on the host. For example, for the first line in the config file, I want to create the directory structure Dir1/Dir3 on my local machine (if it doesn't exist) and then copy sds.exe to .../Dir1/Dir3/

I found that I can download all the URLs in a file using 'wget -i' but how can I create the corresponding directory structure with that

share|improve this question

1 Answer 1

up vote 4 down vote accepted

From man wget:

-x, --force-directories:

[...]

create a hierarchy of directories, even if one would not have been created otherwise. E.g. wget -x http://fly.srk.fer.hr/robots.txt will save the downloaded file to fly.srk.fer.hr/robots.txt.

share|improve this answer
    
Thanks for the man recommendation. I should get used to helping myself. –  NGambit Apr 12 '13 at 21:16
    
Chris: But seems like if I have fly.srk.fer.hr/dir1/robots.txt then wget -x creates fly.srk.fer.hr directory and puts file named dir1/robots.txt in it. What I need it that it create even the subdirectories. i.e. save robots.txt to fly.srk.fer.hr/dir1/ –  NGambit Apr 18 '13 at 14:20
    
@NGambit It's impossible to create a single file named dir1/robots.txt on Unix. –  Chris Down Apr 18 '13 at 14:32
    
You are right. The config file I was using had '\' instead of '/' (since it was parsed from another windows based file) in directory paths which was messing things up. Just replaced the '\' with '/' and "wget -x -i filename" works like a charm –  NGambit Apr 18 '13 at 23:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.