Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am using AIX 7.1.

I made some shell scripts8 and called /bin/bsh.

Inside the script, I connect to a server via FTP and have many get commands because I would like to avoid the wildcard problem. I just declare every filename and get them one by one, for example:

get PDF_02378230_20110609.PDF
get PDF_02432482_20110609.PDF
get PDF_02432565_20110609.PDF
get PDF_02432573_20110609.PDF
get PDF_02432581_20110609.PDF
get PDF_02432599_20110609.PDF
get PDF_02432607_20110609.PDF
get PDF_02432615_20110609.PDF
get PDF_02432623_20110609.PDF
get PDF_02432649_20110609.PDF
get PDF_02432656_20110609.PDF
get PDF_02432672_20110609.PDF
get PDF_02432755_20110609.PDF
get PDF_02432763_20110609.PDF
get PDF_02432821_20110609.PDF
get PDF_02432920_20110609.PDF
get PDF_02433175_20110609.PDF
get PDF_02433266_20110609.PDF
get PDF_02433290_20110609.PDF
get PDF_02433308_20110609.PDF
get PDF_02433373_20110609.PDF
get PDF_02433399_20110609.PDF

There could be up to 100000 files to be transfer in 1 shell scripts. I would run these shell scripts and let them ftp the files down. But in some shell script there are 1 or 2 missing files not been "get". For example, when I count the number of files, there might be 1 or 2 less files than the number of "get" command in the shell script. I cannot found an exception or error inside the log. The truth is, I cannot go through the whole log line by line because it is too long. But I have found which file was missing, and I tried to search the log file with the missing filename, and I cannot found any trace of that filename.

share|improve this question
    
Illuminate me. What "wildcard problem"? –  Ignacio Vazquez-Abrams Aug 22 '11 at 3:02
    
forget about it, I don't want to start it here. –  lamwaiman1988 Aug 22 '11 at 3:08
    
I'm genuinely curious, as I don't have much AIX experience. Is there a link I can read? –  Ignacio Vazquez-Abrams Aug 22 '11 at 3:10
    
That's nothing specific to AIX. It's just a common "argument too long" problem arosed by wildcard in unix syste. –  lamwaiman1988 Aug 22 '11 at 3:34
1  
Huh. I wouldn't have thought that it would affect a ftp script... –  Ignacio Vazquez-Abrams Aug 22 '11 at 3:36

2 Answers 2

Some FTP clients like lftp and ncftp have a mirror command that can be used to avoid problems like this by figuring out what needs to be done to sync a local and remote folder for you.

You might also check if your client supports mget or a way to do a recursive download and work on the parent folder instead of everything in it.

However the best solution would be to switch to a different protocol. This sounds like a great time to start using rsync.

share|improve this answer
    
AIX's default ftp client definitely supports mget. –  EightBitTony Aug 22 '11 at 9:33
    
+1 for rsync. This is a good use case for it. –  mattcaffeine Aug 23 '11 at 16:03

I would switch to wget if the other end has to be an FTP server. Even then you'll still need to handle exceptions where individual files don't transfer properly. With wget however you can at least tell it to retry a few times per file, and you can also run it again but tell it to ignore any files that are the same size locally (so it'll just re-get files that it didn't get first time).

Not sure if AIX 7.1 comes with wget pre-installed, but it'll be on the Linux Toolbox CD, or you can get it from the IBM website.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.