2

I am attempting to download a set of files from an online database, which requires the use of wget. The shell script being used works on one of my computers, but I want to make it run on a different device, and therein lies the problem. For whatever reason, when executing the script, it gives me the following output:

--2021-07-21 12:03:12--  https://archive.eso.org/downloadportalapi/calibrationxml/672f7dfa-65b6-4117-aaf5-9eb2fd4d8759/GRAVI.2017-06-29T02:59:55.346_raw2master.xml
Resolving archive.eso.org (archive.eso.org)... 134.171.46.246
Connecting to archive.eso.org (archive.eso.org)|134.171.46.246|:443... connected.
HTTP request sent, awaiting response... 200 
Length: 3484 (3.4K) [application/octet-stream]
GRAVI.2017-06-29T02:59:55.346_raw2master.xml: Invalid argument

Cannot write to 'GRAVI.2017-06-29T02:59:55.346_raw2master.xml’ (Success).

Instead of writing a line describing where it is being saved as/to, it instead outputs "Invalid argument" and I'm not sure why. Both devices are running the same version of wget and the script used on both devices is identical. I also have plenty of disk space, so that isn't an issue either. If you want to look at the shell script, it is available here.

EDIT: Here are the outputs of

mount

/dev/sdb2 on /media/keegan/LaCie type exfat (rw,nosuid,nodev,relatime,uid=1001,gid=1001,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro,uhelper=udisks2)

and df -h

/dev/sdb2/      9.1T   14G  9.1T   1% /media/keegan/LaCie

as requested in the comments.

Improve this question
11
  • The usual practice is to add script in the question. And google drive do not permit access to the file
    – Romeo Ninov
    Jul 21, 2021 at 10:25
  • 2
    Hello and welcome to Unix & Linux. Can you add the wget line with its options specifically here? I know you've linked to the script, but the direct line will help contributors get started.
    – Danny Staple
    Jul 21, 2021 at 10:26
  • Also - can we safely assume you are wget'ing to a Linux/unix box? Can you add information about the file system it is attempting to write the file to?
    – Danny Staple
    Jul 21, 2021 at 10:27
  • Okay, it appears that the issue arises from wget-ing the files directly to an external hard drive, as I can run the script successfully on the integrated harddrive of the computer. Any idea why that might be?
    – Azog
    Jul 21, 2021 at 12:13
  • I am working with a Linux machine (Ubuntu 20.04) in both cases. In addition, I adjusted the google drive link, you should be able to view it now. I wanted to avoid adding the script directly in the question as it is rather lengthy, but I can do so if you request.
    – Azog
    Jul 21, 2021 at 12:19

2 Answers 2

Reset to default
2

You are trying to create the file on a file system where the name contains illegal characters (the :). As you state in your edit, the file is to be created on an exFAT file system. The Wikipedia page on exFAT explicitly states:

Allowed characters in filenames: all Unicode characters except U+0000 (NUL) through U+001F (US), / (slash), \ (backslash), : (colon), * (asterisk), ? (question mark), " (quote), < (less than), > (greater than) and | (pipe)

To verify, try to create the file with another command:

touch GRAVI.2017-06-29T02:59:55.346_raw2master.xml

If that is the cause, either use another file system, or use another name.

Improve this answer
2
  • As mentioned in the question, there shouldn't be any issues with the script, since it works fine "as is" on another device, and now also on the integrated harddrive of the computer in question. The problem seems to be linked to trying to download the files directly to an external harddrive, for whatever reason.
    – Azog
    Jul 21, 2021 at 12:20
  • To answer your question more directly, I can download individual files using curl and a bit of a workaround, but I would like to avoid needing to rewrite the scripts, which are pre-generated by the database when i make a download request.
    – Azog
    Jul 21, 2021 at 14:24
1

As told before, there are some character limitations in the target file system (exFAT). Therefore you should use --restrict-file-names=windows option for wget. That converts problematic characters in the target file names and directory names.

Improve this answer
1
  • Solved problem for me, thanks!
    – Frankie Drake
    Oct 11, 2022 at 9:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .