wget not retrieving correct-size file (corrupt or incomplete file?)

Question

I don't understand...

The actual download link appears to not be a http link but rather some Javascript action?

javascript:SendFileDownloadCall('PRODIMAGES.CIF.zip','PRODIMAGES.CIF.zip');

So after downloading manually, I went to the browser's download history to copy the direct link https://au.ingrammicro.com/_layouts/CommerceServer/IM/FileDownload.aspx?DisplayName=STD_FULL_FILEFEED.TXT&FileName=STDPRICE_FULL.TXT.zip

And I fed the URL into wget, along with my website credentials:

wget -q --user=XXXX --password=XXXX "https://au.ingrammicro.com/_layouts/CommerceServer/IM/FileDownload.aspx?DisplayName=STD_FULL_FILEFEED.TXT&FileName=STDPRICE_FULL.TXT.zip" -o STDPRICE.zip

Later, I see that adding the --user and --password make no difference so I omit:

[root@server datafiles]# wget "https://au.ingrammicro.com/_layouts/CommerceServer/IM/FileDownload.aspx?DisplayName=STD_FULL_FILEFEED.TXT&FileName=STDPRICE_FULL" -O STDPRICE.zip
--2019-09-15 19:53:29--  https://au.ingrammicro.com/_layouts/CommerceServer/IM/FileDownload.aspx?DisplayName=STD_FULL_FILEFEED.TXT&FileName=STDPRICE_FULL
Resolving au.ingrammicro.com (au.ingrammicro.com)... 104.98.45.15
Connecting to au.ingrammicro.com (au.ingrammicro.com)|104.98.45.15|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: /_layouts/CommerceServer/IM/Login.aspx?ReturnUrl=%2f_layouts%2fCommerceServer%2fIM%2fFileDownload.aspx%3fDisplayName%3dSTD_FULL_FILEFEED.TXT%26FileName%3dSTDPRICE_FULL [following]
--2019-09-15 19:53:29--  https://au.ingrammicro.com/_layouts/CommerceServer/IM/Login.aspx?ReturnUrl=%2f_layouts%2fCommerceServer%2fIM%2fFileDownload.aspx%3fDisplayName%3dSTD_FULL_FILEFEED.TXT%26FileName%3dSTDPRICE_FULL
Reusing existing connection to au.ingrammicro.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 85341 (83K) [text/html]
Saving to: ‘STDPRICE.zip’

100%[===================================================================================================================================================================================================>] 85,341       405KB/s   in 0.2s

2019-09-15 19:53:30 (405 KB/s) - ‘STDPRICE.zip’ saved [85341/85341]

Anyway, instead of getting the equivalent file to the one that I get by human clicking and downloading from the website I get an implausibly smaller file.

Confirming my fears, when I try to unzip, I get:

$ [root@server datafiles]# unzip STDPRICE.zip
Archive:  STDPRICE.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of STDPRICE.zip or
        STDPRICE.zip.zip, and cannot find STDPRICE.zip.ZIP, period.

On file inspection:

$ [root@server datafiles]# file STDPRICE.zip
STDPRICE.zip: HTML document, UTF-8 Unicode text, with very long lines, with CRLF line terminators

So wget has actually downloaded a HTML file which is being presented as a .txt.zip file? wth Can somebody enlighten me?

Answer 1

The site is redirecting you to the login page:

HTTP request sent, awaiting response... 302 Moved Temporarily
Location: /_layouts/CommerceServer/IM/Login.aspx?ReturnUrl=%2f_layouts%2fCommerceServer%2fIM%2fFileDownload.aspx%3fDisplayName%3dSTD_FULL_FILEFEED.TXT%26FileName%3dSTDPRICE_FULL [following]
--2019-09-15 19:53:29--  https://au.ingrammicro.com/_layouts/CommerceServer/IM/Login.aspx?ReturnUrl=%2f_layouts%2fCommerceServer%2fIM%2fFileDownload.aspx%3fDisplayName%3dSTD_FULL_FILEFEED.TXT%26FileName%3dSTDPRICE_FULL

It probably doesn't accept the credentials you provided as basic auth (which is what wget is sending) but instead using session cookies. You can try extracting the cookies from your browser (while logged in) and sending them with wget (--load-cookies). They might also be keeping an eye on other aspects of the request (user agent for example) that you might try modifying also.

If you can use curl instead, open the inspector (Ctrl+Shift+I), go to the network tab, download the file, right-click the entry for the download in the request list, mouse over "Copy", then select "Copy as cURL", and the command now on your clipboard will include the cookies.