1

I have a large tarball file on an OS X web server that I uploaded via sshfs. The file on my computer prior to uploading is 5345834658 bytes. The file after copying is 5345834658 bytes on the server so everything seems to have gone correctly. If I copy the file to another location on the server it is also 5345834658 bytes.

The problem is when I try and download the file. The Firefox dialog box says "Gzip archive (1.0 GB)" and the downloaded file is only 1050867362 bytes. If I use curl to download the file, either my local machine or the server itself, I only get 1050867362 bytes.

I am thinking something about the web server is configure to limit the maximum file size. Is there a way to work around this? I am okay splitting the file up on the server as long as there is a way to recombine the pieces, but I would really like to just be able to download the large file. Is there a reason to limit the maximum file download size?

  • You can use download accelerators that will request different parts of the file which will reassemble the file for you as well and keep each piece under 1Gb as long as the actual HTTP header returns the right size. PS: this does look like a webserver problem/limitation. – Srdjan Grubor Nov 7 '14 at 21:24
  • Your Webserver Download Limit is 1GB, and your Downloading the 5GB File you uploaded. Celeda's answer is correct. The math involved is 4 * (1,024)^2 = 429,4967,296. I came to the same conclusion by dividing both numbers you gave by (1,024)^2. 1st 1,024 converts GB's to KB's and the 2nd converts KB's to Bytes. – eyoung100 Nov 7 '14 at 21:39
3

The size of the file you are getting through the HTTP server is exactly the remainer of 5345834658 mod 4294967296. That means the web server overflowed a 32-bit integer and is using the remainer as the file size.

You need to upgrade to a web server that supports large files (>4GiB).

  • I knew there was a reason I posted the exact file sizes. Is this definitely on the server or could it be a proxy, firewall, or something else in between? The server is running OS X 10.4.11 Tiger. – StrongBad Nov 7 '14 at 21:39
  • I'm pretty confident that it's the server's fault. It could in theory be a proxy but I think you're probably not using one. It can't really be due to a firewall. You can test to be absolutely sure about that by counting how many bytes you get downloading directly from the web server itself with something like curl http://localhost/the/file/path | wc -c executed directly on the web server itself. I think that's likely to result in 1050867362 bytes. Tiger is a bit old and I'm not too surprised it would have this limitation. – Celada Nov 7 '14 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.