1

I've downloaded a website from the net using the wget command-line tool with --restrict-file-names=windows so that once the download is complete I could copy the downloaded files to a mounted Windows 7 partition (more precisely, a directory on a Windows 7 host system mounted from an Ubuntu Linux guest system running inside Oracle VirtualBox). However after mounting the directory and copying the cp command gave me an error for the following files:

cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/\321%81ка\321%87а\321%82\321%8C': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/\321%83\321%81\321%82н\321%8Bй-пе\321%80евод': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/заве\321%80ение': Protocol error
cp: cannot create regular file `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/к\321%83\321%80\321%81-англий\321%81кого.1': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/ново\321%81\321%82и': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/к\321%83\321%80\321%81-англий\321%81кого': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/ка\321%80\321%82а-\321%81ай\321%82а': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/подпи\321%88и\321%82е\321%81\321%8C-на-на\321%88\321%83-\321%80а\321%81\321%81\321%8Bлк\321%83': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/category/ново\321%81\321%82и': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/пи\321%81\321%8Cменн\321%8Bй-пе\321%80евод': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/\321%81в\321%8Fзи': Protocol error
cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/\321%83п\321%80авление-подпи\321%81ками': Protocol error

How come wget didn't properly convert the links so that they could be properly copied to my Windows 7 64-bit NTFS file system?

EDIT #1

As pointed out, the main problem seems to be with the mounting software from Oracle VirtualBox, although IMHO something could be done at the wget level as well, because if I zip the files, copy the zipped folder with cp over to the mounted partition, and then extract there with 7-Zip, then the filenames which were failing to get copied still have messy/unreadable characters in them (I'm referring in particular to the almost completely black squares characters) as displayed in the image below:

Files Downloaded With Wget Restrict File Names Windows, copied to Windows Mounted Foldr, and extracted with 7-Zip

2

These messages look like the underlying software you're using to do the mounting cannot handle the special characters that wget is using due to the --restrict-file-names=windows argument.

cp: cannot create directory `/mnt/Desktop/WebSites/foo/www.johndoeandjanedoe.com/ru/\321%81ка\321%87а\321%82\321%8C': Protocol error

Per the wget man page
--restrict-file-names=modes

Change which characters found in remote URLs must be escaped during generation of local filenames. Characters that are restricted by this option are escaped, i.e. replaced with %HH, where HH is the hexadecimal number that corresponds to the restricted character. This option may also be used to force all alphabetical cases to be either lower- or uppercase.

...

When "windows" is given, Wget escapes the characters \, |, /, :, ?, ", *, <, >, and the control characters in the ranges 0--31 and 128--159. In addition to this, Wget in Windows mode uses + instead of : to separate host and port in local file names, and uses @ instead of ? to separate the query portion of the file name from the rest. Therefore, a URL that would be saved as www.xemacs.org:4300/search.pl?input=blah in Unix mode would be saved as www.xemacs.org+4300/search.pl@input=blah in Windows mode. This mode is the default on Windows.

To confirm this I'd try using 7z to create a compressed archive of the directory (recursively), copy that file over, and then uncompress it on Windows.

NOTE: The issue is likely with those special characters that wget is using when you specify that it should output files for consumption on Windows, in combination with the software being used to do the mounting.

  • Hi, as a matter of fact, most of the downloaded file names used to have a ? followed by a query string, making it impossile to copy the files over to the windows partition. Using the --restrict-file-names=windows option solved the problem by using an @ instead of the ? character, allowing most of the files to be copied to the partition, but the problems with copying some of the files still persisted as pointed out in my post. I've tried zipping the files, copying over the zipped folder to the mounted partition, and extracting the files there. – John Sonderson Dec 2 '14 at 12:57
  • Once extracted with 7-Zip, all files were extracted successfully with no errors, although some of the characters are really weired-looking (and non-alphabetical it seems). I've updated my post to display the folder with the unzipped filenames. The problem indeed seems to be with the VirtualBox share folder mount type vboxsf, command: sudo mount -t vboxsf src dst. Regards. Thanks. – John Sonderson Dec 2 '14 at 13:01
  • @slm Small typo sighted: must be --restric**t** ... Unfortunately due to the (IMO most idiotic) rule on SE that you may only fix typos if you also edit something else (>=6 char limit), I can't fix it myself, as there is really nothing else that needs to be altered. – syntaxerror Jun 24 '15 at 18:24
  • @syntaxerror - thanks, fixed. Yes that has annoyed me many times as well.it – slm Jun 25 '15 at 1:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.