New answers tagged

0

The certificate you stored in ~/.cacert/lets-encrypt-root-x3.pem should not be left alone since it is an intermediate certificate. As you may see, it is issued by O=Digital Signature Trust Co., CN=DST Root CA X3. When passing to wget the ca-certificate argument, the specified file should a list of all CA certificates in PEM format. And this list is used ...


0

I answered this question in another question: Downloading large folder from google drive I wrote a Python code using the PyDrive library, that can recursively retrieve the sub-folders and files inside a parent folder. Using the ids of the files, I then generated a bash script using wget. Step 1 I have used the PyDrive library. To use this library, you have ...


0

I wrote a Python code using the PyDrive library, that can recursively retrieve the sub-folders and files inside a parent folder. Using the ids of the files, I then generated a bash script using wget. Step 1 I have used the PyDrive library. To use this library, you have to complete the instructions described in this link. Step 2 Now, create a python script or ...


0

Using wget -bc should work. By the way, if you want your job to keep running after you disconnect to the remote server, nohup COMMAND >log 2>err & might be a good idea.


0

You can either use screen/tmux or run wget in background (-b option)


1

Install wget using one of the defined below command: sudo apt install wget sudo yum wget sudo pacman -S wget Why you are compiling it? If you need specific version you can compile it this way: wget https://ftp.gnu.org/gnu/wget/wget-1.8.2.tar.gz tar xvzf wget-1.8.2.tar.gz cd wget-1.8.2 ./configure --with-openssl make make install You can choose which wget ...


0

Since there are tools that already get the video from Youtube, you could read their source code to know how they do it, to reimplement it in bash. youtube-dl - parser here (python) VLC - parser here (lua)


0

wget has command-line arguments for it. So, this wget -O - yourlink will sent wget output to stdout. Now, you can redirect it wherever you like. Here -O - sends the downloaded file to stdout. Link to wget Doc


0

EDIT: TL;DR You can't do it with openssh's sftp client, but you can with curl's sftp implementation as demonstrated in this answer, found by Aditya Shaw: https://serverfault.com/questions/400203/how-to-pipe-data-to-sftp-connection/934153#934153 Basically, $ wget -O - google.com | curl -T - -u remote_user sftp://remote_server/home/remote_user/fileName ...


0

You should try to render wget like a human browser: wget --user-agent='Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0' --recursive --no-parent -nd https://*edu/pages/access/search.jsp


1

Yes, there is a reasonable way to set a max file size with wget, the question is enought reason. So in this link there is a great patch for wget, and a wget already compiled. I used it for my reasonable reason I had, and it worked so good. I'd like they add this to de wget in linux repository. https://yurichev.com/wget.html Good luck.


3

The problem is the read which expects to read from stdin, and that fails when you're piping (it fails to read what you are expecting to read and instead reads the text of the actual script, which is being piped in on standard input, causing the syntax error by effectively deleting the line after the read statement from the script). So use command ...


1

You could try a RAID 0 assembly, but I'm not at all convinced it will work with partial blocks (i.e. your source files must be a complete number of blocks). Here is a worked example for you to try before you attempt to join your real data # Prepare two 10MB image files for the example ( echo hello; cat /dev/zero ) | dd bs=1M iflag=fullblock count=10 > /...


Top 50 recent answers are included