0

I am trying to download a file form a HTTP website from my Linux shell using wget and curl.

Below is the manual GUI process of doing it:

  1. Once I click on the Link-A, it redirects me to a different webpage with a different web Address (Lets call it Link-B) . This webpage has a textbox with a list of items below it.

  2. Then I type a string in the textbox, to filter the list of names below it and it narrows down to only one Item below , then I click on that Item, it redirects me to a different webpage with a different web Address (Lets call it Link-C)

  3. On this webpage I Enter Username and Password and click submit, and it brings me back to Link-A but now it has a list of downloadadble files, where I can download files.

I am trying to download files using command line instead of GUI using tools like curl or wget. I tried using wget with --user and --ask-pass but it only downloads the webpage of LinkB and doesnt get to the file itself. Can anybody suggest a solution, Thanks in advance.

1
  • You have to find out what exactly is sent to the web server. Are there cookies? Is there a session ID? Or is it enough to send username and password. Jul 16 '20 at 23:23
0

I was able to do this by the following, This was specific to my case, might be different for each scenarios.

First requesting an authoriztion token:

uri_token="https://xxx.yy/TokenURI"
body="grant_type=password&username=user1&password=password1"  
curl -s -X POST -d $body -H "Content-Type: Text" $uri_token

Then I got a list of FIle IDs using the token obtained in the header:

uri_get_list="https://xxx.yy/FileSearchParameter
headers="Authorization: Bearer $token"
curl -s -X GET -H "$headers" $uri_get_list

Then requested the Job ID using the FILE ID to download files:

uri_request=https://xxx.yy/requestapi
body=$(echo \"$FILE_ID\")
headers="Authorization: Bearer $token"
JobID=$(curl -s $uri_request -X POST -H "$headers" -H "Content-Type: application/json" -d [$body]

Then Downloaded the file using the Job ID obtained in the earlier step

uri_download="https://xxx.yy/jobid/"$JobID"
headers="Authorization: Bearer $token"
curl $uri_download -X GET -H "$headers" -o /file/path/to/downloaded

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.