1

I am trying to download a file form a HTTP website from my Linux shell using wget and curl.

Below is the manual GUI process of doing it:

  1. Once I click on the Link-A, it redirects me to a different webpage with a different web Address (Lets call it Link-B) . This webpage has a textbox with a list of items below it.

  2. Then I type a string in the textbox, to filter the list of names below it and it narrows down to only one Item below , then I click on that Item, it redirects me to a different webpage with a different web Address (Lets call it Link-C)

  3. On this webpage I Enter Username and Password and click submit, and it brings me back to Link-A but now it has a list of downloadadble files, where I can download files.

I am trying to download files using command line instead of GUI using tools like curl or wget. I tried using wget with --user and --ask-pass but it only downloads the webpage of LinkB and doesnt get to the file itself. Can anybody suggest a solution, Thanks in advance.

1
  • You have to find out what exactly is sent to the web server. Are there cookies? Is there a session ID? Or is it enough to send username and password. Jul 16, 2020 at 23:23

1 Answer 1

1

I was able to do this by the following, This was specific to my case, might be different for each scenarios.

First requesting an authoriztion token:

uri_token="https://xxx.yy/TokenURI"
body="grant_type=password&username=user1&password=password1"  
curl -s -X POST -d $body -H "Content-Type: Text" $uri_token

Then I got a list of FIle IDs using the token obtained in the header:

uri_get_list="https://xxx.yy/FileSearchParameter
headers="Authorization: Bearer $token"
curl -s -X GET -H "$headers" $uri_get_list

Then requested the Job ID using the FILE ID to download files:

uri_request=https://xxx.yy/requestapi
body=$(echo \"$FILE_ID\")
headers="Authorization: Bearer $token"
JobID=$(curl -s $uri_request -X POST -H "$headers" -H "Content-Type: application/json" -d [$body]

Then Downloaded the file using the Job ID obtained in the earlier step

uri_download="https://xxx.yy/jobid/"$JobID"
headers="Authorization: Bearer $token"
curl $uri_download -X GET -H "$headers" -o /file/path/to/downloaded

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .