To explain what I want I will give an example (My foreign friends sorry it is in Arabic not English :D ) .

Let's take this URL for example. When you open it a blue square will appear after a 5 second timer. Before pressing this blue button, if you inspect it with your browser, you will notice that the href for this button is the same URL followed by a query ?pt= which appears to be some kind of a token that is generated every time.

If you follow this URL (by pressing the button or copy/paste the URL) it will direct you to a page and again, inspecting the page, more specifically to an element written as (تحميل), the href contains the same URL but followed by a query "download_token", which is generated on every time you request the page.

Now you can get the original URL with wget and with some parsing you can get the inner URL (the one with the query ?pt=), but passing the resulting URL from parsing will get the original URL not the page that contains the download_token

  • I tried wget and curl
  • I tried passing the URL in single and double qoute
  • First token contain %XX which is a hex representation of an ASCII. I replaced them with the corresponding characters but it did not work.

How to make wget, curl or any other similar utility to download a URL following its dynamic query? seems like it is parsing the URL with ?pt= somehow wrong, maybe due to the hex characters? The URL with download_token works fine with wget (at least in few trials).

My purpose for all this is to automate the similar process for the rest 1200+ links.

  • 1
    In addition to the GET argument in the URL, the server may be checking for a client side cookie value.... both cURL and wget can use a cookie file exported from your browser - but this would be an extra step... – ivanivan Nov 13 '17 at 21:06
  • Hi @ivanivan , this is interesting, can you please give more details if possible? – 3bdalla Nov 14 '17 at 7:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.