Use cookies in curl to set language

Question

I need to curl | grep some data from a webpage but after setting language to English. In a browser, I can do this by clicking on a link.

I first get the SelectCulture page and save the cookie in a file, then I use it to get the page I need:

#!/bin/bash

#tmp_file="$(mktemp)"
tmp_file="cookies"

curl -s \
    --location \
    --cookie "$tmp_file" \
    --cookie-jar "$tmp_file" \
    --user-agent Mozilla/4.0 \
    --data-urlencode "ReturnUrl=http://it.bca-europe.com/Default.aspx" \
    "http://it.bca-europe.com/Home/SelectCulture/en-GB-BDIT" | egrep "Ospite|Guest"

curl -s \
    --location \
    --cookie "$tmp_file" \
    --user-agent Mozilla/4.0 \
    "http://it.bca-europe.com/Default.aspx" | egrep "Ospite|Guest"

The problem is that on first run, when the cookies file does not exist yet, somehow the language doesn't change (you get italian Ospite instead of Guest), whereas the script works from the second run.

This, for example, avoids the need to use a temporary file instead of a static file. Moreover, weirdly, I can't get it to work in a single run.

Any advice?

Answer 1

A browser usually sends a list of accepted languages with every request. CMSes often use this when no cookie is set. (In theory, the cookie is just a fallback and the language settings are correct.) This results in <span>Hello Guest</span> for me:

curl -s  --location -H 'Accept-Language: en' "http://it.bca-europe.com/Home/SelectCulture/en-GB-BDIT" | egrep "Ospite|Guest"