How to extract part of URL from curl response in bash script?

Question

I am trying to extract a part of URL returned in the curl response in a bash script. This will be used in the later stages of the script.

The curl response has this URL:

https://www.example.com/1234/text/111?x=999988

Now, I would like to extract text/111?x=999988 from it. Also, I want to store only the number 111 in a variable. I am trying to do this with

#!/bin/bash

sample_link="https://www.example.com/1234/text/111?x=999988"
sample_extract=${sample_link##*/}
echo "$sample_extract"

But this gives me only 111?x=999988

Can anyone please let me know what am I missing and how to resolve it?

P.S: please let me know if any info is missing

Answer 1

With GNU grep:

$ grep -oP 'https://[^/]+/\d+/\K.*' <<< 'https://www.example.com/1234/text/111?x=999988'
text/111?x=999988

Or

With Perl:

$ perl -nE 'say $& if m@https://[^/]+/\d+/\K.*@' <<< 'https://www.example.com/1234/text/111?x=999988'a
text/111?x=999988a

The regular expression matches as follows:

Node	Explanation
https://	'https://'
[^/]+	any character except: / (1 or more times (matching the most amount possible))
/	/
\d+	digits (0-9) (1 or more times (matching the most amount possible))
/	/
\K	resets the start of the match (what is Kept) as a shorter alternative to using a look-behind assertion: look arounds and Support of K in regex
.*	any character except \n (0 or more times (matching the most amount possible))

---

$ grep -oP 'https://[^/]+/\d+/\w+/\K\d+' <<< 'https://www.example.com/1234/text/111?x=999988'
111

or

$ perl -nE 'say $& if m@https://[^/]+/\d+/\w+/\K\d+' <<< 'https://www.example.com/1234/text/111?x=999988'a
111

The regular expression matches as follows:

Node	Explanation
https://	'https://'
[^/]+	any character except: / (1 or more times (matching the most amount possible))
/	/
\d+	digits (0-9) (1 or more times (matching the most amount possible))
/	/
\w+	word characters (a-z, A-Z, 0-9, _) (1 or more times (matching the most amount possible))
/	/
\K	resets the start of the match (what is Kept) as a shorter alternative to using a look-behind assertion: look arounds and Support of K in regex
\d+	digits (0-9) (1 or more times (matching the most amount possible))

---

To assign to variable, I think you know how to do it =) (your original post show how to do so)

Answer 2

You say you want to extract part of the URL but then you continue by saying that you only want part of that stored in a variable.

Extracting part of a value and doing nothing with it doesn't add any value, unless you meant you want to output it:

response='https://www.example.com/1234/text/111?x=999988'
echo "tail is $(grep -o '[^/]*/[^/]*?.*' <<<"$response")"

Output

tail is text/111?x=999988    

To get the last part into a variable:

response='https://www.example.com/1234/text/111?x=999988'
last=${response%\?*}
last=${last##*/}
echo "last is $last"

Output

last is 111