To show all of the URLs in a redirect chain, including the first:
wget -S --spider https://rb.gy/x7cg8r 2>&1 \
| grep -oP '^--[[:digit:]: -]{19}-- \K.*'
Result (tested on Fedora Linux):
https://rb.gy/x7cg8r
https://t.co/BAvVoPyqNr
https://unix.stackexchange.com/
wget options used:
-S
--server-response
Print the headers sent by HTTP servers and responses sent by FTP servers.
--spider
When invoked with this option, Wget will behave as a Web spider, which
means that it will not download the pages, just check that they are there
...
Source: https://www.mankier.com/1/wget
The combination of -S
and --spider
causes wget
to issue HEAD
requests instead of GET
requests.
GNU grep options used:
-o
--only-matching
Print only the matched (non-empty) parts of a matching line, with each such
part on a separate output line.
-P
--perl-regexp
Interpret PATTERNS as Perl-compatible regular expressions (PCREs).
Source: https://www.mankier.com/1/grep
The lines we are interested in look like this:
--2021-12-07 12:29:25-- https://rb.gy/x7cg8r
You see that the timestamp consists of 19 characters comprised of digits, hyphens, colons, and spaces. It is therefore matched by [[:digit:]-: ]{19}
, where we used a fixed quantifier of 19.
The \K
resets the start of the matched portion.
Swap grep with sed
The grep
pipeline stage may be replaced with sed
, if you prefer:
wget -S --spider https://rb.gy/x7cg8r 2>&1 \
| sed -En 's/^--[[:digit:]: -]{19}-- (.*)/\1/p'
Compared to the curl
-based solution...
The curl-based solution omits the first url in the redirect chain:
$ curl -v -L https://rb.gy/x7cg8r 2>&1 | grep -i "^< location:"
< Location: https://t.co/BAvVoPyqNr
< location: https://unix.stackexchange.com/
Furthermore, it incurs a 4354.99% increase in bytes sent to second pipeline stage:
$ wget -S --spider https://rb.gy/x7cg8r 2>&1 | wc -c
2728
$ curl -v -L https://rb.gy/x7cg8r 2>&1 | wc -c
121532
$ awk 'BEGIN {printf "%.2f\n", (121532-2728)/2728*100}'
4354.99
In my benchmarking, the wget solution was slightly (4%) faster than the curl-based solution.
Update: See my curl-based answer for fastest solution.