[Fetching user title]

Seen a post prior of someone posting how to do it by single account

I wish to do this from the range 1-1000 and have it output to >> .txt


x@nine:~#  wget -qO- 'http://www.twitter.com/10' |  perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)(?: - twitter)?\s*<\/title/si' >> cat.txt
x@nine:~#  wget -qO- 'http://www.twitter.com/11' |  perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)(?: - twitter)?\s*<\/title/si' >> cat.txt

x@nine:~# cat cat.txt

Tweets with replies by edo (@10) on Twitter
Tweets with replies by Stephen (@11) on Twitter

Want it to post 1-1000

  • Hello and welcome tho this site! Have a look at the seq command to generate the sequence 1-1000. – Mr Shunz Oct 4 '18 at 13:32
  • use curl instead of wget: curl 'http://www.twitter.com/[1-1000]' | ... – mosvy Oct 4 '18 at 15:19
  • @mosvy im getting an error whilst doing so. - "failed writing body" – Ninth Oct 4 '18 at 20:45
  • try this as an example: curl -sSL 'http://www.twitter.com/[1-12]' | perl -ne 'BEGIN{$/="</html>"}print "$1\n" if /<title.*?>\s*(.*?)(?: - twitter)?\s*<\/title/si'. – mosvy Oct 5 '18 at 17:35

I recommend using a program that actually understands HTML instead of trying to parse it yourself using perl. One problem is that your perl script only parses the first instance of any title tags, not ideal for piping commands.

Below I am using pup to parse the HTML from the web pages, find all title tags, then print their content. Naturally replace 10 and 20 with the range you wanted.

wget -qO - https://twitter.com/{10..20} | pup 'title text{}'

Resulting output

Tweets med svar fra edo (@10) | Twitter
Tweets med svar fra Stephen (@11) | Twitter
Twitter / Konto suspendert
13 (@13) | Twitter
Hazem Emam (@14) | Twitter
Tweets med svar fra 愛と平和の伝道師 (@15) | Twitter
@16 (@16) | Twitter
Sonja Cruz (@17) | Twitter
Twitter / Konto suspendert
One God (@19) | Twitter
Twitter / Konto suspendert
  • Thanks alot for this answer this is what i was looking for! – Ninth Oct 4 '18 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.