0

I'm trying to grep through some logs at the url below to look for a specific username. However I'm getting no results, it just runs without stopping.

grepfor="username"
urls=("https://tgstation13.org/parsed-logs/terry/data/logs/2019/01")
while [ ${#urls[@]} -ne 0 ]
do
    content="$(curl -s "$url[0]")"
    echo "$content" | grep "$grepfor"
    delete=($urls[0])
    add=(`echo "$content" | grep -Po '(?<=href=")[^"]*'`)
    urls=( "${urls[@]/$delete}" "{$add[@]}" )
done
  • Don't correct code in the question. It invalidates the answers. – Kusalananda Feb 7 at 19:57
  • Sure thing, thank you – Shard Feb 7 at 20:00
  • I've figured out my next fuckup, the links I'm adding are relative, i need to prepend urls[0] to the start of each. Also the url is missing a / at the end and I'm not following redirects – Shard Feb 7 at 20:00
2

Use "${urls[0]}" for the first element of the urls array, not $urls[0].

To delete the first element of urls and to add the add array to the end, use

urls=( "${urls[@]:1}" "${add[@]}" )

Always quote every expansion, even ${#urls[@]}.

I haven't looked too closely at your curls and greps, but use

printf '%s\n' "$content"

if you want to be sure that you preserve backslashes in the data.

Related:

  • Thanks, I'll implement those changes. However even if I change $grepfor to a string which is on the first url I can't get it to grep out that line – Shard Feb 7 at 19:51
  • @Shard Well, the curl call previous to that line will look for an url ending in [0], so fix that first. Also $url is unset there (it should be "${urls[0]}"). – Kusalananda Feb 7 at 19:52
  • Ahh thanks that was a big fuck up, I've implemented the changed code into the question. The code is now terminating near immediately. I assume new urls found from the first link aren't correctly being added to $urls – Shard Feb 7 at 19:56

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.