0

I'm trying to grep through some logs at the url below to look for a specific username. However I'm getting no results, it just runs without stopping.

grepfor="username"
urls=("https://tgstation13.org/parsed-logs/terry/data/logs/2019/01")
while [ ${#urls[@]} -ne 0 ]
do
    content="$(curl -s "$url[0]")"
    echo "$content" | grep "$grepfor"
    delete=($urls[0])
    add=(`echo "$content" | grep -Po '(?<=href=")[^"]*'`)
    urls=( "${urls[@]/$delete}" "{$add[@]}" )
done
  • Don't correct code in the question. It invalidates the answers. – Kusalananda Feb 7 at 19:57
  • I've figured out my next fuckup, the links I'm adding are relative, i need to prepend urls[0] to the start of each. Also the url is missing a / at the end and I'm not following redirects – Shardj Feb 7 at 20:00
2

Use "${urls[0]}" for the first element of the urls array, not $urls[0].

To delete the first element of urls and to add the add array to the end, use

urls=( "${urls[@]:1}" "${add[@]}" )

Always quote every expansion, even ${#urls[@]}.

I haven't looked too closely at your curls and greps, but use

printf '%s\n' "$content"

if you want to be sure that you preserve backslashes in the data.

Related:

  • Thanks, I'll implement those changes. However even if I change $grepfor to a string which is on the first url I can't get it to grep out that line – Shardj Feb 7 at 19:51
  • @Shard Well, the curl call previous to that line will look for an url ending in [0], so fix that first. Also $url is unset there (it should be "${urls[0]}"). – Kusalananda Feb 7 at 19:52
  • Ahh thanks that was a big fuck up, I've implemented the changed code into the question. The code is now terminating near immediately. I assume new urls found from the first link aren't correctly being added to $urls – Shardj Feb 7 at 19:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.