I have a bit of a problem making this script work and was wondering if anyone knew what the problem is. It is supposed to take a list of urls as input, scan them for sub directories and then output it to different files.

for i in $(urls.txt); do echo $i; fn=$(echo $i | sed "s/\///g"); echo $fn; dirb $i /usr/share/wordlists/dirb/small.txt > $fn; done

This script does correctly remove slashes for the new file but I get the error (translated from Swedish) that ": no such file or directory". Probably from the > $fn part.

for i in $(urls.txt); do echo $i; fn=$(echo $i | sed "s/\///g"); echo $fn; dirb $i /usr/share/wordlists/dirb/small.txt -o $fn; done

And trying to use dirb's own output I instead get an error like

"OUTPUT_FILE httpwww.website.com
(!) FATAL: Error opening output file: httpwww.website.com"

And some clarification regarding the one liners:
1) loop through a list of urls
2) output the url to terminal
3) generate filename from url using sed
4) output filename to terminal
5) run url through dirb and try to output it to file name

The supposed output file names look fine to me but aren't working. Does anyone know why that is?

  • do you have permissions to write files there? try touch httpwww.website.com – Jeff Schaller Aug 4 '16 at 13:09
  • Yeah I have permission, running as root since I'm lazy. – Latedi Aug 4 '16 at 13:54

Assuming bash or ksh93:

  1. Loop through the list of URLs
  2. Output the URL to terminal

    while read -r url; do
      printf 'URL: "%s"\n' "$url"
    done <urls.txt
  3. Create filename by removing unwanted characters from the URL (changing characters that is not in A-Za-z0-9 to _, removing consecutive _).

  4. Output filename to terminal

    while read -r url; do
      printf 'URL: "%s"\n' "$url"
      outfile="$( tr -s -c 'A-Za-z0-9' '_' <<<"$url" )"
      printf 'File: "%s"\n' "$outfile"
    done <urls.txt
  5. Use dirb and direct output to file

    while read -r url; do
      printf 'URL: "%s"\n' "$url"
      outfile="$( tr -s -c 'A-Za-z0-9' '_' <<<"$url" )"
      printf 'File: "%s"\n' "$outfile"
      dirb "$url" /usr/share/wordlists/dirb/small.txt -o "$outfile"
    done <urls.txt

My version does a bit more sanitization of the filename than just removing forward slashes.

Also, the echo $i will invoke filename globbing (test with i='*').

Since I don't have an example from your urls.txt file, it hard to say anything more about it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.