-1

I have this bash script:

#!/usr/bin/env bash

DOMAINS=( '.com' '.co' )

while read input; do
  for (( i=0;i<${#DOMAINS[@]};i++)); do
  MATCH=$(whois "$input${DOMAINS[$i]}" | grep -oPa '^.*\b(Creation Date)\b.*$')
  if [ $? -eq 0 ]; then
    echo -e "$input${DOMAINS[$i]}\tregistered\t"$(date +%y/%m/%d_%H:%M:%S)"\t$MATCH" | tr '\n' '\t' |& tee --append output/registered.txt
    echo "" |& tee --append output/registered.txt
  else
    echo -e "$input${DOMAINS[$i]}\tavailable\t"$(date +%y/%m/%d_%H:%M:%S)"\t$MATCH" | tr '\n' '\t' |& tee --append output/available.txt
    echo "" |& tee --append output/available.txt
  fi
  done
done < "$1"

input.txt looks like this:

domain1
domain2
domain3

Which is returning this to the console:

$ ./script.sh input.txt
$ domain1.com    registered  Creation date: 15-jan-2015
$ domain1.co     available  Creation date: 15-jan-2015
$ domain2.com     registered  Creation date: 15-jan-2015
$ domain2.co      registered  Creation date: 15-jan-2015
$ domain3.com     registered  Creation date: 15-jan-2015
$ domain3.co      registered  Creation date: 15-jan-2015

How to modify this script, to remove the readed lines from input.txt?

Thank You for your help!

EDIT:

Here is Cas' solution:

#!/bin/bash

DOMAINS='.com .co' # simple, space-separated list of domain suffixes

while read input; do
  for d in $DOMAINS; do
    MATCH=$(whois "$input$d" | grep -oPa '^.*\b(Creation Date)\b.*$')

    if [ $? ] ; then regavail="registered" ; else regavail="available" ; fi

    # what's the `tr` for below? is it really needed?
    # Is $MATCH really going to have more then one line in it?
    out=$(printf '%s\t%s' "$(date +%y/%m/%d_%H:%M:%S)" "$MATCH" | tr '\n' '\t')

    printf '%s\t%s\t%s\n' "$input$d" "$regavail" "$out" |& tee --append "output/$regavail.txt"

    seen+="$input\|"
  done
done < "$1"
seen=$(printf '%s' "$seen" | sed -e 's/\\|$//')
sed -i -e "/^\($seen\)$/d" "$1"

Which is great, but only deleting the lines when the script totally finishes. Is there a way to deleting the lines, when I terminating the script with CTRL+C also?

How I can "trap" (if this is the good term to use here) CTRL+C and send a signal like if the script reached the last line of input.txt? This will delete the already processed lines, or the entire input.txt file, including the unprocessed lines?

UPDATE:

Here is a version where it creates an input-cache.txt file where real-time removing the used lines. If input-cache.txt not present, then input.txt stays intact. So, at the first start, when only input.txt is presented, it will copy everything from this file into input-cache.txt. The script reads from input.txt, but the sed command constantly, real-time removing those lines from the mirrored input-cache.txt. This is when you first time start your script. But what's happening when you halt the script from running with CTRL+C? In this case, everything in input.txt will be reversed back to input-cache.txt, which is now only will have the unprocessed lines, just like in input-cache.txt. One of the drawback is if you give new lines into input.txt, it will instantly removed at the next start. PLUS, the files in the script are hardcoded. I just don't know how to call the input file that I specify with the fire up command: ./script.sh files/input.txt?

Probably there is a better way to create a cache file than re-writing input.txt with it when the script restarted.

#!/bin/bash

# USAGE
# ./script.sh files/input.txt

cat files/input-cache.txt > files/input.txt
cat files/input.txt > files/input-cache.txt

DOMAINS='.com'

while read -r input; do
  for d in $DOMAINS; do
    MATCH=$(whois "$input$d" | grep -oPa '^.*\b(Creation Date)\b.*$')
    if [ $? -eq 0 ]; then
      echo "" |& tee --append files/registered.txt
      echo -e "$input$d\tregistered\t"$(date +%y/%m/%d_%H:%M:%S)"\t$MATCH" | tr '\n' '\t' |& tee --append files/registered.txt
    else
      echo "" |& tee --append files/available.txt
      echo -e "$input$d\tavailable\t"$(date +%y/%m/%d_%H:%M:%S)"\t$MATCH" | tr '\n' '\t' |& tee --append files/available.txt
    fi
  done
  sed -i "/$input/d" files/input-cache.txt
done < "$1"
  • what do you mean by "used line"? – cas Nov 22 '15 at 12:19
  • The line that reads from input.txt. So when reading domain1, then delete it from input.txt. Basically when it ends that file, it needs to become empty. – Lanti Nov 22 '15 at 12:20
3

Add the following line between the fi and the first done:

seen+="$input\|"

Then, after the final done, add the following lines:

seen=$(printf '%s' "$seen" | sed -e 's/\\|$//')
sed -i -e "/^\($seen\)$/d" "$1"

This builds up a regular expression containing all domains already seen and processed in the input file ("$1"), and then deletes them all from that file.

If your version of sed doesn't support the -i (--in-place) option, you can do it with a temporary file instead:

tf=$(mktemp)
sed -e "/^\($seen\)$/d" "$1" > "$tf" && mv -f "$tf" "$1" || rm -f "$tf"

here's a simpler, more readable and maintainable version of your script:

#!/bin/bash

DOMAINS='.com .co' # simple, space-separated list of domain suffixes

while read input; do
  for d in $DOMAINS; do
    MATCH=$(whois "$input$d" | grep -oPa '^.*\b(Creation Date)\b.*$')

    if [ $? ] ; then regavail="registered" ; else regavail="available" ; fi

    # what's the `tr` for below? is it really needed?
    # Is $MATCH really going to have more then one line in it?
    out=$(printf '%s\t%s' "$(date +%y/%m/%d_%H:%M:%S)" "$MATCH" | tr '\n' '\t')

    printf '%s\t%s\t%s\n' "$input$d" "$regavail" "$out" |& tee --append "output/$regavail.txt"

    seen+="$input\|"
  done
done < "$1"
seen=$(printf '%s' "$seen" | sed -e 's/\\|$//')
sed -i -e "/^\($seen\)$/d" "$1"
  • of course, just deleting (rm -f "$1")or erasing (> "$1") the file after the final done is simpler, unless there's another process appending new domain names to the file while it's being processed. – cas Nov 22 '15 at 12:40
  • in the latter case (where another script is appending new domains), commenting out the "used" domains is probably better than deleting them. And then modify your script to ignore comments. And modify the append-domains script to check if a domain exists in commented or uncommented form before appending it. – cas Nov 22 '15 at 12:46
  • Sorry, I putted in the example an older version of the script. I updated my question. – Lanti Nov 22 '15 at 12:54
  • there's nothing in your updated script that requires a change in my answer. however, your script is grossly over-complicated so i'm going to add a simpler version to my answer. – cas Nov 22 '15 at 12:58
  • Unfortunately I getting the following error: : No such file or directory – Lanti Nov 22 '15 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.