3

I'm trying to pass a flatdb with some names through a script in order to ignore certain things fields and check others. Currently this is as far as I've gotten.

What would I use in my then statement in order to ignore the comment and space lines?

EDIT: Sample of file being read

Bob,Brown,Smith,39
#Alex,Jay,Jones,83

Justin,Michael,White,18
 James,Paul,Weis,54

EDIT: Then I am separating each name

IFS_SAVE=${IFS}
IFS=,

while read value1 value2 value3 value4
do
 echo "First name ${value1}"
 echo "Middle name ${value2}"
 echo "Last name ${value3}"
 echo "Age ${value4}"
done < ${1}

Then I'm trying to ignore blank lines in the file

while read value1 value2 value3 value4
do
if [ -z ${value1} ]
 then
   echo "blank line exists"
 fi
done < ${1}

Then I'm trying ignore commented lines in file

while read value1 value2 value3 value4
do
if [ "${value1:0:1}" = "#" ]
 then
    echo "comment exists"
 fi
done < ${1}

From here down I have been unable to get this to actually work

I'm trying to check if fields start or end with space

while read value1 value2 value3 value4
do
if [[ ${value1} = *[[:space:]]* ]]
then
    echo "space exist"
fi
done < ${1}

EDIT: Ideal output

First name Bob
Middle name Brown
Last name Smith
Age 39
First name Justin
Middle name Michael
Last name White
Age 39
 James,Paul,Wise,54 space exist
2
  • 1
    can you provide us the comment, blank, and also space text sample input so we can try it by our self? we may bash script it now but it may be unintended solution if there is no sample input
    – hanysfa
    Oct 14, 2020 at 3:55
  • @hanysfa i have attached a sample of what the file looks like. thanks
    – dylanw
    Oct 14, 2020 at 4:26

2 Answers 2

8

Use a text processing tool to process text, not hundreds of invocations of inappropriate tools such as read or echo in a loop.

Here, the obvious one to process tabular data is awk:

awk -F, -v OFS='\n' '
  ! /^[#[:space:]]/ {
    print "First name "  $1, \
          "Middle name " $2, \
          "Last name "   $3, \
          "Age "         $4
  }' < you-file

The ! /^[#[:space:]]/ condition matches on lines that don't (!) start (^) with # or whitespace character. You could also do /^[^#[:space:]]/ to match on lines that start with a character other than # or whitespace which in effect would also skip empty lines or add && NF == 4 to skip the lines that don't have exactly 4 fields.

Or to pass the lines with leading or trailing whitespace along untouched as in your expected output:

awk -F, -v OFS='\n' '
  /^[[:space:]]/ || /[[:space:]]$/ {print; next}

  /^[^#]/ && NF == 4 {
    print "First name "  $1, \
          "Middle name " $2, \
          "Last name "   $3, \
          "Age "         $4
  }' < you-file

(Here, we're assuming a POSIX compliant awk; with mawk, replace [:space:] with the hardcoded list of whitespace characters you expect to find at the start as mawk doesn't support POSIX character classes).

Also beware that the carriage return character (aka CR / \r / ^M) is considered a [:space:] character. If the input file was a text file coming from Microsoft OSes, where the line delimiter is CRLF instead of just LF in Unix, then each line would end in a whitespace character. You'd want to run the file through dos2unix first to convert it to Unix format.

2

Since word-splitting happens over IFS and default IFS consists of 'tab/space/newline' and you need to set the IFS to a comma ,, so you would need:

while IFS=, read -r  …

and also change ${value1} = *[[:space:]]* to "${value1}" =~ ^[[:space:]] to pattern-matching that starts with whitespace; and there is no real word meaning it's just how IFS splits on (here it's a single comma).

see also Why is using a shell loop to process text considered bad practice?

3
  • sorry I edited my post to better reflect what I was trying to do. I am currently using IFS to separate names by ,
    – dylanw
    Oct 14, 2020 at 4:37
  • @dylanw that IFS is shell-IFS you set, here IFS is for read's IFS Oct 14, 2020 at 4:39
  • so while IFS=, read value1 value2 value3 value4 do if [ -z ${value1} ] then echo "blank line exists" fi done < ${1} is what you're saying?
    – dylanw
    Oct 14, 2020 at 4:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .