bash + how to verify all words in each line with the same count

Question

how to verify that all words/strings in each line are with the same count

if all words in each line with the same count then syntax will return true and number of counted word

if lines are not with the same count the syntax will return false and count=NA

for example regarding the following example , we will get true and count=5

sdb sde sdc sdf sdd
sdc sdb sde sdd sdf
sdb sdc sde sdf sdd
sde sdb sdd sdc sdf
sdc sde sdd sdb sdf

example regarding the following example , we will get false and count=NA

sdb sde sdc sdf sdd
sdc sdb sde sdd sdf
sdb sdc sde sdf 
sde sdb sdd sdc sdf
sde sdd sdb sdf

another example regarding the following example , we will get false and count=NA

sdb sde sdc sdf sdd
sdc sdb sde sdd sdf
sdb sdc sde sdf 
sde sdb sdd sdc sdf
sde sdd sdb sdf sde 

Answer 1

Using awk:

awk 'BEGIN { r = "true" } NR == 1 { n = NF; next } NF != n { r = "false"; n = "N/A"; exit } END { printf("status=%s count=%s\n", r, n) }' somefilename

Or as an awk script:

#!/usr/bin/awk -f

BEGIN { r = "true" }

NR == 1 { n = NF; next }
NF != n { r = "false"; n = "N/A"; exit }

END { printf("status=%s count=%s\n", r, n) }

The script will start off with setting r (as in "result") to true (we're assuming it will be true rather than false). Then it initializes n (as in "number") to the number of fields of the first line.

If any of the other lines in the input data has a different number of fields, r is set to false and n is set to N/A and the script exits (via the END block).

At the end, the current values of r and n are printed.

---

The output of this script will be something like

status=true count=5

or

status=false count=N/A

This could be used with export or bash's declare, or eval:

declare $( awk '...' somefilename )

This would create the shell variable count and status and these would be available in the calling shell.

Answer 2

You can use an associative array to keep the number of each count:

#!/bin/bash
declare -A seen
while read -a line ; do
    (( seen[${#line[@]}]++ ))
done

if [[ ${#seen[@]} == 1 ]] ; then
    echo count=${#seen[@]}
    exit
else
    echo count=NA
    exit 1
fi

Or, you can use external tools to do the work. For example, the following script uses Perl to count the number of words (by its -a autosplit option), sort -u to get unique counts, and wc -l to check whether there's one count or more.

#!/bin/bash
out=$(perl -lane 'print scalar @F' | sort -u)
if ((1 == $(wc -l <<<"$out") )) ; then
    echo count=$out
    exit
else
    echo count=NA
    exit 1
fi

Answer 3

if
  count=$(
    awk 'NR == 1 {print count = NF}
         NF != count {exit 1}' < file
  )
then
  if [ -z "$count" ]; then
    echo "OK? Not OK? file is empty"
  else
    echo "OK all lines have $count words"
  fi
else
  echo >&2 "Not all lines have the same number of words or the file can't be read"
fi

Note that in the last part, you can differentiate between different count and can't open the file with [ -z "$count" ] again.

Answer 4

Awk solution:

awk 'NR==1{ c=NF; st="true" }
     NR>1 && !(NF in a){ c="NA"; st="false"; exit }{ a[NF] }
     END{ printf "count=%s status=%s\n", c, st }' file

Answer 5

#!/usr/bin/perl

use strict; # get perl to warn us if we try to use an undeclared variable.

# get all words on first line, and store them in a hash
#
# note: it doesn't matter which line we get the word list from because
# we only want to know if all lines have the same number of identical
# words.
my %words = map { $_ => 1 } split (/\s+/,<>);

while(<>) {
  # now do the same for each subsequent line
  my %thisline = map { $_ => 1 } split ;

  # and compare them.  exit with a non-zero exit code if they differ.
  if (%words != %thisline) {
    # optionally print a warning message to STDERR here.
    exit 1;
  }
};

# print the number of words we saw on the first line
print scalar keys %words, "\n";
exit 0

(the exit 0 on the last line isn't necessary - that's the default anyway. It's "useful" only to document that the return code is an important part of this program's output.

NOTE: this won't count duplicate words on a line. e.g. sda sdb sdc sdc sdc will count as 3 words, not 5 because the last three words are the same. If that's significant, the hashes should also count the number of times each word appears. Something like this:

#!/usr/bin/perl

use strict;   # get perl to warn us if we try to use an undeclared variable.

# get all words on first line, and store them in a hash
#
# note: it doesn't matter which line we get the word list from because
# we only want to know if all lines have the same number of identical
# words.
my %words=();
$words{$_}++ for split (/\s+/,<>);

while(<>) {
  # now do the same for each subsequent line
  my %thisline=();
  $thisline{$_}++ for split;

  # and compare them.  exit with a non-zero exit code if they differ.
  if (%words != %thisline) {
    # optionally print a warning message to STDERR here
    exit 1;
  }
};

# add up the number of times each word was seen  on the first line  
my $count=0;
foreach (keys %words) {
  $count += $words{$_};
};

# print the total
print "$count\n";
exit 0;

The significant difference is the way that the hashed arrays are populated. In the first version, it just sets the value for each key ("word") to 1. In the second, it counts the number of times each key is seen.

The second version also has to add up the values for each key, it can't just print the number of keys seen.

Answer 6

To verify that all lines have the same number of words, it's enough to check that all other lines have the same number of words as the first line. So, in Bash, as requested:

#!/bin/bash    
read -r -a arr;
count=${#arr[@]}
while read -r -a arr ; do 
        if [[ $count -ne ${#arr[@]} ]] ; then
                echo "not ok"
                exit 1
        fi
done
printf "ok, count=%d\n" $count

The output is either not ok, or ok, count=N.

---

This version should work with all POSIXy shells (tested bash, dash, ksh, and zsh):

#!/bin/sh    
[ "$ZSH_VERSION" ] && setopt sh_word_split
set -f
IFS= read -r line;
set -- $line
count=$#
while IFS= read -r line ; do 
        set -- $line
        if [ "$count" -ne "$#" ] ; then
                echo "not ok"
                exit 1
        fi
done
printf "ok, count=%d\n" "$count"

---

Or just use awk:

awk 'NR==1 { c=NF; } 
     c != NF { print "not ok"; e=1; exit 1 } 
     END { if (!e) printf "ok, count=%d\n", c}' < input

Answer 7

I don't know why I'm always triggered by /sed tags, even if counting is involved:

sed '1{s/[^ ]* */#/g;h;d;};G;:a
  s/ [^ ]*\(\n\)#/\1/;ta
  /^[^ ]*\n#$/!{s/.*/status:false count:NA/;q;};$!d
  s/.*/status:true count:0#0123456789+/;G;:b
  s/\(.\)\(#.*\1\)\(.\)\(.*\n\)#/\3\2\3\4/;:o
  s/:+/:10/;s/\(.\)+\(.*#.*\1\)\(.\)/\30\2\3/;to
  s/#.*\n$//; Tb' yourfile.txt

Beautiful, maintainable code, isn't it? Could besimplified, if count<=99.