1

how to verify that all words/strings in each line are with the same count

if all words in each line with the same count then syntax will return true and number of counted word

if lines are not with the same count the syntax will return false and count=NA

for example regarding the following example , we will get true and count=5

sdb sde sdc sdf sdd
sdc sdb sde sdd sdf
sdb sdc sde sdf sdd
sde sdb sdd sdc sdf
sdc sde sdd sdb sdf

example regarding the following example , we will get false and count=NA

sdb sde sdc sdf sdd
sdc sdb sde sdd sdf
sdb sdc sde sdf 
sde sdb sdd sdc sdf
sde sdd sdb sdf

another example regarding the following example , we will get false and count=NA

sdb sde sdc sdf sdd
sdc sdb sde sdd sdf
sdb sdc sde sdf 
sde sdb sdd sdc sdf
sde sdd sdb sdf sde 
  • How do you want it to return two things? Print to stdout? – muru Jan 16 '18 at 10:29
  • example - ( counted_words=XX ) ( status=false/true ) – yael Jan 16 '18 at 10:33
  • 1
    Bash is a shell, not (much) of a text processing tool. Bash calls those tools on your behalf. – Jeff Schaller Jan 16 '18 at 10:33
2

Using awk:

awk 'BEGIN { r = "true" } NR == 1 { n = NF; next } NF != n { r = "false"; n = "N/A"; exit } END { printf("status=%s count=%s\n", r, n) }' somefilename

Or as an awk script:

#!/usr/bin/awk -f

BEGIN { r = "true" }

NR == 1 { n = NF; next }
NF != n { r = "false"; n = "N/A"; exit }

END { printf("status=%s count=%s\n", r, n) }

The script will start off with setting r (as in "result") to true (we're assuming it will be true rather than false). Then it initializes n (as in "number") to the number of fields of the first line.

If any of the other lines in the input data has a different number of fields, r is set to false and n is set to N/A and the script exits (via the END block).

At the end, the current values of r and n are printed.


The output of this script will be something like

status=true count=5

or

status=false count=N/A

This could be used with export or bash's declare, or eval:

declare $( awk '...' somefilename )

This would create the shell variable count and status and these would be available in the calling shell.

  • how I can use $count outside from awk shell , in case we want to print the $count in the bash script – yael Jan 16 '18 at 10:50
  • @yael See update for an example – Kusalananda Jan 16 '18 at 10:53
  • regarding the last - we get --> bash: declare: `false,': not a valid identifier – yael Jan 16 '18 at 11:03
  • declare $( awk 'BEGIN { r = "true" } NR == 1 { n = NF; next } n != NF { r = "false"; n = "N/A"; exit } END { printf("%s, count=%s\n", r, n) }' /tmp/file ) -bash: declare: `false,': not a valid identifier – yael Jan 16 '18 at 11:11
  • @yael I removed the comma from the printf statement specifically so that this would work. – Kusalananda Jan 16 '18 at 11:53
1

You can use an associative array to keep the number of each count:

#!/bin/bash
declare -A seen
while read -a line ; do
    (( seen[${#line[@]}]++ ))
done

if [[ ${#seen[@]} == 1 ]] ; then
    echo count=${#seen[@]}
    exit
else
    echo count=NA
    exit 1
fi

Or, you can use external tools to do the work. For example, the following script uses Perl to count the number of words (by its -a autosplit option), sort -u to get unique counts, and wc -l to check whether there's one count or more.

#!/bin/bash
out=$(perl -lane 'print scalar @F' | sort -u)
if ((1 == $(wc -l <<<"$out") )) ; then
    echo count=$out
    exit
else
    echo count=NA
    exit 1
fi
  • what is the meaning of - declare -A seen – yael Jan 16 '18 at 10:39
  • It creates an associative array named "seen". See help declare in bash. – choroba Jan 16 '18 at 10:40
  • where to set the file ? , – yael Jan 16 '18 at 11:02
  • I mean how the script read the file - as /tmp/file – yael Jan 16 '18 at 11:13
  • Just redirect input: script < file – choroba Jan 16 '18 at 11:37
1
if
  count=$(
    awk 'NR == 1 {print count = NF}
         NF != count {exit 1}' < file
  )
then
  if [ -z "$count" ]; then
    echo "OK? Not OK? file is empty"
  else
    echo "OK all lines have $count words"
  fi
else
  echo >&2 "Not all lines have the same number of words or the file can't be read"
fi

Note that in the last part, you can differentiate between different count and can't open the file with [ -z "$count" ] again.

0

Awk solution:

awk 'NR==1{ c=NF; st="true" }
     NR>1 && !(NF in a){ c="NA"; st="false"; exit }{ a[NF] }
     END{ printf "count=%s status=%s\n", c, st }' file
0
#!/usr/bin/perl

use strict; # get perl to warn us if we try to use an undeclared variable.

# get all words on first line, and store them in a hash
#
# note: it doesn't matter which line we get the word list from because
# we only want to know if all lines have the same number of identical
# words.
my %words = map { $_ => 1 } split (/\s+/,<>);

while(<>) {
  # now do the same for each subsequent line
  my %thisline = map { $_ => 1 } split ;

  # and compare them.  exit with a non-zero exit code if they differ.
  if (%words != %thisline) {
    # optionally print a warning message to STDERR here.
    exit 1;
  }
};

# print the number of words we saw on the first line
print scalar keys %words, "\n";
exit 0

(the exit 0 on the last line isn't necessary - that's the default anyway. It's "useful" only to document that the return code is an important part of this program's output.

NOTE: this won't count duplicate words on a line. e.g. sda sdb sdc sdc sdc will count as 3 words, not 5 because the last three words are the same. If that's significant, the hashes should also count the number of times each word appears. Something like this:

#!/usr/bin/perl

use strict;   # get perl to warn us if we try to use an undeclared variable.

# get all words on first line, and store them in a hash
#
# note: it doesn't matter which line we get the word list from because
# we only want to know if all lines have the same number of identical
# words.
my %words=();
$words{$_}++ for split (/\s+/,<>);

while(<>) {
  # now do the same for each subsequent line
  my %thisline=();
  $thisline{$_}++ for split;

  # and compare them.  exit with a non-zero exit code if they differ.
  if (%words != %thisline) {
    # optionally print a warning message to STDERR here
    exit 1;
  }
};

# add up the number of times each word was seen  on the first line  
my $count=0;
foreach (keys %words) {
  $count += $words{$_};
};

# print the total
print "$count\n";
exit 0;

The significant difference is the way that the hashed arrays are populated. In the first version, it just sets the value for each key ("word") to 1. In the second, it counts the number of times each key is seen.

The second version also has to add up the values for each key, it can't just print the number of keys seen.

  • how the perl script read the file as /tmp/file ? – yael Jan 16 '18 at 11:20
  • it takes input either from stdin, or from filename arguments on the command line, or both. e.g. save it as myscript.pl, make it executable with chmod _x myscript.pl and run it as ./myscript.pl /tmp/file. – cas Jan 16 '18 at 11:47
0

To verify that all lines have the same number of words, it's enough to check that all other lines have the same number of words as the first line. So, in Bash, as requested:

#!/bin/bash    
read -r -a arr;
count=${#arr[@]}
while read -r -a arr ; do 
        if [[ $count -ne ${#arr[@]} ]] ; then
                echo "not ok"
                exit 1
        fi
done
printf "ok, count=%d\n" $count

The output is either not ok, or ok, count=N.


This version should work with all POSIXy shells (tested bash, dash, ksh, and zsh):

#!/bin/sh    
[ "$ZSH_VERSION" ] && setopt sh_word_split
set -f
IFS= read -r line;
set -- $line
count=$#
while IFS= read -r line ; do 
        set -- $line
        if [ "$count" -ne "$#" ] ; then
                echo "not ok"
                exit 1
        fi
done
printf "ok, count=%d\n" "$count"

Or just use awk:

awk 'NR==1 { c=NF; } 
     c != NF { print "not ok"; e=1; exit 1 } 
     END { if (!e) printf "ok, count=%d\n", c}' < input
  • seems you forget this - done < file ? ( how the script read the file ? – yael Jan 16 '18 at 11:40
  • @yael, save the script to a file, run bash verifycount.sh < input. – ilkkachu Jan 16 '18 at 11:44
0

I don't know why I'm always triggered by /sed tags, even if counting is involved:

sed '1{s/[^ ]* */#/g;h;d;};G;:a
  s/ [^ ]*\(\n\)#/\1/;ta
  /^[^ ]*\n#$/!{s/.*/status:false count:NA/;q;};$!d
  s/.*/status:true count:0#0123456789+/;G;:b
  s/\(.\)\(#.*\1\)\(.\)\(.*\n\)#/\3\2\3\4/;:o
  s/:+/:10/;s/\(.\)+\(.*#.*\1\)\(.\)/\30\2\3/;to
  s/#.*\n$//; Tb' yourfile.txt

Beautiful, maintainable code, isn't it? Could besimplified, if count<=99.

  • I think you accidentally encrypted your answer, or maybe meant to post it to "Just Another Perl Hacker" forum. ;) – Jeff Schaller Jan 16 '18 at 14:06
  • Especially regular expressions are known to be a WOL (write only language) (-: – Philippos Jan 17 '18 at 6:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.