Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a task requiring some awk to verify a few records quickly.

Let' say:


Which would be the best way to check for reciprocity only between the A# the B# and the C# and output only the non reciprocal. For instance: the above should output

A2 -> A1
B2 -> B1

A# belong to one group, B# to another and so on. There's no interest in findind any sort of conexion between A# and C# or B#. Instead it is required to keep the search within the group of As, Bs, Cs and so on.

share|improve this question
Either C1 -> A1 should be in the output as well, or I don't understand the question at all. Either way, please edit the question to clarify how to arrive at the desired output. – tripleee Feb 3 '13 at 12:57
This is a bit underspecified... please elaborate... – njsg Feb 3 '13 at 13:12
C and A don't belong to the same group so it should show up in the output. Only the ones belonging to the same group. There are three groups in the example but in reality there are more. – user30422 Feb 3 '13 at 16:12
It would be possible to use the split funtion to create two more fields and evaluate for reciprocity only when fields $1 and $3 are the same – user30422 Feb 3 '13 at 19:51
Please explain exactly what you want. What is your "reciprocity"? What is wrong with the Cs? And don't overconstrain the solution by asking to use certain tools only, it could well be that it is doable in, say, sed, but such a mess that it is better to write it in C. – vonbrand Feb 11 '13 at 4:12

An alternative in Perl; assuming the same interpretation of your question as sg-lecram's :

perl -lne 'tr{ }{}d;      # Remove whitespace in current line
           $lines{$_}++;  # Record the current line in a hash
           END{           # After all lines have been processed
               for(keys %lines){   # Iterate over hash keys
                 #Skip records with different letters:
                 next unless /([a-z]).*\1/i; 
                 ($first,$second)=split /,/; #Read the two fields
                 #Print the record unless its reciprocal is found:
                 print unless exists $lines{"$second,$first"}; 
           }' your_file
share|improve this answer

As far as I got it the following rules lead to your desired output:

  • A1,A2: same letter (i.e. "group") --> looking for A2,A1: not found --> print A2,A1
  • B1,B2: same letter (i.e. "group") --> looking for B2,B1: not found --> print B2,B1
  • C1,C2: same letter (i.e. "group") --> looking for C2,C1: found --> don't print
  • C2,C1: same letter (i.e. "group") --> looking for C1,C2: found --> don't print
  • A1,C1 : different letters (i.e. "groups") --> don't print
  • A1,B1 : different letters (i.e. "groups") --> don't print
  • B1,A1 : different letters (i.e. "groups") --> don't print

So if there was A1,A3 in the list, it should be printed, too:

  • A1,A3: same letter (i.e. "group") --> looking for A3,A1: not found --> print A3,A1

Given I understood that right, you could do something like the following:

awk -F, '

  # skip records that do not consist of exactly two different fields

  # get groups
     g1=substr($1,1,1) # If the groups are not defined as the first...
     g2=substr($2,1,1) # ...character, adjust theses line accordingly.

  # only consider records with matching groups

  # are we looking for the current record?
  ($2 in fst2scd)&&(fst2scd[$2]~FS""$1""FS){

    # remove "reciprocal" pair from the list (assuming record uniqueness -->...
    sub(FS""$1""FS,FS,fst2scd[$2]) # ...consider piping through sort -u first)

    # was that the last record ending with $2 we were looking for (so far)?

      # remove $2 from the list (for now)
      delete fst2scd[$2]

    # this "reciprocal" pair is done

  # if we reach this point, we found a new pair

    # is this the first non-"reciprocal" record starting with $1?
    if(!($1 in fst2scd)){

      # add $1 to the list

    # start looking for a "reciprocal" record

  # after processing all records, we know all non-"reciprocal" records

    # use the same separator for output that was used in input

    # iterate over all starts of records we are still looking for
    for(fst in fst2scd){

      # remove initial and final FS from list entry

      # get all ends of records with the current start we are still looking for

      # iterate over all the ends obtained in the previous step
      for(i in scd){

        # print the non-"reciprocal" records
        print fst,scd[i]
' <<_INPUT_

wich results in the following output:


Note the usage of FS throughout the script to allow the same code to run on e.g. a TSV file that might contain , in an entry.

In case you need further help to understand how this code works and/or improvements/adjustments, feel free to comment.

Also note, that I assumed you have GNU awk (i.e. gawk) running. If this is not the case, I could help you adjusting the code to run in plain awk.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.