Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have two arrays like this:

A=(vol-175a3b54 vol-382c477b vol-8c027acf vol-93d6fed0 vol-71600106 vol-79f7970e vol-e3d6a894 vol-d9d6a8ae vol-8dbbc2fa vol-98c2bbef vol-ae7ed9e3 vol-5540e618 vol-9e3bbed3 vol-993bbed4 vol-a83bbee5 vol-ff52deb2)
B=(vol-175a3b54 vol-e38d0c94 vol-2a19386a vol-b846c5cf vol-98c2bbef vol-7320102b vol-8f6226cc vol-27991850 vol-71600106 vol-615e1222)

The arrays are not sorted and might possibly even contain duplicated elements.

  1. I would like to make the intersection of these two arrays and store the elements in another array. How would I do that?

  2. Also, how would I get the list of elements that appear in B and are not available in A?

share|improve this question
2  
Use a real programming language, not a shell for this kind of task. –  Stephane Chazelas Dec 12 '13 at 12:53
    
Do you need to retain the order of the elements? If there are duplicated elements (e.g. A and B both contain foo twice), do you need them duplicated in the result? –  Gilles Dec 12 '13 at 22:37
add comment

4 Answers

up vote 3 down vote accepted

comm(1) is a tool that compares two lists and can give you the intersection or difference between two lists. The lists need to be sorted, but that's easy to achieve.

To get your arrays into a sorted list suitable for comm:

$ printf '%s\n' "${A[@]}" | LC_ALL=C sort

That will turn array A into a sorted list. Do the same for B.

To use comm to return the intersection:

$ comm -1 -2 file1 file2

-1 -2 says to remove entries unique to file1 (A) and unique to file2 (B) - the intersection of the two.

To have it return what is in file2 (B) but not file1 (A):

$ comm -1 -3 file1 file2

-1 -3 says to remove entries unique to file1 and common to both - leaving only those unique to file2.

To feed two pipelines into comm, use the "Process Substitution" feature of bash:

$ comm -1 -2 <(pipeline1) <(pipeline2)

To capture this in an array:

$ C=($(command))

Putting it all together:

# 1. Intersection
$ C=($(comm -12 <(printf '%s\n' "${A[@]}" | LC_ALL=C sort) <(printf '%s\n' "${B[@]}" | LC_ALL=C sort)))

# 2. B - A
$ D=($(comm -13 <(printf '%s\n' "${A[@]}" | LC_ALL=C sort) <(printf '%s\n' "${B[@]}" | LC_ALL=C sort)))
share|improve this answer
    
This will only work if your values don't contain \n. –  Chris Down Dec 12 '13 at 16:08
    
@ChrisDown: That's right. I always try to write shell scripts that are properly quoted and handle all chars, but I've given up on \n. I have NEVER seen it in a filename, and a large bunch of unix tools work with \n delimited records that you lose a lot if you try to handle \n as a valid char. –  camh Dec 13 '13 at 10:17
    
I've seen it in filenames when using GUI file managers that do not properly sanitise input filenames that are copied from somewhere else (also, nobody said anything about filenames). –  Chris Down Dec 13 '13 at 11:10
add comment

You can get all elements that are in both A and B by looping through both arrays and comparing:

A=(vol-175a3b54 vol-382c477b vol-8c027acf vol-93d6fed0 vol-71600106 vol-79f7970e vol-e3d6a894 vol-d9d6a8ae vol-8dbbc2fa vol-98c2bbef vol-ae7ed9e3 vol-5540e618 vol-9e3bbed3 vol-993bbed4 vol-a83bbee5 vol-ff52deb2)
B=(vol-175a3b54 vol-e38d0c94 vol-2a19386a vol-b846c5cf vol-98c2bbef vol-7320102b vol-8f6226cc vol-27991850 vol-71600106 vol-615e1222)

intersections=()

for item1 in "${A[@]}"; do
    for item2 in "${B[@]}"; do
        if [[ $item1 == "$item2" ]]; then
            intersections+=( "$item1" )
            break
        fi
    done
done

printf '%s\n' "${intersections[@]}"

You can get all elements in B but not in A in a similar manner:

A=(vol-175a3b54 vol-382c477b vol-8c027acf vol-93d6fed0 vol-71600106 vol-79f7970e vol-e3d6a894 vol-d9d6a8ae vol-8dbbc2fa vol-98c2bbef vol-ae7ed9e3 vol-5540e618 vol-9e3bbed3 vol-993bbed4 vol-a83bbee5 vol-ff52deb2)
B=(vol-175a3b54 vol-e38d0c94 vol-2a19386a vol-b846c5cf vol-98c2bbef vol-7320102b vol-8f6226cc vol-27991850 vol-71600106 vol-615e1222)

not_in_a=()

for item1 in "${B[@]}"; do
    for item2 in "${A[@]}"; do
        [[ $item1 == "$item2" ]] && continue 2
    done

    # If we reached here, nothing matched.
    not_in_a+=( "$item1" )
done

printf '%s\n' "${not_in_a[@]}"
share|improve this answer
    
Exercise: if you interchange A and B, is intersections always the same up to reordering? –  Gilles Dec 12 '13 at 22:39
    
@Gilles If the arrays may contain duplicate elements, no. –  Chris Down Dec 14 '13 at 5:04
add comment

Ignoring efficiency, here is an approach:

declare -a intersect
declare -a b_only
for bvol in "${B[@]}"
do
    in_both=""
    for avol in "${A[@]}"
    do
        [ "$bvol" = "$avol" ] && in_both=Yes
    done
    if [ "$in_both" ]
    then
        intersect+=("$bvol")
    else
        b_only+=("$bvol")
    fi
done
echo "intersection=${intersect[*]}"
echo "In B only=${b_only[@]}"
share|improve this answer
add comment

My pure bash way

As this variables contain only vol-XXX where XXX is an hexadecimal number, there is a quick way using bash arrays

unset A B a b c i                    # Only usefull for re-testing...

A=(vol-175a3b54 vol-382c477b vol-8c027acf vol-93d6fed0 vol-71600106 vol-79f7970e
   vol-e3d6a894 vol-d9d6a8ae vol-8dbbc2fa vol-98c2bbef vol-ae7ed9e3 vol-5540e618
   vol-9e3bbed3 vol-993bbed4 vol-a83bbee5 vol-ff52deb2)
B=(vol-175a3b54 vol-e38d0c94 vol-2a19386a vol-b846c5cf vol-98c2bbef vol-7320102b
   vol-8f6226cc vol-27991850 vol-71600106 vol-615e1222)

for i in ${A[@]#vol-};do
    [ "${a[$((16#$i))]}" ] && echo Duplicate vol-$i in A
    ((a[$((16#$i))]++))
    ((c[$((16#$i))]++))
  done
for i in ${B[@]#vol-};do
    [ "${b[$((16#$i))]}" ] && echo Duplicate vol-$i in B
    ((b[$((16#$i))]++))
    [ "${c[$((16#$i))]}" ] && echo Present in A and B: vol-$i
    ((c[$((16#$i))]++))
  done

This must output:

Present in A and B vol-175a3b54
Present in A and B vol-98c2bbef
Present in A and B vol-71600106

At this state, you bash environment contain:

set | grep ^c=
c=([391789396]="2" [664344656]="1" [706295914]="1" [942425979]="1" [1430316568]="1"
[1633554978]="1" [1902117126]="2" [1931481131]="1" [2046269198]="1" [2348972751]="1"
[2377892602]="1" [2405574348]="1" [2480340688]="1" [2562898927]="2" [2570829524]="1"
[2654715603]="1" [2822487781]="1" [2927548899]="1" [3091645903]="1" [3654723758]="1"
[3817671828]="1" [3822495892]="1" [4283621042]="1")

So you could:

for i in ${!b[@]};do
    [ ${c[$i]} -eq 1 ] &&
        printf "Present only in B: vol-%8x\n" $i
  done

This will render:

Present only in B: vol-27991850
Present only in B: vol-2a19386a
Present only in B: vol-615e1222
Present only in B: vol-7320102b
Present only in B: vol-8f6226cc
Present only in B: vol-b846c5cf
Present only in B: vol-e38d0c94

But this is numericaly sorted! If you want original order, you could:

for i in ${B[@]#vol-};do
    [ ${c[((16#$i))]} -eq 1 ] && printf "Present in B only: vol-%s\n" $i
  done

So you dislay vols in same order as submited:

Present in B only: vol-e38d0c94
Present in B only: vol-2a19386a
Present in B only: vol-b846c5cf
Present in B only: vol-7320102b
Present in B only: vol-8f6226cc
Present in B only: vol-27991850
Present in B only: vol-615e1222

or

for i in ${!a[@]};do
    [ ${c[$i]} -eq 1 ] && printf "Present only in A: vol-%8x\n" $i
  done

for showing only in A:

Present only in A: vol-382c477b
Present only in A: vol-5540e618
Present only in A: vol-79f7970e
Present only in A: vol-8c027acf
Present only in A: vol-8dbbc2fa
Present only in A: vol-93d6fed0
Present only in A: vol-993bbed4
Present only in A: vol-9e3bbed3
Present only in A: vol-a83bbee5
Present only in A: vol-ae7ed9e3
Present only in A: vol-d9d6a8ae
Present only in A: vol-e3d6a894
Present only in A: vol-ff52deb2

or even:

for i in ${!b[@]};do
    [ ${c[$i]} -eq 2 ] && printf "Present in both A and B: vol-%8x\n" $i
  done

will re-print:

Present in both A and B: vol-175a3b54
Present in both A and B: vol-71600106
Present in both A and B: vol-98c2bbef
share|improve this answer
    
Of course, if Duplicate lines are useless, they could simply be dropped. –  F. Hauri Dec 12 '13 at 12:49
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.