3

Restricting "globbing" grammar to, specifically, suffix-globbing, * characters only, suppose I have:

// foo.txt
foo.*
biz

// bar.txt
bar
foo.bar

And I would like to write:

diff <(sort -u foo) <(sort -u bar)

# alternatively
comm -3 <(sort -u foo) <(sort -u bar)

Such that it returns:

# diff
> bar
< biz

# comm
bar 
biz

Is this possible natively?

5
  • Would regex also be fine, or needs to be glob?
    – pLumo
    Jul 17, 2020 at 14:44
  • @pLumo regex will get the job done for now, but glob I think would be the simplest and most usable.
    – Chris
    Jul 17, 2020 at 14:53
  • If the first file the template and the other one the data, or should wildcard in the second one also be considered? Jul 17, 2020 at 15:01
  • If bar.txt contains both foo.bar and foo.baz, what should be the outcome? Jul 17, 2020 at 15:04
  • @StéphaneChazelas for my use case, internal sorting is ok -- does not have to duplicate diff, just the actual expression there -- and it should be globbing. So you'll need to drop a .* in there for the *s to simulate globbing with a regex
    – Chris
    Jul 17, 2020 at 15:12

1 Answer 1

2

If you want to report all the lines of bar.txt that match none of the patterns in foo.txt, in zsh, that could be:

unique_lines=(${(fu)"$(<bar.txt)"})
unique_patterns=(${(fu)"$(<foo.txt)"})
pattern="(${(j[|])unique_patterns})"

print -rC1 -- ${unique_lines:#$~pattern}

Or all in one go:

print -rC1 -- ${${(fu)"$(<bar.txt)"}:#(${(j[|])~${(fu)"$(<foo.txt)"}})}
  • $(<file) is the ksh-like operator that expands to contents of $file striped of trailing newline characters.
  • ${(flags)param} uses parameter expansion flags to affect the param expansion.
  • f flag, splits on linefeeds (here expands to the list of non-empty lines).
  • u (uniq): removes duplicates. So ${(fu)"$(<foo.txt)"} expands to the unique non-empty lines of foo.txt
  • ${array:#pattern} expands to the elements of $array that don't match the pattern. Here the pattern is constructed as:
  • ${(j[|])unique_patterns} where the elements of $unique_patterns are joined with |. So we end up with a (line1|line2|...) pattern.
  • The ~ in $~pattern causes wildcards to be considered as such upon the variable expansion.

Note that wildcard syntax is that of zsh wildcards. That's affected by a few shell options like extendedglob, kshglob, nocasematch...

In bash, you could do something similar with:

shopt -s extglob
pattern="@($(sort -u foo.txt | paste  -sd '|' -))"
sort -u bar.txt |
  while IFS= read -r line; do
    [[ $line = $pattern ]] || printf '%s\n' "$line"
  done

This time, the syntax is that of bash extglob wildcards, similar to that of ksh88 ones.

Note that the order of the lines in bar.txt ends up being changed.

13
  • can you explain what each of these nestings are doing, and what they are calling? also, am in bash
    – Chris
    Jul 17, 2020 at 15:13
  • @Christopher, in bash, just wrap that code in zsh -c '...', but why would you want to use bash? Jul 17, 2020 at 15:14
  • this seems to be O(n^pattern count), or is it pretty quick?
    – Chris
    Jul 17, 2020 at 15:22
  • 1
    @Christopher, more like in O(n*pattern count) though that depends on how likely the lines are to match the patterns (and the order of patterns as they will be tried in order). See edit for a bash solution, but note that bash pattern matching is notoriously inefficient. Jul 17, 2020 at 15:26
  • yeah, O(n*pattern count) is right. my mistake. but there is no linear solution using the built-in unix tools? (i.e., the compiled binaries)
    – Chris
    Jul 17, 2020 at 15:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .