1

Why does a brace expansion behave differently than wildcard in combination with paste?

Example: Assume we have multiple folders, each containing the same-structured tsv and want to create a 'all.tsv' containing the 5th row of each of those. The two commands behave differently:

paste -d, <(cut -d$'\t' -f5 {test,test1,test2}/example.tsv) > all.tsv

vs

paste -d, <(cut -d$'\t' -f5 test*/example.tsv) > all.tsv

The first creates a tsv with 3 columns as expected, the second one creates a single columned tsv with the values beneath each other.

My problem is that list of folders is arbitrarily big, potentially quite long and not sequential.

Is there a way to achieve the same behavior as brace expansion with wildcard without moving to a bash script and iteration over the folders?

Using GNU bash

  • 2
    "The first creates a tsv with 3 columns as expected" I'm struggling to see how: the cut command will produce a single concatenated stream to standard output, and paste will read that as a single "file" from the process substitution, surely? – steeldriver Aug 9 '19 at 12:06
  • 1
    it seems as if the behaviour you're expecting is <(cut -d$'\t' -f5 {test,test1,test2}/example.tsv) will expand to <(cut -d$'\t' -f5 test/example.tsv) <(cut -d$'\t' -f5 test1/example.tsv) <(cut -d$'\t' -f5 test2/example.tsv) – muru Aug 9 '19 at 12:10
  • @iR0Nic, Is the output from cut (only) different in those two cases? Can you add a sample set of input files and the resulting output to demonstrate this? The only difference between those two I can see is that the braces expand to particular set of three filenames, and the glob expands to whatever files there exist. And yes, cut would in effect concatenate the input files, so paste doesn't seem to do much here... – ilkkachu Aug 9 '19 at 12:24
  • @muru Yes, indeed. And it seems to be only occuring on Mac OS' terminal. Ran the same command on a Ubuntu and Ubuntu subroutine for windows and both behaved like the wildcard example. But this behavior is exaclty what i need. But i guess there is no easy one-liner to enforce this? – iR0Nic Aug 9 '19 at 13:40
2

The behaviour you're looking for is a bug that was fixed between bash-3.2 (the version found on macOS), and bash-4.0. From the CHANGES file:

rr. Brace expansion now allows process substitutions to pass through unchanged.

For a one-liner, you might try awk:

awk -F '\t' {FNR != NR {exit} {out=$5; for (i = 2; i < ARGC; i++) {getline < ARGV[i]; out = out "," $5}; print out}' test*/example.tsv

Explanation:

FNR != NR { exit }                # Exit after first file is finished.

{
  out=$5;                         # save the first file's fifth field
  for (i = 2; i < ARGC; i++) {    # loop over the remaining arguments (filenames).
    getline < ARGV[i];            # Read in the next line from i-th file.
    out = out "," $5              # save fifth field of the line just read
  };
  print out                       # print saved columns.
}
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.