I am summarising some test results, so I have this type of text:

FAILED src/path/to/code.test.js
  Test Suite
    🗸 This test passed
    ✕ This test failed

• Test Suite › This test failed
  Expected: Foo
  Received: Bar

  at Object.<anonymous> (src/path/to/code.test.js:31:415) // <-- I'm grabbing this line

I have a sed command sed -nr 's/at.*? \((src.*?\.test.js)\:[0-9]+\:[0-9]+\)/\1/p' test-results.log | sort | unique -c | sort -rg | head -20 that will grab all the paths that start 'at' and end with ':<number>:<number>)' and I get a list of which files have the errors, and the counts.

23 src/path/to/code.test.js  
13 src/path/to/otherCode.test.js  

I have also run the sed command and put the number inside the capture group and I get a list of file + line number (which is handy if one line is causing many issues).

20 src/path/to/code.test.js:31:415  
7 src/path/to/otherCode.test.js:9:26
6 src/path/to/otherCode.test.js:53:5
3 src/path/to/code.test.js:89:793

However, having to switch between these too is a pain.

Is there a way to get this kind of output:

23 src/path/to/code.test.js (20 - 31:415, 3 - 89:793)  
13 src/path/to/otherCode.test.js (7 - 9:26, 3 - 53:5)  

Either by rolling up the sed output, or using a tool like awk (Arghh!).

I've found how to do the basic search with awk on StackOverflow, but I'm not confident in how to even approach storing all the number matches, and then printing them out like the above.

My guess would be something like:

awk '{for(i=1;i<=NF;i++){
    split($1, a, ":")
    fileMatches[a[0]] = fileMatches[a[0]] - a[1]:a[2]
} }' file

But that doesn't seem even close enough to work, given $i==filePattern is not a regex match.

How should I be doing this?

  • You should be doing everything with 1 awk command, please edit your question to provide sample input and the exact expected output given that input which we can simply copy/paste to test with so we can help you with that.
    – Ed Morton
    Sep 14 at 17:24
  • @EdMorton is the input and output I originally posted not sufficient? Sep 22 at 13:28
  • 1
    No, the sample input you provided is a block of text under I have this type of text but then the output is a list of file + line number. If you want to get a list of file names plus line numbers output then you need to provide those files' contents as the sample input otherwise we don't have sample input we can copy/paste to run a potential solution against to see whether or not it produces the expected output you provided. Just showing us an example of what one input file might look like isn't adequate for us to test with.
    – Ed Morton
    Sep 22 at 13:33
  • 1
    You're welcome. The idea when posting a question is that you create a minimal example that demonstrates your problem. Any files you have lying around or can find online would have too much content that's irrelevant to your question and so dissuade many people from trying to help you and make it harder than necessary for the remaining few to do so. See stackoverflow.com/help/minimal-reproducible-example and everything it says there about "code" also apply to sample input/output.
    – Ed Morton
    Sep 22 at 14:06
  • 2
    That's not true. You have 1 example in your question already. Copy/paste another almost identical block to it if you can have multiple blocks in a file. Make a copy of it and modify the relevant lines and that's your 2nd input. Then just add the couple of lines of output given that input. It'd take you less than 10 mins. Good luck either way.
    – Ed Morton
    Sep 22 at 15:56

3 Answers 3

awk '/^[[:blank:]]*at.*\(src.*\.test\.js:[0-9]+:[0-9]+\)/{
    split($0, tmp, /[)(:]/) ## Assuming there is no ), ( and : in the filePath
    fileCnt [tmp[2]]++
    fileErrs[tmp[2], tmp[3]":"tmp[4]]++

    for (fileName in fileCnt){
        printf ("%s %s (", fileCnt[fileName], fileName)
        for(file in fileErrs){
            split(file, tmp, SUBSEP)
                printf("%s - %s", sep fileErrs[file], tmp[2])
                sep=", "
        print ")"
}' infile   ## or infiles*

The fileCnt associated array keeps the count of the files name only; the key of the array is the files' name and values are how many times they seen.
The fileErrs associated array keeps the count of the errors for each fileName; where the key is the combination of the fileName+errNums.

At the END block we are processing over the fileCnt array and print the file count with fileCnt[fileName] and then the fileName itself fileName; then we are processing the fileErrs array and split the key part which then we are comparing first part with fileName to find the indiviadual errors per each file.


That's more of a job for perl:

perl -lne '
  if (/^\h*at\h.*?\((src.*?\.test\.js):(\d+:\d+)\)/) {
  END {
    for $file (sort {$total{$b} <=> $total{$a}} keys %total) {
      my $l = $lines{$file};
      print "$total{$file} $file (" . 
      join(", ", map "$l->{$_} - $_", sort {$l->{$b} <=> $l->{$a}} keys %$l) . ")"

Note that *? as the non-greedy version of * is a perl regexp operator. I know of only one sed implementation which supports it (in addition to -r the non-standard equivalent of -E), but I doubt that's the one you're using. That's ast-open's sed which AFAIK is not shipped by default on any system, not even Illumos which has been known to include or at least consider including ast-open tools other than ksh93. In the GNU implementation of sed (the one that introduced the -r option), .*? is ? (0 or 1 one of preceding atom) applied to .* (0 or more characters), so the same as .* and .+? (0 or 1 of 1 or more) would be the same as .* as well.


Using Raku (formerly known as Perl_6)

~$ raku -ne 'state %h; state $i; ++$i;  \
             if m/ ^ \h* at \h .*?  \( (src .*? \.test \.js) \: (\d+ \: \d+) \) / 
               -> { %h.push: $0 => $i ~"|"~ $1 }; END .say for %h;'  file

Briefly, code is run linewise over the input file using the (non-autoprinting) -ne command line flags. A hash %h is stated. An iterator $i is stated. For each line, the $i iterator is incremented. An if conditional tests for a match to a regex similar-if-not-identical to the Perl5 code posted by @Stephane_Chazelas. The regex will capture the path into $0 and the colon-separated digits into $1.

When a (linewise) match is found it is pushed onto the %h hash: the $0 (path) is used as a key, while the value becomes the line number concatenated (with ~) to the $1 digits portion of the capture, using the code: $i ~"\t"~ $1. Since duplicate keys cannot exist in the hash, different values accumulate per key (i.e path). At the END of reading lines, the %h hash is output.

Sample Input:

#See above. OP's original file was concatenated 5 times, 
#one record was altered (path and digits).

Sample Output (1):

src/path/to/code.test.js => [13|31:415 27|31:415 55|31:415 69|31:415]
src/path/to/othercode.test.js => 41|32:416

To get a leading column of counts, replace the code after END with the following:

for %h.kv -> $k,$v {say $v.elems, "\t", $k => $v};

Sample Output (2):

4   src/path/to/code.test.js => [13|31:415 27|31:415 55|31:415 69|31:415]
1   src/path/to/othercode.test.js => 41|32:416

[Note: Generally say produces arrow-separated key => value output where hashes are concerned. Above you can get tab-separated key-value output simply by changing the final say command(s) to put].


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.