2

My input files are file_1.txt, file_2.txt, file_3.txt and so on. These files contains data such as

$ head file_*.txt
==> file_1.txt <==
----- Reset Loop 1 -------

Test #1
data

Test #2
data

Test #3

Test #4
data

==> file_2.txt <==
----- Reset Loop 2 -------

Test #1

Test #2
data

Test #3

Test #4
data


==> file_3.txt <==
----- Reset Loop 3 -------

Test #1
data

Test #2
data

Test #3

Test #4

The code I have right now to get filename and the sequence number behind Test only if the data available under each Test from the input file is:

#!/bin/bash

awk '
    FNR==1 {
        testId = ""
        split(FILENAME,f,/[_.]/)
        fileId = f[4]
    }
    testId != "" {
        if (NF) {
            print testId > "1_val.txt"
            print fileId > "2_val.txt"
        }
        testId = ""
    }
    sub(/^Test #/,"") {
        testId = $0
    }
' file_*.txt

The output i get from this code:

1_val.txt

1
2
4
2
4
1
2

2_val.txt

਱਱਱਱਱਱਱਱਱਱ਲਲਲਲਲਲਲਲਲਲਲ਼ਲ਼ਲ਼ਲ਼ਲ਼ਲ਼ਲ਼ਲ਼ਲ਼ਲ਼਴਴਴਴਴਴਴਴਴਴ਵਵਵਵਵਵਵਵਵਵ

There might be some problem in my OS or something else as there are weird characters in my output file. I have thought of an alternative of getting the number in the first line of the input file for the data listed in 1_val.txt.

The code I have for this is awk 'NR==1' file_*.txt but I am not so sure where to insert this particular command in the script.

The expected output:

2_val.txt

1
1
1
2
2
3
3

Edit: This is the exact command I am running to create the output files.

thulasyc > cat data_collect.sh
#!/usr/bin/env bash

awk '
    FNR==1 {
        testId = ""
        fileId = $4
    }
    testId != "" {
        if (NF) {
            print testId > "1_val.txt"
            print fileId > "2_val.txt"
        }
        testId = ""
    }
    sub(/^TX PTP Command #/,"") {
        testId = $0
    }
' "${@:--}"
thulasyc > ./data_collect.sh ptp_log_reset_*.txt
thulasyc > head *_val*
==> 1_val.txt <==
1 
2 
3 
5 
6 
11 
12 
13 
15 
16 

==> 2_val.txt <==
1
1
1
1
1
1
1
1
1
1

The display of the content of the output files:

1_val.txt enter image description here

2_val.txt enter image description here

10
  • This stackoverflow help topic is relevant: How to create a Minimal, Reproducible Example Dec 13, 2021 at 16:39
  • 1
    @Ed Morton sorry that i didnt include the first line in the input file. I have edited it.
    – Basil
    Dec 13, 2021 at 17:49
  • edit your question to copy/paste from your desktop the exact commands you are running to create the output files and then to display the content of the output files.
    – Ed Morton
    Dec 13, 2021 at 18:40
  • @Ed Morton I have edited to show the exact command I am running and the display of the content of output file.
    – Basil
    Dec 13, 2021 at 19:08
  • 1
    sure, thank you so much for the help.
    – Basil
    Dec 16, 2021 at 3:07

2 Answers 2

0
$ cat tst.sh
#!/usr/bin/env bash

awk '
    FNR==1 {
        testId = ""
        fileId = $4
    }
    testId != "" {
        if (NF) {
            print testId > "1_val.txt"
            print fileId > "2_val.txt"
        }
        testId = ""
    }
    sub(/^Test #/,"") {
        testId = $0
    }
' "${@:--}"

$ ./tst.sh file_*.txt

$ head *_val*
==> 1_val.txt <==
1
2
4
2
4
1
2

==> 2_val.txt <==
1
1
1
2
2
3
3
2
  • i still get the weird character in 2_val.txt file.
    – Basil
    Dec 13, 2021 at 18:33
  • 1
    Without access to your desktop (which I'm definitely not asking for) there's no way to debug that, sorry. The script I posted does what you want with the files you provided and whatever problem you're having with weird characters would not happen when you run the posted script on a functioning Unix box using the posted sample input.
    – Ed Morton
    Dec 13, 2021 at 18:38
0

Do you have to use awk? To do it in perl you could start with something like:

#!/usr/bin/perl

use strict;
use warnings;
use diagnostics;

#put your files here:
my @files = ('file_1.txt','file_2.txt');

foreach my $filename (@files) {
    my $test;
    my $number;
  open(my $fh, "<", $filename)
    or die "Can't open $filename ";
        print "$filename:\n";
        while(my $row = <$fh>) {
            if ($row =~ /^Test #.*/){
                $test = 1;
                $number = $row;
                $number =~ s/\D//g;
            }
            elsif ($test and (length($row) > 1) ) {
                print "$number\n";
                $test = 0;  
            }
        }
        close $fh;
}

Edit: Also your question says ' first line of a file ', however the data you posted seems to imply the files contain more than one test so this code takes that into account.

4
  • i really appreciate your help but i dont have perl libraries to execute this code.
    – Basil
    Dec 13, 2021 at 17:57
  • Ahhh OK, I am curious what Unix/Linux system are you on that does not have Perl? Its always been my go-to for this kinda thing becasue even somewhat old systems have a working Perl, and its pretty good backwards compatibility wise. I have found AWK can be different on different systems.
    – turtle
    Dec 13, 2021 at 22:05
  • @turtle FWIW I only recently had perl become available on the systems I work on and that's because I changed jobs, not because perl became available on that system. They're 3B20s used as the admin center behind major telecoms equipment. I've had other systems over the years that also only had the mandatory POSIX toolset, which includes awk but excludes perl, and we see questions asked here a few times a year from people who also only have the mandatory POSIX tools. Awk can be different on different systems but you can write awk portably to work on all Unix systems if that's a concern.
    – Ed Morton
    Dec 15, 2021 at 14:41
  • Fascinating thanks for explaining.
    – turtle
    Dec 16, 2021 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.