0

How to make a space between filename and the matched line? When you run "grep" for many files, it looks grep uses '-' or ':' (for number or string?) for delimiter between filename and the line. So depending on the line the delimiter can be connected with the first character or seperated with that. How can you make this be consistent for the consecutive work. In the consecutive work, I want to get the numbers on the same column using "awk". The details are like this.

The files can be written like this and the first column looks a 5-digit number to the right-aligned. And the last line of the loop can be different depending on the finish condition of the loop. And I am finding the last line of the loop using the next line of "...optimization bla bla".

case1 of a file

 9969 2020-11-23T14:20:09   6.7433e-02 1.0544e-03 - 2.1166e-03 C 1.2973e-01 - 1.4468e+00 C
...optimization unsuccessful. 

case2 of another file

10072 2020-11-23T14:30:48   6.7384e-02 1.0569e-03 - 2.1148e-03 C 1.2968e-01 - 1.4539e+00 C
...optimization unsuccessful.

I want to get the number at $8, the 8-th value using "awk". Then I run

grep "optimization un" */log.txt -B 1

The output is

NN14/log.txt- 5015 2020-11-21T00:13:37   7.4772e-02 9.5215e-04 - 2.2506e-03 C 1.3664e-01 - 1.2459e+00 C
NN14/log.txt:...optimization unsuccessful. 392.1 min.
NN18/log.txt-10080 2020-11-23T22:41:12   6.5363e-02 9.1362e-04 - 2.4192e-03 C 1.2775e-01 - 1.3058e+00 C
NN18/log.txt:...optimization unsuccessful. 1517.2 min.

Then I want to get the number using

grep "optimization un" */log.txt -B 1 | awk '/T/ {print $8}'

Then the results become

C
1.2775e-01

So, I want to change "NN14/log.txt- 5015" and "NN18/log.txt-10080" into the same format with or without spacing. How can it be done? or is there any other way? I tried using -T for tab but the delimiter attached to the line-side rather than filename-side so it is not effective.

4
  • It looks it is not possible to separate the delimiter and the line. But in my case, it would be possible to print $8 or $9 depending on the the number of line elements. Feb 15 '21 at 3:33
  • Since you aren't using the filename at all, you could do grep -h to exclude it, and then the intermediate data parses consistently for both lines beginning with space and not. Actually you don't need grep here, you can do the whole thing with (one) awk and then there is no intermediate data to be inconsistent. But you apparently don't want that, since you exclude the possibilty of any better approach. Feb 15 '21 at 6:07
  • @dave_thompson_085, I am scanning all the directory with the same filename. Would you explain more exactly? Note that firstly it needs to find a last line in a sequential numbering in the middle of the file. Feb 15 '21 at 6:22
  • Comments are limited in length and formatting; see invalid 'answer' while/if it lasts. Feb 16 '21 at 8:09
1

Meta: too much to explain in comment, but not an answer to the question asked.

I have no clue what you mean by "Note that firstly it needs to find a last line in a sequential numbering in the middle of the file.". The commands you show select the line before a line containing the string "optimization un" if it also contains the letter "T"; in the data you show this line contains some numbers and it may actually be that one or more of these numbers is sequential, but your selection has nothing to do with the number being sequential, or even being a number; your selection is based solely on the string and the letter.

You are grepping multiple files with */log.txt so the default grep output starts each line with the filename and dash or colon. (And this has nothing to do with 'number or string'; it uses colon for the line matching the regexp, and dash for lines before or after the match that are included because of options -B (which you used) or -A.) But since neither colon or dash is whitespace, if the line sometimes begins with a space as yours do this affects the default field parsing by awk which uses whitespace. (awk field parsing can be changed, but for your data format I don't see any change that would work better.)

But you are then selecting in awk a line and field that does not ever depend on or include the filename, so if you omit the filename(s) it works. And grep -h omits the filenames. Try grep -h -B1 "optimization un" */log.txt (options like -h -B1 are supposed to be before the regexp/pattern and filename(s), although GNU grep lets you put them after as an extension) and you should see data more like this:

 5015 2020-11-21T00:13:37   7.4772e-02 9.5215e-04 - 2.2506e-03 C 1.3664e-01 - 1.2459e+00 C
...optimization unsuccessful. 392.1 min.
10080 2020-11-23T22:41:12   6.5363e-02 9.1362e-04 - 2.4192e-03 C 1.2775e-01 - 1.3058e+00 C
...optimization unsuccessful. 1517.2 min.

(although IME you should also have -- lines separating the chunks, which you didn't show) (but if present they will be skipped by the awk and don't matter).

If you now pipe that through your original | awk '/T/{print $8}' it should work.

But as I said you don't need grep (and its formatting) at all, just do:

awk '/optimization un/{print x} {x=$8}' */log.txt

Either of these should solve your problem, but neither answers the question you asked (to put a space between the filename and the matched line) and thus by Stack policy this is not a valid answer and should be deleted.

0

If there is no way to separate the delimiter such as ':' or '-' from the line, there is a bypass to count the the number of fields in awk.

grep "optimization un" */amp-log.txt -B 1 | awk '{if(NF==11) {print $8} else if(NF==12) {print $9} }'

Then it gets:

1.3725e-01
1.3664e-01
1.2968e-01
1.2775e-01

regardless of "

NN12/log.txt- 5014 2020-11-20T23:34:02   7.5423e-02 8.5699e-04 - 2.2759e-03 C 1.3725e-01 - 1.1654e+00 C
NN16/amp-log.txt-10072 2020-11-23T14:30:48   6.7384e-02 1.0569e-03 - 2.1148e-03 C 1.2968e-01 - 1.4539e+00 C

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.