0

For multiple files, I'm trying to extract columns containing a specific string, but the field separators in the files are different, and the string will appear in a different column depending on the file.

Example line in file 1 (delimiter is :):

sss:string1:rrr

Example line in file 2 (delimiter is ,):

sss,sss,string321,sss,sss

Example line in file 3 (delimiter is '):

example'e123'string2'lol

How could I go about finding "string" in the above three files using awk or other command-line tool?

I know that for file 1 I could perform:

awk -F: '$2 ~ /string/ {print $2}'

and the output would be:

string1

yet I want to do this for multiple files with multiple separators and multiple columns, because do each file individually would be very time consuming? Is it possible to do?

closed as unclear what you're asking by αғsнιη, Kiwy, Archemar, Timothy Martin, phk Mar 24 '18 at 18:26

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • seems what you need is just grep -Eo 'string[0-9]+' multiple-files*, don't you? – αғsнιη Mar 22 '18 at 1:14
  • 1
    You have to know your input in order to tell the computer how to extract it. Since you already know what files have what delimiters, I would just write different scripts that deal with the different files. Why try to combine everything? – Jeff Schaller Mar 22 '18 at 1:29
  • Is it acceptable to assume that a separator is a character of punctuation, whereas the strings you are matching against are (only) alphanumeric? – roaima Mar 22 '18 at 23:32
  • "because do each file individually would be very time consuming" – Why? Huge number of tiny files? – Hauke Laging Mar 24 '18 at 14:47
2

Doing 3 separate calls:

awk -F: '{print $2}' file1
awk -F, '{print $3}' file2
awk -F\' '{print $3}' file3

most likely will be just as fast as something like

awk '
    FILENAME == "file1" && FNR == 1 {FS = ":"; col = 2; $0 = $0}
    FILENAME == "file2" && FNR == 1 {FS = ","; col = 3; $0 = $0}
    FILENAME == "file3" && FNR == 1 {FS = "'\''"; col = 3; $0 = $0}
    {print $col}
' file1 file2 file3

Certainly the former is easier to read and maintain.

As the Zen of Python says, simple is better than complex.

  • 1
    That is a nice approach but it has a serious bug: Assigning a value to FS affects the next line only. Thus you are missing the first line of each file. You can correct that by adding ; $0=$0; after col = ... – Hauke Laging Mar 24 '18 at 14:59
  • 1
    Thanks fixed. With GNU awk it would be a good use for a BEGINFILE block – glenn jackman Mar 24 '18 at 22:31
0

If you want to do it one command then the following will work:

awk -F : '{print $2}' file1 && awk -F : '{print $3}' file2 && awk -F : '{print $3}' file3

That will output:

string1
string321
string2

If you just want to print string and nothing included with it then:

awk -F : '{print substr($2,1,6)}' file1 && awk -F , '{print substr($3,1,6)}' file2 && awk -F \' '{print substr($3,1,6)}' file3

To explain:

substr($2,1,6) prints a substring of the 2nd field starting at the 1st position with 6 of the characters for string. The output will be

string
string
string

Not the answer you're looking for? Browse other questions tagged or ask your own question.