2

I have a file with around 12 000 lines.

pk=91001011964;fck=85;fcv=true;fpi=1;fci=cacf;fmd=1422745568,;fck=83;fcv=decoccm;fpi=1;fci=cacf;fmd=1423070648,;fck=87;fcv=false;fpi=1;fci=cacf;fmd=1422745568,;fck=86;fcv=true;fpi=1;fci=cacf;fmd=1422745568,;fck=35;fcv=54bed48ad84397433fef0a350d65a893;fpi=1;fci=cacf;fmd=1422745568,;fck=83;fcv=travide;fpi=1;fci=cacf;fmd=1422811693,;fck=88;fcv=true;fpi=1;fci=cacf;fmd=1422745568,;fck=84;fcv=10;fpi=1;fci=cacf;fmd=1422745568;vpi=ppi=1;pci=cacf;pmd=1423070648;pos=null;pod=0;pso=null;psd=0;vos=null
pk=91001045537;fck=86;fcv=true;fpi=1;fci=cacf;fmd=1421428238,;fck=85;fcv=true;fpi=1;fci=cacf;fmd=1421428238,;fck=87;fcv=false;fpi=1;fci=cacf;fmd=1421428238,;fck=88;fcv=true;fpi=1;fci=cacf;fmd=1421428238,;fck=84;fcv=10;fpi=1;fci=cacf;fmd=1421428238,;fck=35;fcv=cd9b9ed4fc167d8007dd9544114bc83f;fpi=1;fci=cacf;fmd=1421428238,;fck=83;fcv=demelog;fpi=1;fci=cacf;fmd=1421557377;vpi=ppi=1;pci=cacf;pmd=1421557377;pos=null;pod=0;pso=null;psd=0;vos=null   
pk=91001440737;fck=86;fcv=true;fpi=1;fci=cacf;fmd=1421687966,;fck=85;fcv=true;fpi=1;fci=cacf;fmd=1421687966,;fck=87;fcv=true;fpi=1;fci=cacf;fmd=1421687966,;fck=88;fcv=true;fpi=1;fci=cacf;fmd=1421687966,;fck=84;fcv=10;fpi=1;fci=cacf;fmd=1421687966,;fck=35;fcv=7360072ca14cb5d5578a3902fbe51792;fpi=1;fci=cacf;fmd=1421687191;fck=78;fcv=de724a544277d79c14d19809fe51ab71;fpi=1;fci=cacf;fmd=1421687966,;fck=83;fcv=demelog;fpi=1;fci=cacf;fmd=1421816564;vpi=ppi=1;pci=cacf;pmd=1421816564;pos=null;pod=0;pso=null;psd=0;vos=null

I need to parse the file, look for specific string values in a specific order in the line and print or pick the closest column for that line. If that string value does not exist, print an empty column.

Let's say I have fck=35. if it exists , print the searched string and the closest fmd. For line 3, it would be fck=35;fmd=1421687191. If fck=35 does not exist, add an empty column and that would make fck=35;;

Let's say I'm looking after the for fck=78. If it does exist, print the searched string and the closest fmd. For line 3, I will have fck=78;fmd=1421687966. If fck=78 does not exist, add an empty column and that would make fck=78;;.

The other conditions are the following

For fck=84 look for the next fcv For fck=85 look for the next fcv For fck=86 look for the next fcv For fck=83 look for the next fcv The next fcv for fck=83 can be naismc, decoide, decoccm, travide, travccm or mariccm

I will use line 3 as my example :

fck=35;fmd=1421687191;fck=78;fmd=1421687966;fck=84;fcv=10;fck=85;fcv=true;fck=86;fcv=true;fck=83;fcv=demelog

or (in case fmd or fcv are missing or at least some: this part is no longer line 3, only another line I've not put in my example but I guess it does not matter)

fck=35;;fck=78;;fck=84;fcv=10;fck=85;fcv=true;fck=86;;fck=83;fcv=demelog

I was thinking to do this in awk or sed. But I have a doubt on how powerful it is and for what I have seen so far, it is not helpful. Maybe Python can do the trick but I'm quite unfamiliar with Python.

Any help is welcome.

A mate of mine helped me out with this

awk -F\; -v fck=78 '{for (i=1;i<=NF;i++) if($i=="fck="fck) print "fck="fck";"$(i-1)}' name_of_the_file

As you can see, it is only taking one condition , here fck=78 when I need to have all the conditions above.

Any hints on what I can improve on that tiny script or anything that can make a difference would be great. But another script is possible. I was thinking about Python but I'm not sure on how to do that.

TIA

  • "the closest column" can be assumed to be the one before (like the example code does)? – Hauke Laging Feb 10 '15 at 14:25
  • Hi Hauke, no. Always the closest from left the right so always after. But it means that the fck=35 can be at the end of the line and the fck=78 at the beginning of the line. – Andy K Feb 10 '15 at 14:25
  • The fmd should be on the same string or you do not want to count strings at all? (becouse there are 2 fsk=35 in lines) – Costas Feb 10 '15 at 14:29
  • Hi Costas, it is quite unlikely but if there are two fck=35, I will need both fck=35 for that line. Each fck=35 should be for a specific line. – Andy K Feb 10 '15 at 14:31
  • I tried that but it is not working at all awk -F\; -v fck=35 fck=78 '{for (i=1;i<=NF;i++) if($i=="fck="fck) print "fck="fck" else if($i=="fck="fck) print "fck="fck";"$(i-1)}' file_name – Andy K Feb 10 '15 at 14:34
1

I assume you have a file with the various strings you are looking for. Something like this:

fck=35 fmd
fck=78 fcv
bnv=12 fcv

For each of those, you want to search the file and if a line matches any of the patterns, you want the first fmd value following the matched string. If so, I would do it in perl as follows:

#!/usr/bin/env perl

## Open the list of search patterns.
## The script expects it to be the 1st argument.
open(my $list,"$ARGV[0]");
## Read the file and save the patterns
## in the %pat hash.
while (<$list>) {
    ## remove trailing newlines
    chomp;
    ## separate the search pattern from the target
    my @fields=split(/\s+/);

    ## Save the search pattern and accompanying target in
    ## in the hash (%pats). 
    $pats{$fields[0]}=$fields[1];
}

## Open the list of search patterns.
## The script expects it to be the 2nd argument.
open(my $file,"$ARGV[1]");

## Read the file
while (<$file>) {
    ## split the line on ';' into the @fields array
    my @fields=split(/;/);

    ## This is the string that will be printed for
    ## the current line.
    my $outstring="";
    ## Check each of the search patterns against
    ## each of the fields.
    foreach my $pat(keys(%pats)) {
        ## Add the pattern to the outstring
        $outstring.="$pat;";
        ## save all all 1st fmd values that follow
        ## this pattern. 
        my @matches= ( /$pat.+?($pats{$pat}=[^;]+)/g );
        ## Add this pattern's matches to the output string.
        $outstring.= join(";",@matches) . ";";
    }
    ## Print the output string for this line
    print "$outstring\n";
}

If you save the script above as, for example, parser.pl in your $PATH and make it executable (chmod 755 ~/bin/parser.pl), you can run it like this:

$ parser.pl list.txt file.txt 
bnv=12;;fck=35;fmd=1422745568,;fck=78;;
bnv=12;;fck=35;fmd=1421428238,;fck=78;;
bnv=12;;fck=35;fmd=1421687191 fmd=1111111111;fck=78;fcv=de724a544277d79c14d19809fe51ab71;
0

If you're looking for the closest column and know the delimiter this should be an easy task for grep and sed.

grep -e "fck=35" 

Will return the whole line fck=35 is on. Then pipe it to two seds to get what you want.

grep -e "fck=35" | sed s/.*fck=35;//g | sed s/;.*//g

the first sed replaces everything before fck=35; and fck=35; itself with nothing (removes it), the second sed removes everything after the next delimiter.

But it sounds like you also want to be able to select a specific column (fmd) on the line, for that you need something more like:

grep -e "fck=35" | sed s/.*fmd=/fmd=/g | sed s/[;,].*//g

This removes all the elements before 'fmd=', then removes everything after the next delimiter (or comma, seems like you need to handle that).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.