1

everyone. I have two files: ports.lst and master.tbl

ports.lst looks like this:

hawaii-P1
hawaii-P2
hawaii-P3
losangeles-P1
losangeles-P3

master.tbl looks like this:

#Site 1 Honolulu
servername HAWAII-A hawaii-P1 InitFileA OutFileA otherfields
servername HAWAII-A hawaii-P2 InitFileA OutFileA otherfields
#servername HAWAII-A hawaii-P3 InitFileA OutFileA otherfields
servername HAWAII-A hawaii-P4 InitFileA OutFileA otherfields

#Site 16 Dallas
servername DALLAS-A dallas-P1 InitFileA OutFileA otherfields
servername DALLAS-A dallas-P2 InitFileA OutFileA otherfields
servername DALLAS-A dallas-P3 InitFileA OutFileA otherfields

#Site 8 L.A.
#servername LOSANGELES-A losangeles-P1 InitFileA OutFileA otherfields
servername LOSANGELES-A losangeles-P2 InitFileA OutFileA otherfields
#servername LOSANGELES-A losangeles-P3 InitFileA OutFileA otherfields

I need to search the master.tbl file for each port listed in ports.lst, and replace "InitFileA" and "OutFileA" leaving the file looking like this:

#Site 1 Honolulu
servername HAWAII-A hawaii-P1 InitFileB-hawaii-username-ALPHA-password OutFileB-hawaii-username-ALPHA otherfields
servername HAWAII-A hawaii-P2 InitFileB-hawaii-username-ALPHA-password OutFileB-hawaii-username-ALPHA otherfields
#servername HAWAII-A hawaii-P3 InitFileB-hawaii-username-ALPHA-password OutFileB-hawaii-username-ALPHA otherfields
servername HAWAII-A hawaii-P4 InitFileA OutFileA otherfields

#Site 16 Dallas
servername DALLAS-A dallas-P1 InitFileA OutFileA otherfields
servername DALLAS-A dallas-P2 InitFileA OutFileA otherfields
servername DALLAS-A dallas-P3 InitFileA OutFileA otherfields

#Site 8 L.A.
#servername LOSANGELES-A losangeles-P1 InitFileB-losangeles-username-ALPHA-password OutFileB-losangeles-username-ALPHA otherfields
servername LOSANGELES-A losangeles-P2 InitFileA OutFileA otherfields
#servername LOSANGELES-A losangeles-P3 InitFileB-losangeles-username-ALPHA-password OutFileB-losangeles-username-ALPHA otherfields

Here is where I am, right now, but it fails--clearly.

awk 'NR==FNR{z[$0];next}{if ($3 in z && $4 == "InitFileA"){ c=(echo $3| awk -F '-' {print $1});$4="InitFileB-"c"-username-ALPHA-password";$5="OutFileB-"c"-username-ALPHA"}}1' ports.lst master.tbl > output.tbl

I have also tried:

awk 'NR==FNR{z[$0];next}{if ($3 in z && $4 == "InitFileA"){ c=$3; sub(/-.*/, "", $c);$4="InitFileB-"c"-username-ALPHA-password";$5="OutFileB-"c"-username-ALPHA"}}1' ports.lst master.tbl > output.tbl

I have been pulling my hair out over this. Can anyone here offer any insight as to what I'm doing wrong?

1

You have the right basic idea of splitting the task into two rounds, but then you go and invoke awk within an awk rule.. that's where I stopped reading it; it is just too complicated a way to solve such a simple problem.

Consider this awk snippet:

awk 'BEGIN {
         RS = "[\t\v\f ]*(\r\n|\n\r|\r|\n)";
         FS = "[\t\v\f ]+"
     }

     FNR==1 {
         file++
     }

     /^#/ {
         next
     }

     file==1 {
         port[$1] = $1
     }

     file>=2 && ($3 in port) {
         base = $3;
         sub(/-[^-]*$/, "", base);
         $4 = "InitFileB-" base "-username-ALPHA-password";
         $5 = "OutFileB-" base "-username-ALPHA";
     }

     file>=2 {
         printf "%s\n", $0
     } ' ports.lst master.tbl

Note: I added the necessary semicolons, so you can write all of the above in a single line.

If you run the above with the example input files, you get

losangeles-P1
losangeles-P3
servername HAWAII-A hawaii-P1 InitFileB-hawaii-username-ALPHA-password OutFileB-hawaii-username-ALPHA otherfields
servername HAWAII-A hawaii-P2 InitFileB-hawaii-username-ALPHA-password OutFileB-hawaii-username-ALPHA otherfields
servername HAWAII-A hawaii-P4 InitFileA OutFileA otherfields

servername DALLAS-A dallas-P1 InitFileA OutFileA otherfields
servername DALLAS-A dallas-P2 InitFileA OutFileA otherfields
servername DALLAS-A dallas-P3 InitFileA OutFileA otherfields

servername LOSANGELES-A losangeles-P2 InitFileA OutFileA otherfields

The BEGIN rule just sets universal newline support, in case the files were transferred from some other system (Windows, for example) with a different newline encoding.

The FNR==1 rule is used to update the file variable, so that it reflects the file being processed (1 for first, 2 second).

The /^#/ { next } rule skips all lines beginning with a hash mark. They are comments, so they don't need to be kept. We could also add a rule /^[\t\v\f ]*$/ { next } to skip all empty lines, if you wanted to compact the output file.

The file == 1 { port[$1] = $1 } rule adds all first fields in the first file to associative array port. The value assigned (= $1) does not matter, so we could actually just use = 0 here.

The file >= 2 && ($3 in port) rule applies to second and any following files, and is executed if the third field matches one of the keys in the associative array port. (The values do not matter; only the keys are checked.) In other word, this rule is only applied when the third field is one of the keys specified in the port list.

The third field is copied to a variable base -- this matched one of the keys in port[] --, and everything after the last - is removed using sub(). Then, we modify the fourth and fifth fields. Note that in awk, there is no string concatenation operator; we just state the strings next to each other. In other words, ("foo" a "bar") is one string, consisting of "foo", immediately followed by the value of variable a converted to a string, immediately followed by "bar".

The final rule prints the (possibly modified) record, but ensures a \n newline is used. Only the records in the second and subsequent files are used.

Now, if ports.lst contained the respective usernames and passwords, I'd slightly modify the above (maybe three lines changed?), but I hope you can see the overall approach.

  • imho - adding some of your comments on what the code is doing, how it does it, etc. would enhance this answer greatly. Scrolling up/down - relocating where I am at while reading - also becomes "I give up trying to understand what is presented as 'a simple problem'" – Michael Felt Nov 25 '16 at 8:28
  • @MichaelFelt: Copy the code, paste into a no-frill text editor, use the window controls to keep the editor window on top, then position your text editor and browser window so you can read both at the same time. I guess I could write an awk tutorial using this approach -- examining real world problems, then presenting an awk solution to each, dissecting the solution line by line --, but I'm not going to do just because one person asked me to. It is not cost-effective enough. – Nominal Animal Nov 25 '16 at 16:26
0

I seem to have figured out the answer. My trouble appears to have been in the placement of the dollar signs, near the 'c' variable. To wit, this worked:

awk 'NR==FNR{z[$0];next} { if ($3 in z && $4 == "InitFileA"){ c=$3; sub(/-.*/, "", c);$4="InitFileB-"c"-username-ALPHA-password";$5="OutFileB-"c"-username-ALPHA"}}1' ports.lst master.tbl > output.tbl

Now, as to WHY it worked, I'm afraid I cannot explain. I was literally grasping at straws, and getting desperate. I had resorted to stripping away layers of the command, until the error disappeared, and then slowly adding commands and tinkering until each one worked.

  • 1
    awk variables don't use $ before every variable. The one exception is the field variables like $1 and $2, to access a variable called VAR you would just put VAR not $VAR. – Zachary Brady Nov 22 '16 at 20:43
0

I do not do enough awk scripting to just type the statements in, but I would look for a way to use the structure of 'master' file and have multiple blocks.

Conceptual Solution Plan

BEGIN
{
#  get it ready ...
}

/^$/
{
# maybe just skip lines
# otherwise potential post processing for #Site XX Name
}

/^#Site/
{
# initialize processing for a new site
}

{
# default block for the site processing 'input'
}

FINISH
{
# 'master' is parsed - now fill in the blanks using 'port'
# i.e, start of second pass to complete the work
}

I know there are no awk commands in this - but I am also curious as to whether awk specialists see this as a generic approach to awk solutions. Too often I am turned off to using awk because I get lost in 'one-liner awk statements', i.e., one command that is going to process all lines regardless of whether there are recurring blocks of input.

And, even should this prove to be ridiculous approach - I hope the comments will enlighten me (and others) such that I get better with awk. Thanks!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.