Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I ran into another challenge that I am completely stumped on. I have a file filled with data in the following format. There are 3 lines that need to stay together and they have a predictable pattern:

dn: uid=N-NAME-02, ou=data01, dc=data02, dc=data03
uidNumber: 3423
sambaSID: S-1-1-11-1111111-111111111-11111111-12342
<blank line>
dn: uid=N-NAME-03, ou=data01, dc=data02, dc=data03
uidNumber: 3245
sambaSID: S-1-1-11-1111111-111111111-11111111-32212

I need to do some math based the second string of data in the group of 3 and then place the resulting calculation into the third string of the group:

dn: uid=NAME02, ou=data01, dc=data02, dc=data03
uidNumber: (3423 + 2 * 100)
sambaSID: S-1-1-11-1111111-111111111-11111111-342500
<blank line>
dn: uid=NAME03, ou=data01, dc=data02, dc=data03
uidNumber: (3245 + 2 * 100)
sambaSID: S-1-1-11-1111111-111111111-11111111-324700

I figured I could again use AWK for this since AWK can do the math and correct placement of the resulting value. I got the math part on it's own and stored it in a variable

variable1=`awk -F ': ' '/uidNumber:/ { new = $2 * 2 + 1000; print new }' infile`

I was then able to use this variable in another command on the same file to do the replace

awk -F '-' -v variable2=$variable1 '/pattern of string 3 / { print $1"-"$2"-"$3"-"$4"-"$5"-"$6"-"$7"-"variable2 }'

I should mention that the string 3 data is a long string separated by dashes '-'. After the very last dash is where the calculated value needs to be placed.

This all works but has one major flaw... it only works if there is one record in the file. So I don't really have it figured out. I am trying to wrap my head around the most efficient way to do this. I would prefer to stick with bash as I am most familiar with it. Can someone maybe give me a hint as to a way to accomplish this?

share|improve this question

3 Answers 3

up vote 2 down vote accepted

You can use the match() function in awk:

$ cat file
somedata45
somedata47
somedata67

somedata53
somedata23
somedata12

awk '
BEGIN { RS = ""; OFS = "\n"; ORS = "\n\n" }
match($2, /[0-9]+/) { value = (substr($2, RSTART, RLENGTH) + 5) * 100 }
match($3, /[0-9]+/) { $3 = substr($2, 1, RSTART - 1) value }1' file
somedata45
somedata47
somedata5200

somedata53
somedata23
somedata2800

We set the Record Separator to nothing effectively enabling the paragraph mode (separated by blank line). The second line in each paragraph becomes our $2, third line becomes $3 etc. We set the Output Field Separator to newline. Due to the paragraph mode, we also set Output Record Separator to two newlines. The output will give you an extra newline at the end.

We use the match() function to identify the start of number. When a match is found, the function populates two variables for us, RSTART and RLENGTH indicating when the match starts and how long it is. We use those variables to do our calculation and store the result in variable called value. We use the substr function to locate the numbers.

We repeat the same for $3 and this time we use substr function to print up to where our numbers start and replace the number piece with our variable that contains the calculated value from previous line.

Please refer the String Functions in the user guide for more details.


Update based on real data:

Your real data actually makes it a lot simpler.

awk '
/^uidNumber/ { value = $NF } 
 /^sambaSID/ { 
    n = split ($NF, tmp, /-/)
    tmp[n] = ((value + 2)* 100)
    for (i=1; i<=n; i++) { nf = (nf ? nf "-" tmp[i] : tmp[i]) }
    $NF = nf
    nf = ""
}1' file
dn: uid=NAME02, ou=data01, dc=data02, dc=data03
uidNumber: 3423
sambaSID: S-1-1-11-1111111-111111111-11111111-342500

dn: uid=NAME03, ou=data01, dc=data02, dc=data03
uidNumber: 3245
sambaSID: S-1-1-11-1111111-111111111-11111111-324700

You look for the line with uidNumber and capture the last field. When you see a line with sambaSID you split the last field on - and modify the last element to your new calculated value. You then use a for loop to re-assemble your last field.

share|improve this answer

The way I would do this sort of thing in an awk script is by keeping some variable to count the lines. You can do this in a single script.

/^[ \t]*$/ { lineCnt = 0; continue } # this matches blank lines and resets the count
{ lineCnt++ }
lineCnt == 2 { ... strip off your numeric ... }
lineCnt == 3 { ... do whatever you need to with the math ... }

You could make it slightly more efficient by adding 'continue' statements at the end of the blocks for lines 2 and 3, since you know you are done.

share|improve this answer
    
Thank you! counting of the lines makes sense and I was wondering if there was a way to do this. I will work on your suggestion and see if I can make it work! –  bourne May 29 at 12:40
    
One thing to be aware of - variables are untyped and do not need to be declared in advance of use. At first use they have the value of empty when used as string and zero when used as int. I wasn't sure of this myself so tried it out before posting. You can declare a BEGIN block if variables need initialization to non-default values. –  BobDoolittle May 29 at 21:29

It sounds like you're just asking how to combine multiple pattern-action expressions in awk, using the values of variables from one in another - which is very simple e.g.

awk '/pattern2/ { calculate and assign var }; /pattern3/ {do something with var}'

The ; is not strictly necessary in this case but helps comprehension. In an awk script (as opposed to than a one-liner) you'd probably separate the expressions with newlines.

For example, if file is

pattern1
pattern2 3 5
pattern3

then

$ awk '/pattern2/ {var = $2+$3}; /pattern3/ {print; print var}' file
pattern3
8

Also, you may wish to take a look at awk's builtin OFS (output field separator) variable, which lets you format the output more easily e.g. given

pattern1
pattern2-3-5
pattern3-4-6

then

$ awk -F- '/pattern2/ {var = $2+$3}; /pattern3/ {OFS="-"; print $1,$2,$3,var}' file
pattern3-4-6-8
share|improve this answer
    
Thank you very much. When I started working with this solution I realized that the way I represented my data was bad. I think this would work perfectly if I wasn't dealing with multiple different possible delimiters. I will continue to tinker with it! –  bourne May 29 at 12:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.