1

I have about 6 items I need extracted from and LDAP base dump file

fullName:
uid:
email:
...

the colon is required for the search, mainly because uid appears in a number of places in the file but the uid: is what I need. I have tried to escape out the colon but any combination I use might upchuck the first search item, but not the second item. Then I have three or four more items to add to the list.

existing output:

gw1:~ # cat dump2.txt|awk "/email:/" && "/fullName:/"
-bash: /fullName:/: No such file or directory

gw1:~ # cat dump2.txt|grep -e "email:" -e "fullName:"
fullName: LDAP Proxy2
fullName: Student Email Administrator
fullName: Richard C. Holly
fullName: Jene E. Brown

expected output:

gw1:~ # cat dump2.txt|awk "/email\:/" print{','} && "/fullName\:/" print{','} && "/gid\:/"
email: proxy2@domain.com , fullName: LDAP Proxy2, gui: 987

Note the csv format.

1
  • You should probably use the ^ beginning-of-line anchor in your regex. I tend to use sed -n /begin/,/end/p for multiline matching, but it's not clear what the separator between records is.
    – o11c
    Oct 2, 2014 at 5:45

2 Answers 2

3

Let's start with a sample input file:

$ cat dump.txt
some: other
email: proxy2@domain.com
fullName: LDAP Proxy2
gui: 987
other: thing

To extract the desired lines and reformat them as CSV:

$ awk '/email:|fullName:|gui:/{s=s", "$0} END{print substr(s,3)}' dump.txt
email: proxy2@domain.com, fullName: LDAP Proxy2, gui: 987

How it works:

  • /email:|fullName:|gui:/{s=s", "$0}

    This is a condition followed by a statement. The condition is the regular expression /email:|fullName:|gui:/ which is true only for lines that contain one of the strings that you are looking for. In regular expressions, the | symbol means logical-or.

    If the condition is true, then the statement is executed. The statement causes a comma, a space, the current line to be appended to the variable s.

    awk implicitly loops through all lines in the file and the above is performed on each line.

  • END{print substr(s,3)}

    When we are done reading through the input file, we want to print all but the first two characters of s. The first two characters are ", " which are superfluous. The function substr is used to remove them.

Sample output

Using the file that you supplied at paste.ee and selecting for the fields that you specified in the comments ("mail:", "fullName", and "uid"), I obtain:

$ awk '/mail:|fullName:|uid:/{s=s", "$0} END{print substr(s,3)}' dump.txt
mail: pgroce@midland.edu, uid: pgroce, fullName: Patti K. Groce

All fields are found.

Input with blank fields

As per the comments, consider dump3.txt which has some blank entries at the end:

$ cat dump3.txt
other: thing
mail: pgroce@midland.edu
uid: pgroce
fullName: Patti K. Groce
mail:
mail:
Other: Thing
mail:
$ awk '/mail:|fullName:|uid:/{s=s", "$0} END{print substr(s,3)}' dump3.txt
mail: pgroce@midland.edu, uid: pgroce, fullName: Patti K. Groce, mail:, mail:, mail:

As you requested, the empty entries are printed as empty entries.

13
  • it still only prints out 2 out of 3 fields. gw1:~ # awk '/email:|fullName:|uid:/{s=s", "$0} END{print substr(s,3)}' dump2.txt uid: NFAUUser, uid: NFAUUser, DFS-Volume-Guid:: QORfRvqx1w, uid: LDAPProxy2, fullName: LDAP Proxy2, uid: nimspostmaster, fullName: Student Email Administrator, uid: rholly, fullName: Richard C. holly
    – brad
    Oct 2, 2014 at 5:38
  • also, to insert a new line, just include '\n' in the print?
    – brad
    Oct 2, 2014 at 5:39
  • @brad Hmmm. To make things perfectly clear, would you put (a) your sample input (the one which you said didn't work) and (b) your desired output into the question? (Comments are quite poor for complex formatting.)
    – John1024
    Oct 2, 2014 at 5:52
  • Sample data paste.ee/p/BC6xk Sample output is list above.
    – brad
    Oct 2, 2014 at 6:09
  • @brad Please clarify: (1) The desired output contains gui: 987 but no such line appears in the paste.ee file, (2) The desired output contains "email: proxy2@domain.com" but the paste.ee file has no such line. Did you mean "emailAddress:"?
    – John1024
    Oct 2, 2014 at 6:18
0

You can parse the whole file into a Perl hash and print out the required fields (given as input arguments after the file name):

perl -nle '
    BEGIN{
        $input_file = shift;
        $required_fields = shift
    }
    my ($field,$val) = split/:/;
    next unless defined $field; #Skip lines with no field names
    $fields{$field} = $val;
    END{
        print join ",",@fields{split/,/,$required_fields}
    }' your_file 'email,fullName,gui'
6
  • I put your code into a .pl file and ran it with perl -nle and it just hung. sprunge.us/AagB
    – brad
    Oct 2, 2014 at 6:50
  • @brad That's because the code was never meant to be put into a file. Please try it: paste the above as-is on the command line (note that I have updated it to fix a minor bug) and if it works, I can modify it for you into a version that would fit in a file.
    – Joseph R.
    Oct 2, 2014 at 6:53
  • I did try that and it didn't work out to well paste.ee/p/311yI
    – brad
    Oct 2, 2014 at 6:56
  • @brad As I've said, please take the extra comma I added into account: print join ',',...
    – Joseph R.
    Oct 2, 2014 at 7:02
  • same error with print join ',' as with print join ',' ,
    – brad
    Oct 2, 2014 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.