0

I need to extract text strings from a file and put them in a new file. Each string is always between the same text (tags).

Here's an example (there are hundreds of blocks like this one where I need the data from to be put into one file):

1731 0 obj
<</Page 250/Type/Annot/Subtype/Highlight/Rotate 0/Rect[ 95.4715 347.644 337.068 362.041]/NM(929cd95c-f962-4fa3-b734-2e0e67d7b321)/T(iPad)/CreationDate(D:20160818145053Z00'00')/M(D:20160818145204Z00'00')/C[ 0.454902 0.501961 0.988235]/CA 1/QuadPoints[ 95.4715 362.041 337.068 362.041 95.4715 347.644 337.068 347.644]/Contents(EXAMPLE OF TEXT TO BE EXTRACTED)/F 4/Subj(Highlight)>>
endobj

I need to extract Page 250 and EXAMPLE OF TEXT TO BE EXTRACTED

For the Page 250 example, the relevant tags seem to be:

<</

and

/Type

For the EXAMPLE OF TEXT TO BE EXTRACTED example, the relevant tags seem to be:

/Contents(

and

)/F

Eventually I would like the pages and the corresponding text to be sorted in ascending order, but I could manage that in a spreadsheet.

I tried to use some answers from here, but I did not manage to make it work...

I am most confortable with Unix command line, but I know a little bit of Python and AppleScript

migrated from apple.stackexchange.com Sep 4 '16 at 7:02

This question came from our site for power users of Apple hardware and software.

  • Yes, I put the question on the apple.stackexchange.com site because what I am requesting is on/comes from a Mac. However, the answer(s) seem to work from here too... – pdeli Sep 4 '16 at 21:18
1

With awk:

awk -F/ '$1=="<<" {i=$2; for(j=3;j<=NF;j++) \
         if($j~/^Contents/) split($j,a,"[()]"); print i " _ " a[2]}' file.txt
  • Setting the field separator as /, if the first field is <<, then saving the second field as variable i to be printed later

  • Iterating over the rest of the fields and if any field starts with Contents, then splitting the field on () to create array a, for(j=3;j<=NF;j++) if($j~/^Contents/) split($j,a,"[()]")

  • printing the variable i and second element of array a separating them by _

Example:

% cat file.txt                                                                                                        
1731 0 obj
<</Page 250/Type/Annot/Subtype/Highlight/Rotate 0/Rect[ 95.4715 347.644 337.068 362.041]/NM(929cd95c-f962-4fa3-b734-2e0e67d7b321)/T(iPad)/CreationDate(D:20160818145053Z00'00')/M(D:20160818145204Z00'00')/C[ 0.454902 0.501961 0.988235]/CA 1/QuadPoints[ 95.4715 362.041 337.068 362.041 95.4715 347.644 337.068 347.644]/Contents(EXAMPLE OF TEXT TO BE EXTRACTED)/F 4/Subj(Highlight)>>
endobj

% awk -F/ '$1=="<<" {i=$2; for(j=3;j<=NF;j++) if($j~/^Contents/) split($j,a,"[()]"); print i " _ " a[2]}' file.txt
Page 250 _ EXAMPLE OF TEXT TO BE EXTRACTED
  • As I said in my answer "I'm an awk noob", however we do not know what the actual contents of EXAMPLE OF TEXT TO BE EXTRACTED contains! If the actual text contains a slash / or parenthesis ( and or ), then your awk program will fail, where mine wouldn't. Now I know mine could be written without some of the braces and semi-colons, but as I said "I'm an awk noob" and just learning awk. Mine could be, awk '{sub(/\<\<\//, "") sub(/\/Type.*\/Contents\(/, "\t") sub(/\)\/F.*$/, "")}/Page [0-9]/{print}' file, and it works. Just not sure which is more proper or correct or matters. – user3439894 Sep 4 '16 at 13:48
  • 1
    @heemayl: the command worked as advertised. Thanks. @user3439894: I don't know either what the content of EXAMPLE OF TEXT TO BE EXTRACTED contains, and yes, as you suggest it might just well contain slashes and/or parenthesis... – pdeli Sep 4 '16 at 21:41
  • @pdeli, The fact that it was an unknown is why I choose the method in the manner I did and if it does contain slash or parenthesis then heemayl's method will fail, mine shouldn't because the likelihood of it containing the explicit patten I'm targeting in mine is highly unlikely. That's not to say that heemayl's way isn't interesting, it truly is but it will break if there is even only one slash or parenthesis anywhere in the actual text of EXAMPLE OF TEXT TO BE EXTRACTED. Do the same to my method and it doesn't fail. – user3439894 Sep 4 '16 at 22:15
1

I added your example content to a disk file named file and then show the contents of file using the cat command. Then using awk on file I essentially strip out the parts you're callings tags and printed out the two pieces of data in a tab delimited format. Is this what you're looking for?

$ cat file
1731 0 obj
<</Page 250/Type/Annot/Subtype/Highlight/Rotate 0/Rect[ 95.4715 347.644 337.068 362.041]/NM(929cd95c-f962-4fa3-b734-2e0e67d7b321)/T(iPad)/CreationDate(D:20160818145053Z00'00')/M(D:20160818145204Z00'00')/C[ 0.454902 0.501961 0.988235]/CA 1/QuadPoints[ 95.4715 362.041 337.068 362.041 95.4715 347.644 337.068 347.644]/Contents(EXAMPLE OF TEXT TO BE EXTRACTED)/F 4/Subj(Highlight)>>
endobj
$ awk '{sub(/\<\<\//, "")};{sub(/\/Type.*\/Contents\(/, "\t")};{sub(/\)\/F.*$/, "")};/Page [0-9]/{print}' file
Page 250    EXAMPLE OF TEXT TO BE EXTRACTED
$

What the awk program is doing:

  • /Page [0-9]/ is searching for lines that contain, i.e., 'Page' a space, followed by a number, e.g.,Page 250. I'm assuming that EXAMPLE OF TEXT TO BE EXTRACTED will not contain that pattern. Not that I think it matters although the code can be easily modified to accommodate that.

  • sub(/\<\<\//, "") strips: <</

  • sub(/\/Type.*\/Contents\(/, "\t") is replacing everything between /Type and /Contents( with a tab.
  • sub(/\)\/F.*$/, "") is stripping everything from )/F to the end of the line.

So what's left is what's printed out. The two pieces of wanted data separated by a tab.

I know this doesn't cover all aspects you mentioned, however your other requirements are not clear enough. Is it just one file your need to process or multiplied files. In either case, do you want all extracted data into a single file and the data sorted how exactly, etc.

So if you could clarify things I could probably write a bash script to cover it.

Obviously with the awk program I've provided, you can just redirect the output to an outfile and continue to process it with the sort command. awk can do sorting too however getting the target data in one command line was what I could offer with awk, at this point.

  • The awk command seems to work. Thank you very much. Everything comes from one file only. I redirected the output to an outfile and it worked. Yes, it would be great if the content of the outfile could be sorted (will try in a spreadsheet though). – pdeli Sep 4 '16 at 21:19
  • @pdeli, Yes, it does work, as I wouldn't have posted it if it didn't. ;) It may not be perfectly written, I know it could be written without some of the braces and semi-colons and work without issue however just not sure which is more proper or correct or matters as I'm no expert with awk but learning what I can more now then in past years. – user3439894 Sep 4 '16 at 21:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.