0

I'm having trouble directing the output of awk into individual files in a specified directory. I can successfully do this in the working directory as well as chaining a mv command to the end, but I would like to be able to use only awk to save the files to the specified directory.

For reference, here is a simplified messages.txt that I want to split into individual message files:

messageHeader:
  mh_field1: mh_val1
  mh_field2: mh_val2
messageData:
  md_field1: md_val1
  md_field2: md_val2

messageHeader:
  mh_field1: mh_val1
  mh_field2: mh_val2
messageData:
  md_field1: md_val1
  md_field2: md_val2

messageHeader:
  mh_field1: mh_val1
  mh_field2: mh_val2
messageData:
  md_field1: md_val1
  md_field2: md_val2

I was able to successfully split the messages file into individual messages and output each to a separate file in the working directory using:

awk -v RS= '{print > ("message" NR ".txt")}'

Which creates message1.txt, message2.txt, and message3.txt in the working directory. I can also chain a mv command to the end of this basic command like so to move the output into a specified directory:

out = "$(pwd)/messages"
mkdir -p $out
awk -v RS= '{print > ("message" NR ".txt")}' && mv message*.txt $out

Which creates message1.txt, message2.txt, and message3.txt in ./messages. However, I haven't been able to figure how to save the separate message files to a specified directory only using awk.

For example, I attempted to split the message file and output individual messages to a specified output directory as follows:

out="$(pwd)/messages"
mkdir -p $out
awk -v RS= '{print > ($out "message" NR ".txt")}' messages.txt

Which results in the following error:

awk: cmd. line:1: (FILENAME=- FNR=1) fatal: can't redirect to 'messageHeader:
  mh_field1: mh_val1 
  mh_field2: mh_val2
messageData:
  md_field1: md_val1
  md_field2: md_val2message1.txt' (file name too long)

I found the following related question "Split files using awk and generate the results in another directory" but still wasn't able to get the correct syntax.

out="$(pwd)/messages"
mkdir -p $out
awk -v RS= path=$out '{f=path "message" NR ".txt"; print > f}' messages.txt 

awk: fatal: cannot open file `{f=path "message" NR ".txt"; print > f}' for reading (No such file or directory)
  • 1
    You're just missing a -v I think ... awk -v RS= -v path="$out" – steeldriver Nov 19 '18 at 3:27
  • Did you really type out = "$(pwd)/messages"? Did it give you an error message? – Scott Nov 19 '18 at 4:09
  • @steeldriver Thanks, that was it, I was missing the -v before path=$out. – cneiderer Nov 19 '18 at 4:52
  • @Scott Sorry, that was error in writing the question. I didn't initially have the out defined in that code block and made and error when I was trying to add that in for completeness. – cneiderer Nov 19 '18 at 4:55
0

Per @steeldriver's comment, there was a missing -v, making the correct syntax:

out="$(pwd)/messages"
mkdir -p $out
awk -v RS= -v path="$out" '{f=path "message" NR ".txt"; print > f}' messages.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.