2

I have a large file that contains lot MQ messages with RFH2 header. Each message in the file is separated by a blank line. Now I need to split this large file into small files each containing single message with RFH2 header.

I tried with the below awk command

awk '{RS=""} {print $0}' inputfile

This prints the first line without the control characters which is not useful. The first MQ RFH header line starts like RFH ^B^C^X^A^Q^C3MQSTR ^D¸ followed by message data. The awk output only prints the text RFH. If the input file has 50 messages after running this command I get 50 files having only the text RFH in them. I am expecting 50 files with RFH2 header and data.

I could not give you the real file input because it has sensitive data. The file starts with

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
.........some text of many lines.....

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
........some text of many lines.....

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
...

And the output files should have

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
.........some text of many lines
  • Can you give some real input? – cuonglm Jun 11 '14 at 10:23
  • Hi Gnouc - I could not give you the real file input because it has sensitive data. The file starts with "RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>.........some text of many lines.....blank line...RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>........some text of many lines.....blank line...RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>...". Let me know if this gives you an idea.. – Raj Jun 11 '14 at 10:32
  • @AvinashRaj Added the comment to question – Raj Jun 11 '14 at 10:50
  • 1
    Does the file contain actual control characters (octal values \002,\003,\030 and so on) or ASCII representations of control characters (literal carat-B, carat-C etc.)? – steeldriver Jun 11 '14 at 12:10
  • 1
    What platform are you on (GNU/Linux? BSD?). Are you sure it is not a matter of your terminal not being able to display the control characters? What happens if you pipe the output to a hex viewer e.g. awk 'BEGIN{RS=""}; {print $0}' file | xxd? BTW you probably need your RS assignment to be in a BEGIN block to achieve the desired splitting. – steeldriver Jun 11 '14 at 13:00
2

Here you go. Input (testfile):

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....

RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....

Code:

awk '{print $0 > "file" NR}' RS='\n\n' testfile

Replace "file" with the name of the files that you want to have. With this example, you'll have:

$ cat file1
RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....

$ cat file2
RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....

$ cat file3
RFH ^B^C^X^A^Q^C3MQSTR ^D¸X<jms>
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
.........some text of many lines.....
1

This is close:

awk '{RS=""} {print $0}' inputfile

However, you need to define the RS variable before awk starts reading the file. Choose one of these:

awk 'BEGIN {RS=""} {print}' inputfile
awk -v RS="" '{print}' inputfile

To view the control characters, pipe the awk output into cat -v

awk -v RS="" 1 inputfile | cat -v

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.