1

The following is perl command given by my colleague?

{perl -lne 's/^/\\n/ if $.>1 && /^(\d+)~(\d+)~(\d+)~(\d+)~(\d+)/; printf "%s",$_' filename;} 

He said its for removing blank lines and to check for first five fields for a number.

Thanks in advance.

2

So fields are separated by a tilde sign? What is interesting is the delimiter between the lines. It will be a \n but not a newline.

So as explanation:

the s/^/\\n/ command substitutes the line start with \n but only if the conditions after if are both true:

  1. The line number has to greater than 1 ($.>1)
  2. The line need to contain at least 5 numbers separated by tilde signs.

Afterwards each line (except empty ones (will be avoided by -l parameter) will be printed (printf "%s", $_). The code does not guarantee, that the first 5 fields contain a number!

user@pc:/tmp$ cat inputfile.txt 
1~2~3~4~5

6~7~8~9~0

1~2~3~4~5
1~2~3~4

b~2~3
user@pc:/tmp$ perl -lne 's/^/\\n/ if $.>1 && /^(\d+)~(\d+)~(\d+)~(\d+)~(\d+)/; printf "%s",$_' filename
1~2~3~4~5\n6~7~8~9~0\n1~2~3~4~51~2~3~4b~2~3user@pc:/tmp$

I am not sure, about your curly brackets around the command, you might want to call it from somewhere else. Another (maybe shorter) version which guarantees the existence of 5 fields containing numbers is:

user@pc:/tmp$ perl -ne 'next unless (/^\d+~\d+~\d+~\d+~\d+/); print $_;' inputfile.txt 
1~2~3~4~5
6~7~8~9~0
1~2~3~4~5
user@pc:/tmp$

It will skip all lines not starting with those five fields and therefore skip empty lines.

  • Thank you Frank for the detailed answer. The goal is to remove line breaks between the fields ie whenever a field consists of newline characters, we need to delete the newline characters and make it as a single field. Could you please suggest me? – Arun Kumar Jun 1 '17 at 12:41
  • I am not sure, if I got it right, but it might happen, that a newline is inserted into the fields of a line? Therefore, you would require a reassembly of your lines, right? Is the number of fields constant? Or do you require to have the latest example output just without the newlines having \n instead? – Frank Förster Jun 2 '17 at 6:35
  • Yes Frank this is database file, so the number of fields is constant. In some cases we see new lines are entered by users in CLOB fields. So we need to remove those newlines cause the record delimiter is a newline(unix). Thanks in advance. – Arun Kumar Jun 3 '17 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.