0

How can I change the record separator of a file? For example, concatenating multiple lines into a single line, by changing record separator from newline to space. Another example is to split a line at its space into multiple lines, by changing record separator from space to newline. What env variables control the record separators? IRS and ORS?

Similarly, how can I change the field separator of a file? Do env variables IFS OFS work with cat?

4
  • Is this all specific to awk?
    – Jeff Schaller
    May 28, 2016 at 15:27
  • No - one could say similar things about perl. May 28, 2016 at 16:26
  • It's just a little misleading to ask about "environment variables" that are unique to specific utilities without mentioning those utilities. Is the question asking which UNIX utilities support env vars named IRS, ORS, IFS, or OFS? Or how to change field / record separators in a specific utility ?
    – Jeff Schaller
    May 28, 2016 at 17:05
  • I have often wondered why no one ever picked up the habit of using ASCII characters FS, GS, RS, and US (0x1C - 0x1F) for their obvious purpose. (As well as SOH, STX, ETX. (0x1-0x3)) May 28, 2016 at 17:14

2 Answers 2

4

Record- and field-separator behavior can be set in certain programs such as awk, and rarely is done via environment variables. Common utilities such as cat do not have this type of feature: only programs which are documented (such as cut) have options for using different record or field separators.

awk is the most flexible of those mentioned, with variables which can be set for both record- and field-separators:

FS
Input field separator regular expression; a <space> by default.
OFS
The print statement output field separator; <space> by default.
ORS
The print statement output record separator; a <newline> by default.
RS
The first character of the string value of RS shall be the input record separator; a <newline> by default. If RS contains more than one character, the results are unspecified. If RS is null, then records are separated by sequences consisting of a <newline> plus one or more blank lines, leading or trailing blank lines shall not result in empty records at the beginning or end of the input, and a <newline> shall always be a field separator, no matter what the value of FS is.

You could set those variables within the script, e.g.,

awk ' BEGIN { RS=" "; } { print; }' < inputfile

to print each word of a file on a new line. You can also assign to any variable using the -v option:

awk -v  RS=" " '{ print; }' < inputfile

Finally, awk also has an option -F (input field separator) which is the same as the FS variable.

Further reading:

4
  • can you give examples using awk
    – Tim
    May 28, 2016 at 15:06
  • For example to replace newlines with spaces (e.g. concatenate lines) using awk: awk 'BEGIN{ORS=" "} {print $0}' your_input_file.
    – ph0t0nix
    May 28, 2016 at 15:12
  • Thanks. How would you change record separator from space to newline
    – Tim
    May 28, 2016 at 15:19
  • Record separator is normally a newline. If you want a field separator to be a newline, you'd use the -F option with a newline as value (awkward but portable). May 28, 2016 at 15:27
2

If you are only concerned with replacing characters, you can have a look at the tr utility (which stands for "translate"). For example, to convert newlines to spaces you can use:

tr '\n' ' ' < your_input_file

Similarly, to translate spaces to newlines use:

tr ' ' '\n' < your_input_file

Note that tr reads from stdin, so you either need to use input redirection (<) like I did in these examples, or pipe your input into it (using |).

Not the answer you're looking for? Browse other questions tagged .