6

I have a file that looks like

$ cat IP
10.3.1.1
10.4.1.1
10.6.3.1
10.19.4.2
10.22.3.4

How do I make it look like:

$ cat IP 
10.3.1.1, 10.4.1.1, 10.6.3.1, 10.19.4.2, 10.22.3.4
0

13 Answers 13

17

With perl:

perl -pe 's/\n/, / unless eof' IP

Add the -i option to edit the file inplace. Or -i.orig to edit it in place but keep the original as IP.orig.

With the zsh shell, you can also load those IPs into an array with:

ips=( ${(f)"$(<IP)"} )

And then write them back joined with commas with:

print -r -- ${(j[, ])ips} > IP

Contrary to the perl one which processes one line at a time, that one loads the file whole in memory.

Two other differences:

  • empty lines are discarded
  • a newline character is always added at the end even if the original file didn't contain any like for an empty file or a file whose last line was not properly delimited.
7

With the GNU implementation of sed:

sed -z 's/\n*$//;s/\n/, /g;s/$/\n/' IP
4
  • The output is missing a trailing newline character though, which you could add back with $s/$/\n/ Commented Dec 14, 2023 at 13:32
  • i have changed that .... Commented Dec 14, 2023 at 13:39
  • Thanks, for posting an answer. This should be an answer and not a comment. Please never post answers in comments, we need them as answers so they can be voted on.
    – terdon
    Commented Dec 14, 2023 at 13:48
  • 3
    s/$/\n/ adds a newline at the end of every (NUL-delimited here with -z) record, $s/$/\n/ adds it only to the end of the last record (would only make a difference if the file contained NUL characters). Commented Dec 14, 2023 at 13:56
7

Perhaps

perl -e 'print join(", ", map { chomp; $_ } <>), "\n"' IP

Or

awk '{ printf "%s%s", c, $0; c = ", " } END { print "" }' IP

Or for a quick and dirty hack, subject to constraints on the number of lines in your file (potentially as few as 8000), and bearing in mind that xargs wasn't really designed for this

xargs <IP | sed 's/ /, /g'

All these produce the same output from your example data set:

10.3.1.1, 10.4.1.1, 10.6.3.1, 10.19.4.2, 10.22.3.4

You can then redirect that output to a temporary file and then overwrite the original. Here, replace command_from_above with your choice of any command (here in this answer or anyone else's) that writes the changes to stdout rather than to the original file:

command_from_above >"IP.$$.tmp" && mv -f -- "IP.$$.tmp" IP
rm -f -- "IP.$$.tmp"
3
  • xargs run /bin/echo in batches for each word in the input, so that will fall apart if there are many lines or as soon as the lines contain anything but single IP addresses. Commented Dec 14, 2023 at 17:30
  • 1
    The example file gives no indication that there might be anything other than a single IP address or line. Neither does it give any indication of file size. (That could work either in my favour or against it, I agree, @StéphaneChazelas) Commented Dec 14, 2023 at 18:52
  • xargs (without the -r/-0 GNU extensions) is bad enough when used for what it's meant to (pass arguments to a command when read from a stream), I'd rather we stop advertising its use for something it's not designed for and where it's very bad at: doing text processing. While the OP might get away with using it with their dataset, other people may come here with different datasets and get the impression that it can be used to join lines with spaces. You could at least point out the many limitations. Commented Dec 17, 2023 at 8:40
6

With bash, zsh, or ksh:

( x=$(cat IP) && echo "${x//$'\n'/, }" > IP )
  • Create a subshell to prevent the x variable from leaking out
  • Set x to the contents of the IP file, including all newlines except the last
  • Replace newlines in x with , (comma,space)
  • Echo the result back to the IP file, adding (back) a final newline

It works even if there are spaces in the lines, but it sounds like you don't need that. However in zsh if the data contains backslashes they are interpreted, often producing wrong results, unless you use echo -E.

4
  • The builtin echo or bash and ksh can also expand backslash-escapes (as POSIX requires!) depending on how those shells were built or invoked or the environment. You may also have problems if the first line starts with a -. See Why is printf better than echo? for details. Commented Dec 16, 2023 at 16:13
  • 2
    In those shells, you can also do x=$(<IP). Commented Dec 16, 2023 at 16:14
  • I'm curious: Why the parens to create a subshell? Is the purpose just to scope the variable x so it doesn't remain in memory after the command is complete? Commented Dec 16, 2023 at 22:07
  • @CharlesDuffy, see the first item: Create a subshell to prevent the x variable from leaking out Commented Dec 17, 2023 at 8:42
4

Using awk:

awk '{printf "%s%s", (NR==1) ? "" : ", ", $0}END{print ""}'

This will prefix every line except for the first with a comma.

3

Adding comma with sed but excluding the last line, then replacing every new line with space:

sed '$!s/$/,/' file | paste -sd ' ' -
0
2

Using AWK:

$ awk 'NR>1{printf "%s", a "," OFS} {a=$0}END {print a}'

If only comma is needed without space and all records have only one fields, then datamash may be used.

$ datamash -W collapse 1 <file
1
  • 1
    Thank you very much for your help everyone. Many good answers. Please let me know if or how I can show appreciation, by feedback, points, etc. I am new to this. Thanks agsain Commented Dec 14, 2023 at 15:24
2

If you can't remember / don't want to learn the more powerful tools like awk and sed (and don't mind a trailing comma) here's a simple and understandable solution:

OUT=""; for i in $(cat IP); do OUT="$OUT $i,"; done; echo $OUT

Output:

10.3.1.1, 10.4.1.1, 10.6.3.1, 10.19.4.2, 10.22.3.4,
2

If you don’t actually care about the space after each comma, this is what paste is good at:

paste -sd , -

You could restore the space by piping to sed if you know that no values contain a comma:

… | sed s/,/,\ /g
0

Using Raku (formerly known as Perl_6)

~$ raku -ne '$++ && print ", "; .print;'  file

#OR

~$ raku -e 'slurp.chomp.subst(:global, / \n /, ", ").put;'  file

Herein are answers written in Raku, a member of the Perl-family of programming languages.

In the first example, the file is read-in using the -ne non-autoprinting linewise flags, which auto-chomps (removes) the trailing newline by default. When the anonymous $++ post-incrementing counter variable reads the first line (i.e. $++ equals zero), the ", " comma-space is skipped while the line is output without a newline terminator using .print. Note: .print is short for $_.print meaning to print the $_ topic variable. Otherwise, for every subsequent line, a comma-space is printed followed by the line (i.e. topic) itself.

In the second example, the file is slurped, i.e. read-into memory all at once, preserving internal/trailing newlines, etc. The trailing newline is chomped off, internal \n newlines are substituted globally with ", " comma-space, and the resultant file is output, which (in contrast to print) adds a newline at the end of the string (here, the end of the file).

Sample Input:

10.3.1.1
10.4.1.1
10.6.3.1
10.19.4.2
10.22.3.4

Sample Output (both code examples):

10.3.1.1, 10.4.1.1, 10.6.3.1, 10.19.4.2, 10.22.3.4

https://docs.raku.org
https://raku.org

0
0

with ( I don't know if it's posix ):

sed -n -e '1h;1!H;${g;s/\n/, /g;p;}' data

Save all lines to the hold space and then, at the end, substitute all newlines with a , .


With (this should be posix portable. I don't remember if all let you unset the RS variable but i tested it with the --posix and --lint set to fatal in and it worked ):

awk -v RS= 'gsub(/\n/, ", ")' data

Unset the Record Separator variable to get all the file content as a single line and then replace all the new line charcter's with a ,

3
  • Note that empty RS is the paragraph mode, where records are separated by sequences of empty lines, not the slurp mode Commented Dec 17, 2023 at 20:19
  • 1
    The sed command is POSIX, as long as the input is guaranteed to be no larger than 10 x LINE_MAX byte large and is valid text in the user's locale Commented Dec 17, 2023 at 20:21
  • @StéphaneChazelas yes for awk. I don't mean that unsetting RS will place awk in slurp mode, it's just my exression limit with english language ( i will edit ). Thanks for sed clarification. Commented Dec 17, 2023 at 22:18
0

This would be a simple way to do it:

sed 's/$/,/' |xargs echo |sed 's/,$//'

However, it would fail:

  • for very long files (that exceed the maximum size of a program's arguments on your system),
  • if the first line of your file looks like an option to your system's echo program.

Edit: see the comments for more failure modes. Only use this for short files with no characters that are special to xargs or echo.

3
  • Also fails if lines contain single quotes, double quotes or backslashes or whitespace (list of which varying with locale and xargs implementation) and depending on the xargs implementation: if the input cannot be decoded as text in the locale, if there are words bigger than some limit which can be ridiculously low on some systems (I've seen 255). See also my comment to Chris' answer. Note that echo is the default command run by xargs, so xargs echo is the same as xargs alone. Commented Dec 18, 2023 at 13:39
  • @StéphaneChazelas Thanks, I had no idea xargs processed quotes in its arguments. (And I wonder why: xargs gets its input via argv and can just pass them on to exec(ve) since most programs work perfectly fine with arguments containing quotes. Historical reasons, I guess. An option to suppress that behaviour without resorting to -0 would be nice.)
    – gpvos
    Commented Dec 18, 2023 at 14:33
  • 1
    xargs gets a list of words on stdin which have to be delimited in some way (and quotes are used to escape the delimiters (and other types of quotes)) and passes those words across (the x in its name) as arguments to commands. To delimit words on newline only with no way to escape those newlines (and thus no way for a command argument to contain a newline), see the -d '\n' of the GNU implementation of xargs. Commented Dec 18, 2023 at 14:55
0

Some more simple variations:

cat IP | sed 's/$/, /' | tr -d '\n' | sed 's/, $/\n/'

or

cat IP | perl -pe 's/\n/, /' | perl -pe 's/, $/\n/'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .