10

How can I convert multi lines to a single line with spaces and quotes using awk or tr or any other tool which makes it simple (but not for loops)?

$ cat databases.txt
Wp_new
Frontend DB
DB_EXT

Expected:

$ cat databases.txt
"Wp_new" "Frontend DB" "DB_EXT"  

Edit 1:

Thanks for all the useful answers. But the one I marked as correct is the one which can be typed on terminal in short time and with fewer hassles (simplicity) so that I(syadmins) can do the operations very fast without making the systems downtime more.

11 Answers 11

13

It can be done with

awk '{printf("\"%s\" ",$0)} END { printf "\n" }' databases.txt

Output:

"Wp_new" "Frontend DB" "DB_EXT" 
5
  • @roaima please post your solution as an answer :) Jun 21, 2020 at 11:21
  • @roaima thanks updated. Jun 21, 2020 at 12:40
  • @roaima Adding printf "\n" is wrong. It doesn't solve the trailing blank problem and it's hard-coding the value you hope ORS will have (\n) instead of simply using ORS as would occur if you used print "" instead of printf "\n". If the OP cares about an added blank or a mising newline then the right way to do this with awk is what I show at unix.stackexchange.com/a/594059/133219 - use the value OFS as appropriate and end with the value of ORS as appropriate.
    – Ed Morton
    Jun 21, 2020 at 13:41
  • @EdMorton we're joining lines as quoted words with a trailing newline. What other value of ORS were you considering!? Jun 21, 2020 at 14:15
  • 1
    @roaima \r\n and \0 are common ones that come to mind but beyond that philosophically - awk has a variable ORS that contains the value of the record terminator, any time you have a variable for a specific purpose writing code that then hardcodes the value of that variable is generally bad, especially if it's lengthier code (printf "\n") than code that just uses the variable (print "").
    – Ed Morton
    Jun 21, 2020 at 14:19
8

With sed + paste

$ sed 's/.*/"&"/' databases.txt
"Wp_new"
"Frontend DB"
"DB_EXT"

$ sed 's/.*/"&"/' databases.txt | paste -sd' ' -
"Wp_new" "Frontend DB" "DB_EXT"

Or, just paste (courtesy https://unix.stackexchange.com/a/593240)

$ <databases.txt paste -d '"' /dev/null - /dev/null | paste -sd' ' -
"Wp_new" "Frontend DB" "DB_EXT"


If input has empty lines that should be ignored:

$ cat ip.txt
Wp_new

Frontend DB



DB_EXT
$ sed -n 's/..*/"&"/p' ip.txt | paste -sd' ' -
"Wp_new" "Frontend DB" "DB_EXT"
0
3
$ awk '{printf "%s\"%s\"", sep, $0; sep=OFS} END{print ""}' file
"Wp_new" "Frontend DB" "DB_EXT"
3

There are a number of ways of achieving the result using sed. Here are two solutions:

$ sed 's/^/"/;s/$/"/' infile | sed ':a;{N;s/\n/ /;ba}'
"Wp_new" "Frontend DB" "DB_EXT"

or

$ sed 's/.*/"&"/' infile | sed ':a;{N;s/\n/ /;ba}'
"Wp_new" "Frontend DB" "DB_EXT"

First, double quotes are added to the start and end of each text line, then the text lines are joined together.

4
  • I always prefer sed over awk. thank you.
    – Akhil
    Jun 21, 2020 at 5:25
  • (1) Your sed command with the N;s/\n/ /;ba ends up with the entire input file (plus the added quotes) in the pattern space (i.e., it treats it as one very long line), right? (Or am I misreading it?) While GNU sed seems to allow the pattern space to grow as large as it can get with dynamically allocated memory, POSIX specifies only that the pattern space be able to handle 8192 bytes, so this might fail on non-GNU systems. … (Cont’d) Jun 21, 2020 at 17:00
  • (Cont’d) …  (2) I believe that you don’t need the braces in that sed command — :a; N; s/\n/ /; ba seems to work. Jun 21, 2020 at 17:00
  • @G-ManSays'ReinstateMonica' Yes the braces are optional in this particular case.
    – fpmurphy
    Jun 22, 2020 at 1:19
2

GNU xargs

<databases.txt xargs -d'\n' printf '"%s"\n' | paste -sd\  ;

Or we can use perl also:

perl -l40 -012 -pe 's/^|$/"/g;$\=$/ if eof' databases.txt

Results:

"Wp_new" "Frontend DB" "DB_EXT"
2

It could be done in some sed with:

$ sed ':a;N;$!ba;s/\n/" "/g;s/.*/"&"/' databases.txt 

"Wp_new" "Frontend DB" "" "DB_EXT" "" "empty"

Or, if you don't like branch and branch labels:

$ sed -n '/^$/!{                                          
    ${H;x;s/\n/" "/g;s/.*/"&"/;p;d;}
    $!{H;1h}
    }' databases.txt 

"Wp_new" "Frontend DB" "DB_EXT" "empty"

But both load the whole file into memory, which could use a lot for long files.

With some help from paste, sed could read lines one-at-a-time:

$ sed 's/.*/"&"/' databases.txt | paste -sd ' '

"Wp_new" "Frontend DB" "" "DB_EXT" "" "empty"

There is no need for the usual '-' for paste as it reads from stdin by default when given no FILE.

It could also be done in awk with explicit values:

$ awk 'BEGIN{dq="\""; sp=""} {
     printf "%s%s%s%s", sp, dq, $0,dq; sp=" "
     }END{print ""}' databases.txt 

"Wp_new" "Frontend DB" "DB_EXT"
2
  • Kudos for using the & replacement! :)
    – fbicknel
    Jun 25, 2021 at 10:49
  • Thx for the explanation regarding the usual ending ' -' for paste in your last example! Sep 13, 2021 at 16:35
1

Here’s another sed solution.  Inevitably, it has some overlap with the answers by fpmurphy and Sundeep.

{ sed 's/.*/"&"/; 1!s/^/ /' databases.txt | tr -d '\n'; echo;}
  • Wrap each line in quotes.
  • Add a space to the beginning of every line but the first.
  • Delete all newlines.
  • Add a newline at the end.

As Sundeep points out, you can move < databases.txt to the beginning of the command if you particularly prefer to see the input specification in that position (rather than in the middle of the command).  And you can replace the echo with printf '\n' — although, following up on Ed Morton’s comments, echo might do a better job of generating the appropriate end-of-line character(s) in something other than a pure Unix/Linux context (e.g., a Windows/GNU hybrid).

1

I love to use tr(1) to convert mult line strings into single line strings first and then work on them via sed(1) by first inserting the trailing and initial quote, then replacing all 'internal' semicolons, i.e.:

$ cat databases.txt | tr '\n' ';' | sed 's/;$/"\n/; s/^/"/; s/;/" "/g'
"Wp_new" "Frontend DB" "DB_EXT"

Of course you can choose any other character than ;, as long as it isn't contained in the the input itself.

While this should be easy to reason about, it is not necessarily portable and definitely not POSIX as correctly pointed out by @g-man-says-reinstate-monica. The issue lies in sed needing to read the whole line---here the entire file---while POSIX only requiring 8192 byte lines (1). Additionally, the input to sed must be a file (i.e. end with a newline 2).

I want to leave this trick here as it's still convenient if the above restrictions do not apply, however I want to stress it shouldn't be used as part of a script :)

9
  • 2
    what if file has already ; in lines ?
    – Akhil
    Jun 22, 2020 at 8:05
  • 3
    @AkhilJ '[...] as long as it isn't contained in the the input itself.' More often than not I find myself in the situation that there's one character that is not in any of the lines, thus enabling me to use this simple solution. I acknowledge however, that it's not universally applicable.
    – ljrk
    Jun 22, 2020 at 11:02
  • 1
    The good news is this seems to work in GNU/Linux (i.e., using GNU sed) and that it contains an arguably clever(?) trick not seen in the other answers. The bad news is that it’s not POSIX-compliant and, IMHO, not very clear. (1) The input to sed is the (entire) file, with newlines replaced by semicolons — i.e., the entire file, treated as one line, read into the pattern space all at once. (1a) While GNU sed seems to allow the pattern space to grow as large as it can get … (Cont’d) Jun 23, 2020 at 2:26
  • (Cont’d) …  with dynamically allocated memory, POSIX specifies only that the pattern space be able to handle 8192 bytes, so this might fail on non-GNU systems for large files.  (1b) The input to sed does not contain any newlines, not even at the end.  (Obviously you understand that, as your s/;$/"\n/g command adds the required newline at the end.)  However, POSIX specifies that the input to sed must be a text file, … (Cont’d) Jun 23, 2020 at 2:26
  • (Cont’d) …  which means it must end with a newline.  So, theoretically, this could fail on non-GNU systems for files of any size.  (2) This is about clarity, opinion and style.  (2a) For the three-line input in the question (or any three-line input, for that matter), there will be six quote characters in the output.  All the other answers insert them in the order 1 & 2, then 3 & 4, then 5 & 6.  Your answer inserts them in the order 6, then 1, then 2 & 3, then 4 & 5.   … (Cont’d) Jun 23, 2020 at 2:26
1

Using Raku (formerly known as Perl_6)

raku -e 'lines>>.raku.put;' 

OR

raku -e 'lines.map(*.raku).put;' 

Sample Input:

Wp_new
Frontend DB
DB_EXT
This "quoted" ending comment

Sample Output:

"Wp_new" "Frontend DB" "DB_EXT" "This \"quoted\" ending comment"

If you play with the various commands above, you'll find that the lines command reads input lazily (stripping newlines by default), the raku (or perl) command gives you an internal 'Perlish/Rakuish' representation of the input (i.e. adds quotes), and the put command means 'print-using-terminator'.

If the above code is unmemorizable and/or too long to type, you could try 'meeting-Raku-halfway' with the following code (i.e. below returns more "Rakuish" output):

raku -e 'lines.list.raku.put;'  databases.txt  #OP's database.txt file

("Wp_new", "Frontend DB", "DB_EXT")

Or more simply:

raku -e 'lines.raku.put;'  databases.txt  #OP's database.txt file

("Wp_new", "Frontend DB", "DB_EXT").Seq

https://docs.raku.org/routine/lines
https://raku.org/

2
  • Note: I added the line This "quoted" ending comment to illustrate Raku's elegant handling of quote characters (escaping them). The accepted sed/paste answer returns the result "Wp_new" "Frontend DB" "DB_EXT" "This "quoted" ending comment". Sep 13, 2021 at 16:23
  • 1
    great. looks simple. This is the first time I encountered raku command in text processing, and I think it's not available in POSIX like sed. But I am going to use this from now. Thanks. @jubilatious1
    – Akhil
    Sep 14, 2021 at 4:49
0
awk 'ORS=" "{print "\""$0"\""}' filename

output

"Wp_new" "Frontend DB" "DB_EXT"
0

All answer are nice, I guess this depend about what output did you expect. If you want a raw line text with no additional characters or if is necessary to add some, I tried this and it works very nice to me:

Simply collapse the text:

cat yourFile | paste -sd' ' -
>item1 item2 item3

Surround item with double-quotes:

sed -n 's/..*/"&",/p' yourFile | paste -sd' ' -
>"item1", "item2", "item2",

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .