6

How can I convert multi lines to a single line with spaces and quotes using awk or tr or any other tool which makes it simple (but not for loops)?

$ cat databases.txt
Wp_new
Frontend DB
DB_EXT

Expected:

$ cat databases.txt
"Wp_new" "Frontend DB" "DB_EXT"
7

With sed + paste

$ sed 's/.*/"&"/' databases.txt
"Wp_new"
"Frontend DB"
"DB_EXT"

$ sed 's/.*/"&"/' databases.txt | paste -sd' ' -
"Wp_new" "Frontend DB" "DB_EXT"

Or, just paste (courtesy https://unix.stackexchange.com/a/593240)

$ <databases.txt paste -d '"' /dev/null - /dev/null | paste -sd' ' -
"Wp_new" "Frontend DB" "DB_EXT"


If input has empty lines that should be ignored:

$ cat ip.txt
Wp_new

Frontend DB



DB_EXT
$ sed -n 's/..*/"&"/p' ip.txt | paste -sd' ' -
"Wp_new" "Frontend DB" "DB_EXT"
| improve this answer | |
12

It can be done with

awk '{printf("\"%s\" ",$0)} END { printf "\n" }' databases.txt

Output:

"Wp_new" "Frontend DB" "DB_EXT" 
| improve this answer | |
  • @roaima please post your solution as an answer :) – Skynet Jun 21 at 11:21
  • @roaima thanks updated. – Skynet Jun 21 at 12:40
  • @roaima Adding printf "\n" is wrong. It doesn't solve the trailing blank problem and it's hard-coding the value you hope ORS will have (\n) instead of simply using ORS as would occur if you used print "" instead of printf "\n". If the OP cares about an added blank or a mising newline then the right way to do this with awk is what I show at unix.stackexchange.com/a/594059/133219 - use the value OFS as appropriate and end with the value of ORS as appropriate. – Ed Morton Jun 21 at 13:41
  • @EdMorton we're joining lines as quoted words with a trailing newline. What other value of ORS were you considering!? – roaima Jun 21 at 14:15
  • 1
    @roaima \r\n and \0 are common ones that come to mind but beyond that philosophically - awk has a variable ORS that contains the value of the record terminator, any time you have a variable for a specific purpose writing code that then hardcodes the value of that variable is generally bad, especially if it's lengthier code (printf "\n") than code that just uses the variable (print ""). – Ed Morton Jun 21 at 14:19
3
$ awk '{printf "%s\"%s\"", sep, $0; sep=OFS} END{print ""}' file
"Wp_new" "Frontend DB" "DB_EXT"
| improve this answer | |
2

There are a number of ways of achieving the result using sed. Here are two solutions:

$ sed 's/^/"/;s/$/"/' infile | sed ':a;{N;s/\n/ /;ba}'
"Wp_new" "Frontend DB" "DB_EXT"

or

$ sed 's/.*/"&"/' infile | sed ':a;{N;s/\n/ /;ba}'
"Wp_new" "Frontend DB" "DB_EXT"

First, double quotes are added to the start and end of each text line, then the text lines are joined together.

| improve this answer | |
  • I always prefer sed over awk. thank you. – Akhil J Jun 21 at 5:25
  • (1) Your sed command with the N;s/\n/ /;ba ends up with the entire input file (plus the added quotes) in the pattern space (i.e., it treats it as one very long line), right? (Or am I misreading it?) While GNU sed seems to allow the pattern space to grow as large as it can get with dynamically allocated memory, POSIX specifies only that the pattern space be able to handle 8192 bytes, so this might fail on non-GNU systems. … (Cont’d) – G-Man Says 'Reinstate Monica' Jun 21 at 17:00
  • (Cont’d) …  (2) I believe that you don’t need the braces in that sed command — :a; N; s/\n/ /; ba seems to work. – G-Man Says 'Reinstate Monica' Jun 21 at 17:00
  • @G-ManSays'ReinstateMonica' Yes the braces are optional in this particular case. – fpmurphy Jun 22 at 1:19
2

to meet your output to be a POSIX text file:

awk '
function output(sep, record){ printf "\"%s\"%s", record, sep } 
prev{ output(FS, prev) } { prev=$0 } END{ output(ORS, prev) }' infile
| improve this answer | |
1

Here’s another sed solution.  Inevitably, it has some overlap with the answers by fpmurphy and Sundeep.

{ sed 's/.*/"&"/; 1!s/^/ /' databases.txt | tr -d '\n'; echo;}
  • Wrap each line in quotes.
  • Add a space to the beginning of every line but the first.
  • Delete all newlines.
  • Add a newline at the end.

As Sundeep points out, you can move < databases.txt to the beginning of the command if you particularly prefer to see the input specification in that position (rather than in the middle of the command).  And you can replace the echo with printf '\n' — although, following up on Ed Morton’s comments, echo might do a better job of generating the appropriate end-of-line character(s) in something other than a pure Unix/Linux context (e.g., a Windows/GNU hybrid).

| improve this answer | |
0

I love to use tr(1) to convert mult line strings into single line strings first and then work on them via sed(1) by first inserting the trailing and initial quote, then replacing all 'internal' semicolons, i.e.:

$ cat databases.txt | tr '\n' ';' | sed 's/;$/"\n/; s/^/"/; s/;/" "/g'
"Wp_new" "Frontend DB" "DB_EXT"

Of course you can choose any other character than ;, as long as it isn't contained in the the input itself.

While this should be easy to reason about, it is not necessarily portable and definitely not POSIX as correctly pointed out by @g-man-says-reinstate-monica. The issue lies in sed needing to read the whole line---here the entire file---while POSIX only requiring 8192 byte lines (1). Additionally, the input to sed must be a file (i.e. end with a newline 2).

I want to leave this trick here as it's still convenient if the above restrictions do not apply, however I want to stress it shouldn't be used as part of a script :)

| improve this answer | |
  • 1
    what if file has already ; in lines ? – Akhil J Jun 22 at 8:05
  • 3
    @AkhilJ '[...] as long as it isn't contained in the the input itself.' More often than not I find myself in the situation that there's one character that is not in any of the lines, thus enabling me to use this simple solution. I acknowledge however, that it's not universally applicable. – larkey Jun 22 at 11:02
  • 1
    The good news is this seems to work in GNU/Linux (i.e., using GNU sed) and that it contains an arguably clever(?) trick not seen in the other answers. The bad news is that it’s not POSIX-compliant and, IMHO, not very clear. (1) The input to sed is the (entire) file, with newlines replaced by semicolons — i.e., the entire file, treated as one line, read into the pattern space all at once. (1a) While GNU sed seems to allow the pattern space to grow as large as it can get … (Cont’d) – G-Man Says 'Reinstate Monica' Jun 23 at 2:26
  • (Cont’d) …  with dynamically allocated memory, POSIX specifies only that the pattern space be able to handle 8192 bytes, so this might fail on non-GNU systems for large files.  (1b) The input to sed does not contain any newlines, not even at the end.  (Obviously you understand that, as your s/;$/"\n/g command adds the required newline at the end.)  However, POSIX specifies that the input to sed must be a text file, … (Cont’d) – G-Man Says 'Reinstate Monica' Jun 23 at 2:26
  • (Cont’d) …  which means it must end with a newline.  So, theoretically, this could fail on non-GNU systems for files of any size.  (2) This is about clarity, opinion and style.  (2a) For the three-line input in the question (or any three-line input, for that matter), there will be six quote characters in the output.  All the other answers insert them in the order 1 & 2, then 3 & 4, then 5 & 6.  Your answer inserts them in the order 6, then 1, then 2 & 3, then 4 & 5.   … (Cont’d) – G-Man Says 'Reinstate Monica' Jun 23 at 2:26
0

GNU xargs

<databases.txt xargs -d'\n' printf '"%s"\n' | paste -sd\  ;

Or we can use perl also:

perl -l40 -012 -pe 's/^|$/"/g;$\=$/ if eof' databases.txt

Results:

"Wp_new" "Frontend DB" "DB_EXT"
| improve this answer | |
0

It could be done in some sed with:

$ sed ':a;N;$!ba;s/\n/" "/g;s/.*/"&"/' databases.txt 

"Wp_new" "Frontend DB" "" "DB_EXT" "" "empty"

Or, if you don't like branch and branch labels:

$ sed -n '/^$/!{                                          
    ${H;x;s/\n/" "/g;s/.*/"&"/;p;d;}
    $!{H;1h}
    }' databases.txt 

"Wp_new" "Frontend DB" "DB_EXT" "empty"

But both load the whole file into memory, which could use a lot for long files.

With some help from paste, sed could read lines one-at-a-time:

$ sed 's/.*/"&"/' databases.txt | paste -sd ' '

"Wp_new" "Frontend DB" "" "DB_EXT" "" "empty"

There is no need for the usual '-' for paste as it reads from stdin by default when given no FILE.

It could also be done in awk with explicit values:

$ awk 'BEGIN{dq="\""; sp=""} {
     printf "%s%s%s%s", sp, dq, $0,dq; sp=" "
     }END{print ""}' databases.txt 

"Wp_new" "Frontend DB" "DB_EXT"
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.