2

I would like to use paste to add new column in original file "first.xls". My files:

first.xls

1
1
1

second.xls

2
2
2

I'm using:

paste -d'\t' first.xls  second.xls >> first.xls

and I get
first.xls:

1
1
1

1 2
1 2
1 2

but what I want is
first.xls:

1 2
1 2
1 2

I don't want to save it to new file I want to overwrite first.xls from beginning. I need this because I want to paste new column to my last results to add new column with every time I run script

1 2 2
1 2 2
1 2 2

then

1 2 2 2
1 2 2 2
1 2 2 2

and so on...

  • 2
    paste file file2 >tmp && mv tmp file – 123 Nov 8 '16 at 10:06
  • 1
    Terrible example text; almost impossible to tell what you actually want. Try using fruits or vegetables or animal names. – Wildcard Nov 8 '16 at 10:23
  • you need a better question than "bash paste command" e.g. "bash paste command without temp files" – gaoithe Nov 8 '16 at 13:10
2

Using sponge from the moreutils package:

paste -d'\t' first.xls  second.xls | sponge first.xls

man sponge

sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.

0

I don't think you can do that with paste alone.

Here's how I would do it:

paste -d' ' first.txt second.txt > tmpout && mv tmpout first.txt

Result after first run:

cat first.txt                   
1 2
1 2
1 2

After second run:

cat first.txt 
1 2 2
1 2 2
1 2 2

...and third run:

cat first.txt 
1 2 2 2
1 2 2 2
1 2 2 2
  • the default delimiter is tab which is what OP wanted.. so don't use -d' ' but yeah, expected output seems to just have single space... – Sundeep Nov 8 '16 at 11:18
  • I really doubt OP wants a tab based on the expected output! – maulinglawns Nov 8 '16 at 12:09
0

Alternate solution with awk

$ seq 3 > first.xls
$ seq 4 6 > second.xls

$ awk -v OFS="\t" 'NR==FNR{a[NR]=$0; next} {print a[FNR], $0 > "first.xls"}' first.xls second.xls 
$ cat first.xls 
1   4
2   5
3   6

$ awk -v OFS="\t" 'NR==FNR{a[NR]=$0; next} {print a[FNR], $0 > "first.xls"}' first.xls second.xls 
$ cat first.xls 
1   4   4
2   5   5
3   6   6
0

sponge as suggested on another answer is the right tool for the job!

sudo yum install moreutils
paste first.xls second.xls | sponge first.xls 

This bash command using tee works for small files (<2048 lines):

BEWARE: this hack doesn't work for larger files.

paste first.xls second.xls | tee first.xls >/dev/null

first.x second. result
1       2       1       2
1       2       1       2
1       2       1       2

This command runs the paste on two files and then pipes to 'tee'. The tee command RE-OPENs the file (and stdout) for write. So it works . . but it's a bit of a hack . . will it work on all systems? does this also work for big files?

See here not exactly the same question but relevant: https://stackoverflow.com/questions/1569730/paste-without-temporary-files-in-unix

Your problem is that your command opens first.xls for writing(append) and reading simultaneously. Data is read from the file (then combined with paste with other file). The output data is then written(appended) into the file. The newly written data is read back resulting in the append - as would be expected. Your append command:

paste first.xls second.xls >> first.xls

first.x second. result
1       2       1
1       2       1
1       2       1
                1       2
                1       2
                1       2

Changing the redirect to open for overwrite doesn't work either. It results in first.xls being opened and wiped before being read. And you get a blank column + 2 column. Overwrite instead of append doesn't work in a different way:

paste first.xls second.xls > first.xls

first.x second. result
1       2               2
1       2               2
1       2               2

TESTING larger files

1000 lines it works:

yes "1" | fmt -2 |head -1000 >1000.1
yes "2" | fmt -2 |head -1000 >1000.2
paste 1000.1 1000.2 |tee 1000.1 >/dev/null
uniq -c 1000.1 
 1000 1 2

10000 lines we start to see similar problem again:

Interesting we see it after 2048 lines :-) Am guessing is caused by disk buffer limit or perhaps pipe buffering limit . . .

yes "1" | fmt -2 |head -10000 >10000.1
yes "2" | fmt -2 |head -10000 >10000.2
paste 10000.1 10000.2 |tee 10000.1 >/dev/null
uniq -c 10000.1 
 2048 1 2
 7952   2
  • I quite like how this answer turned out! It is very 'interesting'. A good example of a hack solution that works in simple situation - but which will stop working (in this example if files get bigger). :-D – gaoithe Nov 8 '16 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.