Questions tagged [merge]

Merging is the operation of taking two or more source contents to produce a single one. Things that are commonly merged are files (source code, configuration, etc.), branches in a version control system, redundant entries in databases, etc.

Filter by
Sorted by
Tagged with
3
votes
2answers
67 views

merging 5 million files efficiently

Due to some poor planning i have a directory containing > 5 million files totalling roughly 20GB. Each file contains 32 lines of garbage at the top followed by an unknown number of lines of ...
1
vote
1answer
51 views

merging data of two data sets

I am attempting to merge two datasets..... File 1 is a single column JDay list 1 to 366 File 2 has 3 columns 2002 1 9.5938 2002 2 9.5938 2002 3 -28672.0000 2002 5 -28672.0000 At times there is ...
1
vote
1answer
110 views

sort multiple files and merge

I want to sort several text files in reverse order and then merge/cat to one single text file. a.txt 0 33.1 2 33.0 10 21.1 20 21.8 b.txt 0 30.1 2 33.0 10 28.1 20 27.8 and so on *.txt ...
0
votes
1answer
50 views

Merge two txt files into one using AWK and add missing items

my goal is merge this two files into one, read first if first item is present in second file update it, if not just copy first file a.txt AAA;2020-09-01;Y BBB;2020-09-01;Y CCC;2020-09-01;Y second ...
0
votes
1answer
29 views

Concatenate each two files with awk

I have several files like this BR1.txt NH1.csv BR2.txt NH2.csv BR3.txt NH3.csv and I want to concatenate each BR with NH in one file.
2
votes
5answers
176 views

Using awk or sed to concatenate lines having a pattern

I essentially want to put the name of the query in front of each line containing the SourceField. I have tried this concatenation scheme awk '/<\^Query/ && p{print p;p=""}{p=p $0}...
1
vote
1answer
175 views

Merge two CSV files into one using AWK

I am trying to merge two csv files using AWK. If the 1st column in file 1 matches the 1st column in file 2 I want to change file1's 3rd column to the value of file2's 2nd column. If there is no match, ...
0
votes
0answers
34 views

meld + vim : Is it possible to use Vi/Vim buffers while doing 3 way merge in Meld

Is it possible to use Vim's power (Vim buffers...) while doing 3 way merge in Meld ? I can see in Meld you can change the default editor: edit -> Preferences -> Editor -> default editor I ...
1
vote
0answers
44 views

Merge two directories with files in common

I have two directories that I need to merge: B (base), which is very large U (update), which is moderately sized and contains some modifications and additions to files in B Both B and U are updated ...
0
votes
2answers
75 views

Merge two columns of a gzipped file and replace the merged column with another column

I have a gzipped file which looks like this : 1 rs571093408 rs571093408 13380 C G I want to merge the first and fourth column with : inbetween and replace the second column with the new content, so ...
1
vote
1answer
601 views

ffmpeg merge multiple sets of 2 videos in for loop

I am still pretty new to linux and have been trying very hard at getting this right. Please help me to get this right. I am trying to merge 2 videos (1 from each folder) multiple times in like a ...
1
vote
2answers
73 views

Merge rows using common values in any column

I have a tab-delimited file like shown below, and would like to merge the rows based on matches in any of the columns. The number of columns are usually 2, but could vary in some cases and be 3. ...
0
votes
4answers
98 views

merge multiple txt files

I have multiple txt files, 3 of them are like: file1: sample input filtered 5809378 1 2 5811151 3 4 5811237 5 6 file2: sample chi tri 5809378 7 8 5811151 5811237 9 ...
0
votes
0answers
26 views

Set named pipe as pattern

Some introduction: I have a program, which works with named pipe and gives as output a directory with a certain name. The directory will be filling by several files with certain patterns (e.g., *R1* ...
0
votes
2answers
36 views

Merging files in a specific order of two different name lists

In one folder I have two sets of 6 files: first set: 1st-letter-00 - 1st-letter-05 second set: verse-00 - verse-05 How can I concatenate these two sets such that the result is one file that has all ...
1
vote
0answers
28 views

using rsync to move merge files into new directory fails to show progress or create backups

I have a bunch of old backups, each of which is a directory containing incremental changes from the previous backup. There are arbitrarily nested directories and files in each directory. I need to ...
1
vote
1answer
1k views

how to combine 2 arrays into one associative array

i need to combine ARRAY1 and ARRAY2 into an associative array like ARRAY. i'm using this code: mapfile -t ARRAY1 < <(/bin/awk '{ print $ 1 }' /output/gen_branch) mapfile -t ARRAY2 < <(...
0
votes
1answer
46 views

Combine the value of the variables into one with a new line after first variable

I have below two variables: $var1 which has value like ABC XYZ SDF and $var 2 which has value like SDF SGDGH hfg I want the output like ABC XYZ SDF SDF SGDGH hfg I have the below script and i ...
1
vote
2answers
525 views

How to JOIN two JSONs in environment variables?

I have environment variable, which contains JSON text. I want to add some data to it with jq tool. I want to keep all old fields but add and/or overwrite new ones. The idea is the same as with adding ...
0
votes
2answers
137 views

Merging strings as columns to create a table in bash

Consider the following strings: $ columnA="A1\nA2\nA3" $ columnB="B1\nB2\nB3" $ columnC="C1\nC2\nC3" Using Bash, how can I merge these so that I get another string with the following content: $ ...
0
votes
1answer
295 views

updating one file based on values in another with AWK

I have two files file1.csv (20 columns 410k rows) and data.csv (4 columns 1800 rows). What I am trying to do is if data.csv 1st column matches file1.csv 2nd column overwrite 1st column in file1.csv ...
0
votes
1answer
137 views

How to merge/join two file based on common field with repeated rows [duplicate]

I have 2 file: file1 SC 65.696 618 SC 61.92 625 KN 82.546 1037 KN 82.946 1032 KN 82.849 1032 CN 70.55 618 and file2 SC SC7 SCARE 588 2 75702 KN KN4 Kines ...
0
votes
1answer
44 views

copy-merge several folders contents, prioritizing newer files

I have few folders like this with their contents data/ -2011/ ---images/ ------logo.jpg ------banner.jpg ---reports/ ------monthly.pdf -2012/ ---images/ ------logo.jpg ---reports/ ------monthly.pdf -...
1
vote
1answer
92 views

How can I merge two sorted named pipes with uneven sizes?

My question is similar to "Merge two sorted files based on sorting values in the same field" but extending it to named pipes. Say I have two text files with sorted integers and I want to merge them. ...
1
vote
0answers
88 views

How do I merge identically named files?

My question is similar to this one but the answers don't address how I would like to solve it in my case: Background: I have two computers generating experimental results that are filed in ...
1
vote
2answers
303 views

Merging csv files with Key columns

I need to write a shell script to merge csv files. Both the files have same columns with fixed number of columns designated as "Keys". If the Keys are same, then I want to merge them by adding the non ...
1
vote
2answers
107 views

Merging two files using matching characters

File1: X X P X N X Q File2: P 1 N 5 Q 0 Desired output: X 0 X 0 P 1 X 0 N 5 X 0 Q 0 I have tried a lot of ways using merge command on bash. I couldn't make it work.
1
vote
0answers
141 views

merge commands for line-by-line interactive merging with sdiff, in MacOS unix. Is there a working version?

On MacOS, the sdiff command is described as able to do line-by-line interactive merging, and lists several merge commands for the latest chunk. e.g. sdiff -o mergefile fileleft fileright The single ...
6
votes
1answer
2k views

Command-line diff tool

So we all have the diff command installed by default. I certainly don't know how to use it well. But I would expect some reasonable default usage. But when I type diff file1 file2 I get ... a text ... ...
0
votes
1answer
422 views

Merging PDF files with similar names

I have a folder populated with PDF files in this format: SI-T-xxxxxx.pdf SI-T-xxxxxx-sol.pdf SI-T-yyyyyy.pdf SI-T-yyyyyy-sol.pdf and so forth, with the middle part being digits. I want a way for ...
4
votes
4answers
698 views

How to merge pre-sorted files into a single BIG file, without excessive memory or temporary disk use

I'm trying to sort (and ideally remove duplicate lines) from a 1.4TB file. Splitting and sorting the individual chunks is not an issue, but reassembling them is turning out to be a challenge. I ...
2
votes
1answer
168 views

Does sort --unique -k drop duplicates in original order?

I'm doing a unique sort on a concatenated set of index files where the first column will sometimes change between each index and the second column will be a key value (actually hex addresses). Each ...
1
vote
2answers
2k views

Joining two csv files on common column and removing the second last column

I have two csv files: file1: C1, 1, 0, 1, 0, 1 C2, 1, 0, 1, 1, 0 C3, 0, 0, 1, 1, 0 file2: C3, 1.2 C1, 2.3 C2, 1.8 I want to merge these two files based on C column (which produces): C1, 1, 0, 1, ...
0
votes
2answers
68 views

Joining two space-separated files based on matching values in two different columns

I have tried various solutions given in this site for this kind of problem and none of these are helping me out. I have two files (space-separated, contains columns). Both the files contain the same ...
1
vote
3answers
99 views

Merging in Unix

I have a CSV file with vertical bars (|) as the delimiter, like below, for which I need to apply merging technique in Unix. The file contains hundreds of thousands of records (four fields), but I gave ...
1
vote
2answers
52 views

How to merge different columns in the same file?

I have three files, as shown below: FILE 1: DATE PGTO_CRED 20180801 50.00 20180802 150.00 20180803 130.00 20180804 110.00 20180805 200.00 FILE 2: DATE PGTO_TOTAL 20180801 ...
1
vote
1answer
1k views

Awk for merging multiple files with common column

I have a question concerning awk command in unix to merge multiple tables with a common value Tab1 Geneid Chr Start End Strand Length Sample_1 ENSG00000278267 1 17369 17436 - 68 0 ...
2
votes
2answers
135 views

Fetching, merging and exporting data from CURL

Before anything else, english isn't my laguage. I'll try to be understandable :-) Second, I am not a regular Linux user, I'm just starting out. I'm using curl and the API of a web chat platform to ...
0
votes
1answer
43 views

merging files with the same 1 column

I have multiple files named p1.bbmap.mapping.csv, p2.bbmap.mapping.csv, p3.bbmap.mapping.csv ... p3.bbmap.mapping.csv Each file has two columns. p1_60_length_504_cov_1.580902 12 ...
2
votes
3answers
7k views

How to concatenate files from different sub-directories?

I have a large folder containing many sub-directories each holding many .txt files. I want to concatenate all of these files into one .txt file. I am able to do it for each of the sub-directories with ...
2
votes
1answer
297 views

Git not creating merge commit when merging signed/annotated tag

I'm expecting Git to create a merge commit when I merge a signed or annotated tag but instead it fast-forwards the branch pointer. I'm using Git version 2.17.0 on openSUSE Tumbleweed that comes with ...
0
votes
1answer
511 views

Mount and merge folder contents

Is there any way to mount a folder content inside to another one? Let's say I have 2 folders with following content; Folder1 Folder2 file1 file2 I basically want to make visible all ...
0
votes
1answer
1k views

How to merge two gz files into new outputfile with specific name

I have 96 *fastqc.gz raw read files from 24 samples. Each sample was sequenced on two lanees for each pair. I would like to merge reads for each pair from both lanes into one output file with same ...
1
vote
0answers
432 views

socat to merge TCP streams

I'm trying to merge 2 TCP streams into one and then send the result outside through a TCP port server. The first part is Ok, I'm able to capture incoming streams as below: STREAM1: socat TCP:127.0.0....
0
votes
1answer
43 views

simple merge of CSV file

I want to merge two CSV without any condition and no primary.It should be first row file1 and file2. file1 id,name,city file2 mobileno,address,email output id,name,city,mobileno,address,,...
1
vote
3answers
2k views

Combine multiple .csv files with different lengths in a single one

I have a similar problem as Merging contents of multiple .csv files into single .csv file but for different length of csv files. What should I add to the perl script in this answer, in order to ...
0
votes
1answer
70 views

Merging columns from 200+ big files into one table

I have 200+ large files that all have exactly 1 column and 76M rows. I want to start a newfile.txt and slap the columns next to each other (match Line 1 of File 1 with Line 1 of File 2... and continue ...
0
votes
2answers
632 views

How to merge two /etc/group files

I have two files in style of /etc/group : root:x:0: daemon:x:1: bin:x:2: sys:x:3: adm:x:4:username tty:x:5: disk:x:6: lp:x:7: mail:x:8: news:x:9: ubuntu:x:999: And I want to merge them into a third ...
2
votes
3answers
96 views

Merge columns and add

I have a file with the following content (there are additional spaces to pad the first column) 1 account1 192.168.0.1 1 account1 192.168.0.2 19 account2 192.168.0.1 100 account3 192.168....
0
votes
2answers
151 views

Interleave seven fastQ files

I have 7 FastQ files and I want to merge them into one in the following way: File1 line1 File1 line2 File1 line3 File1 line4 File2 line1 File2 line2 File2 line3 File2 line4 File3 line1 File3 line2 ...