Questions tagged [merge]

Merging is the operation of taking two or more source contents to produce a single one. Things that are commonly merged are files (source code, configuration, etc.), branches in a version control system, redundant entries in databases, etc.

Filter by
Sorted by
Tagged with
1 vote
6 answers
104 views

AWK one liner to merge three fields in a single file

I have a file with records (lines) having two types of field delimiters | and ! as given below: Name|Age|Physics|Chemistry|Maths|English|Batch!Year!AdmisnNo!Grade!Score Student1|81|65|70|80|88|EWS!...
user avatar
1 vote
3 answers
55 views

concatenate all the text files in all the sub folders into one giant text file

Ok so the situation is that I have an unknown number of sub-directories that all follow the same naming profile folder0, folder1, folder2, folder3 etc Now each folder will have 3 text files and these ...
user avatar
1 vote
0 answers
42 views

How merge two directories? [closed]

For example i have dirA, dirB - all with subdirectories. I want to merge dirA into dirB with "force" dirA. I tryied to use this /usr/bin/rsync -avh --progress dirA dirB But "force"...
user avatar
0 votes
0 answers
29 views

Delete duplicated contents from files

I have many backups of a same file. Is there a way to transform that into incremental backup? Those files aren't exactly the same (not same timestamps sometimes, sometimes new data appended here and ...
user avatar
  • 13
0 votes
0 answers
31 views

Virtual filesystem to merge multiple filesystems using patterns for resulting structure

I'm looking for a virtual file system, that would be capable of merging multiple filesystems into a new structure using some pattern. The underlying sources can be different filesystems by itself (...
user avatar
  • 3,785
-2 votes
2 answers
73 views

how to merge and amend columns of two files

I need to merge two files based on the 1st and 2nd columns. I need to add the difference between 3rd and 4th columns and add TOTAL rows per type of 2nd column. Here are the two input files: File 1: ...
user avatar
  • 239
0 votes
0 answers
76 views

merge 2 files in linux based on 4 columns on the command line

I have two data sets. Dataset 1 is a txt.gz file with 4 columns with information and another column filled with na. Dataset 2 is a vcf format .gz file and I need to use this second data set to 'merge' ...
user avatar
  • 1
4 votes
3 answers
279 views

Merging files based on potentially incomplete keys

I want to merge two files like How to merge two files based on the matching of two columns? but one file may not have all results. So for example file1 1 dog 2 cat 3 fox 4 cow file2 1 woof 2 meow 4 ...
user avatar
0 votes
0 answers
27 views

How to join two files into one in multiple folders using specific columns and add file names where they originated from

I would like to ask, how can I join multiple files into one using a specific column and name it by the originating folder. Let me explain: I have the following files, having multiple columns and rows: ...
user avatar
  • 5
0 votes
2 answers
126 views

How to merge txt file with the same name from all subdirectories

I have folders with a lot of files, Each folder has txt files of the same name, I want to merge all txt files from all subdirectories into a single folder. Example: /home/parent/Folder1 ===> ...
user avatar
0 votes
1 answer
57 views

How to merge two files based on a single key column and auto fix fixed columns and fill missing data

I have two sets of files which attempting to merge into a single file and filling in associated missing data. Files are all , delimited The first file contains 13 columns with the 8th column ...
user avatar
0 votes
0 answers
8 views

How to choose numeric precision when creating a file with STILTS?

I am using the STILTS command-line tools, popular in astronomical data processing, to merge many (~330k) .fits tables into 1 large table. I am doing this with the STILTS command in the terminal stilts ...
user avatar
  • 139
0 votes
0 answers
38 views

How to merge unallocated disk space in a device in another device?

when I'm doing "fdisk -l" we can see : Device Start End Sectors Size Type /dev/nvme0n1p1 2048 1050623 1048576 512M EFI System /dev/nvme0n1p2 1050624 500117503 ...
user avatar
0 votes
1 answer
70 views

Linux: How to merge identical named folders with different case from the CLI (BASH preferred)

I have on my Linux NAS miscellaneous folders with identical names, except for case,like this: /data/libraries /Data/libraries/presets /Data/Libraries/presets /Data/Libraries/Presets /merged/readme /...
user avatar
0 votes
0 answers
90 views

Merge unallocated partition into my mounted partition (/dev/nvme0n1p5)

As you can see, I am mounted on /dev/nvme0n1p5. Gparted does not allow me to merge the unallocated partition (51.94GB) into the extended partition (i.e. mounted partition) using Move/Resize action. I ...
user avatar
  • 1
1 vote
1 answer
176 views

Merging columns from multiple files and adding headers

I have over 50 files that have the following format: File1.tab A 25 B 56 C 87 D 82 E 91 F 34 File2.tab A 89 B 55 C 92 D 36 E 28 F 19 File3.tab A 98 B 76 C 82 D 76 E 49 F 42 etc. I would like the ...
user avatar
  • 13
-4 votes
3 answers
109 views

merge two columns

I am trying to merge two columns into one column. My dataset look like this: RSID1 RSID2 rs7475652 rs7475652 rs7475652 rs7918643 rs7475652 rs3125034 rs7475652 rs3750730 rs7475652 rs883728 rs7475652 ...
user avatar
0 votes
2 answers
43 views

Function for mergin every five .txt files

I have a problem, my folder contains 1500 .txt files. I need to write a function to merge every five into one. Now I'm doing it like this: cat 1.txt 2.txt 3.txt 4.txt 5.txt >> 1a.txt But ...
user avatar
  • 31
2 votes
0 answers
72 views

Is there a way to "map" multiple files into a single file, without writing to disk or RAM?

I've been thinking about different ways files could be hidden, and while encryption is excellent at hiding the contents of files, they're not really good at hiding the fact that files exist. In the ...
user avatar
3 votes
1 answer
99 views

Merge two binary image files by boolean OR (ddrescue output filename mistake)

I made a silly mistake by using the wrong output filename when resuming a ddrescue. This is what happened: ddrescue -b 2048 -d -v /dev/sr1 IDTa.img IDTa.ddrescue.log Then the computer crashed and I ...
user avatar
  • 75
0 votes
0 answers
25 views

The weird order when concating variables and string with '_' [duplicate]

I want to concatenate a variable and a string. while read subject; do pathproc=/projects/guo_lab/cbis/users/yang/DICA_result echo ${pathproc}/${subject}"_"weight done < /projects/...
user avatar
1 vote
1 answer
68 views

How to merge 2 directories without overwriting *different* files with same relative path?

Example Suppose I have 2 directories a/ and b/. Let's say they contain the following files: a/ a/foo/1 a/bar/2 a/baz/3 b/ b/foo/1 b/bar/2 such that a/foo/1 and b/foo/1 are identical but a/bar/2 and ...
user avatar
0 votes
1 answer
84 views

Merging two files based on a common column; add 0 for variants not present in file 2, and keep the original value in those which are present

I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column (3) following the order of file 1 and I want to ...
user avatar
  • 71
1 vote
1 answer
61 views

merging two files according to a common column and add 0 to those which are not present in file 2

I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column following the order of file 1 and I want to add ...
user avatar
  • 71
1 vote
1 answer
21 views

Merge sort and extract new uniques?

I found a post that answers how to merge sort and remove duplicates. I'm interested in expanding on this and extracting new unique entries from the post-merged file. What would be the command to do so?...
user avatar
2 votes
2 answers
59 views

Is there any git option to automatically merge commits from master branch into a feature branch?

I have a feature branch in git that needs to be up to date with the master branch at all times. Is there any way to automatize this in git? I basically want on every commit in master branch a merge ...
user avatar
0 votes
1 answer
50 views

How to merge multiple files which are not in order and assign the name of file to a specific column in linux?

I have around 250 .gtf files. Here I'm showing 3 .gtf files. All the .gtf files have three columns that look like below: TUSCC120A.gtf TUSCC36.gtf TUSCC89B.gtf TUSCC120A.gtf looks like below: ...
user avatar
  • 245
3 votes
2 answers
127 views

merging 5 million files efficiently

Due to some poor planning i have a directory containing > 5 million files totalling roughly 20GB. Each file contains 32 lines of garbage at the top followed by an unknown number of lines of ...
user avatar
1 vote
1 answer
71 views

merging data of two data sets

I am attempting to merge two datasets..... File 1 is a single column JDay list 1 to 366 File 2 has 3 columns 2002 1 9.5938 2002 2 9.5938 2002 3 -28672.0000 2002 5 -28672.0000 At times there is ...
user avatar
1 vote
1 answer
1k views

sort multiple files and merge

I want to sort several text files in reverse order and then merge/cat to one single text file. a.txt 0 33.1 2 33.0 10 21.1 20 21.8 b.txt 0 30.1 2 33.0 10 28.1 20 27.8 and so on *.txt ...
user avatar
0 votes
1 answer
134 views

Merge two txt files into one using AWK and add missing items

my goal is merge this two files into one, read first if first item is present in second file update it, if not just copy first file a.txt AAA;2020-09-01;Y BBB;2020-09-01;Y CCC;2020-09-01;Y second ...
user avatar
  • 15
-1 votes
1 answer
58 views

Concatenate each two files with awk

I have several files like this BR1.txt NH1.csv BR2.txt NH2.csv BR3.txt NH3.csv and I want to concatenate each BR with NH in one file.
user avatar
2 votes
5 answers
360 views

Using awk or sed to concatenate lines having a pattern

I essentially want to put the name of the query in front of each line containing the SourceField. I have tried this concatenation scheme awk '/<\^Query/ && p{print p;p=""}{p=p $0}...
user avatar
  • 121
2 votes
1 answer
2k views

Merge two CSV files into one using AWK

I am trying to merge two csv files using AWK. If the 1st column in file 1 matches the 1st column in file 2 I want to change file1's 3rd column to the value of file2's 2nd column. If there is no match, ...
user avatar
1 vote
0 answers
66 views

Merge two directories with files in common

I have two directories that I need to merge: B (base), which is very large U (update), which is moderately sized and contains some modifications and additions to files in B Both B and U are updated ...
user avatar
0 votes
2 answers
176 views

Merge two columns of a gzipped file and replace the merged column with another column

I have a gzipped file which looks like this : 1 rs571093408 rs571093408 13380 C G I want to merge the first and fourth column with : inbetween and replace the second column with the new content, so ...
user avatar
  • 3
2 votes
1 answer
1k views

ffmpeg merge multiple sets of 2 videos in for loop

I am still pretty new to linux and have been trying very hard at getting this right. Please help me to get this right. I am trying to merge 2 videos (1 from each folder) multiple times in like a ...
user avatar
1 vote
2 answers
92 views

Merge rows using common values in any column

I have a tab-delimited file like shown below, and would like to merge the rows based on matches in any of the columns. The number of columns are usually 2, but could vary in some cases and be 3. ...
user avatar
0 votes
4 answers
366 views

merge multiple txt files

I have multiple txt files, 3 of them are like: file1: sample input filtered 5809378 1 2 5811151 3 4 5811237 5 6 file2: sample chi tri 5809378 7 8 5811151 5811237 9 ...
user avatar
0 votes
2 answers
51 views

Merging files in a specific order of two different name lists

In one folder I have two sets of 6 files: first set: 1st-letter-00 - 1st-letter-05 second set: verse-00 - verse-05 How can I concatenate these two sets such that the result is one file that has all ...
user avatar
1 vote
0 answers
104 views

using rsync to move merge files into new directory fails to show progress or create backups

I have a bunch of old backups, each of which is a directory containing incremental changes from the previous backup. There are arbitrarily nested directories and files in each directory. I need to ...
user avatar
  • 474
2 votes
1 answer
3k views

how to combine 2 arrays into one associative array

i need to combine ARRAY1 and ARRAY2 into an associative array like ARRAY. i'm using this code: mapfile -t ARRAY1 < <(/bin/awk '{ print $ 1 }' /output/gen_branch) mapfile -t ARRAY2 < <(...
user avatar
0 votes
1 answer
96 views

Combine the value of the variables into one with a new line after first variable

I have below two variables: $var1 which has value like ABC XYZ SDF and $var 2 which has value like SDF SGDGH hfg I want the output like ABC XYZ SDF SDF SGDGH hfg I have the below script and i ...
user avatar
1 vote
2 answers
2k views

How to JOIN two JSONs in environment variables?

I have environment variable, which contains JSON text. I want to add some data to it with jq tool. I want to keep all old fields but add and/or overwrite new ones. The idea is the same as with adding ...
user avatar
  • 2,517
0 votes
2 answers
403 views

Merging strings as columns to create a table in bash

Consider the following strings: $ columnA="A1\nA2\nA3" $ columnB="B1\nB2\nB3" $ columnC="C1\nC2\nC3" Using Bash, how can I merge these so that I get another string with the following content: $ ...
user avatar
  • 367
0 votes
1 answer
945 views

updating one file based on values in another with AWK

I have two files file1.csv (20 columns 410k rows) and data.csv (4 columns 1800 rows). What I am trying to do is if data.csv 1st column matches file1.csv 2nd column overwrite 1st column in file1.csv ...
user avatar
  • 17
0 votes
1 answer
261 views

How to merge/join two file based on common field with repeated rows [duplicate]

I have 2 file: file1 SC 65.696 618 SC 61.92 625 KN 82.546 1037 KN 82.946 1032 KN 82.849 1032 CN 70.55 618 and file2 SC SC7 SCARE 588 2 75702 KN KN4 Kines ...
user avatar
  • 13
1 vote
1 answer
53 views

copy-merge several folders contents, prioritizing newer files

I have few folders like this with their contents data/ -2011/ ---images/ ------logo.jpg ------banner.jpg ---reports/ ------monthly.pdf -2012/ ---images/ ------logo.jpg ---reports/ ------monthly.pdf -...
user avatar
1 vote
1 answer
221 views

How can I merge two sorted named pipes with uneven sizes?

My question is similar to "Merge two sorted files based on sorting values in the same field" but extending it to named pipes. Say I have two text files with sorted integers and I want to merge them. ...
user avatar
  • 111
1 vote
0 answers
100 views

How do I merge identically named files?

My question is similar to this one but the answers don't address how I would like to solve it in my case: Background: I have two computers generating experimental results that are filed in ...
user avatar
  • 129