Joins the lines of two files or more

learn more… | top users | synonyms

9
votes
5answers
9k views

Compare two files with first column and remove duplicate row from 2nd file in shell script

I will ask my question with an example. I have 2 files: File #1: 118D FC300_R5_TP FX.B 32775 2112 6 2038 6 2112 0 118E FC300_R5_TP FX.B 32775 2136 7 ...
8
votes
2answers
19k views

Join two files with matching columns

File1.txt id No gi|371443199|gb|JH556661.1| 7907290 gi|371443198|gb|JH556662.1| 7573913 gi|371443197|gb|JH556663.1| 7384412 gi|371440577|gb|JH559283.1| ...
7
votes
3answers
30k views

How to merge two files based on the matching of two columns?

I have file1 likes: 0 AFFX-SNP-000541 NA 0 AFFX-SNP-002255 NA 1 rs12103 0.6401 1 rs12103_1247494 0.696 1 rs12142199 0.7672 And a file2: 0 AFFX-SNP-000541 1 0 ...
7
votes
5answers
2k views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
7
votes
2answers
4k views

A shell script for joining two files

I want to write a shell script that get two files A and B, and get a result like this: File A: user_a tel_a addr_a user_b tel_b addr_b File B: process_1 user_a process_2 user_a process_3 user_b ...
7
votes
4answers
2k views

How to merge two files with different number of rows in shell?

I have file1 like this: CHR SNP TEST A1 A2 GENO O(HET) E(HET) P 0 AFFX-SNP-000541 ALL 0 0 0/0/0 ...
7
votes
4answers
340 views

Matching and pasting to line

So I have 2 very big text files, consisting of lines like so: First: Robert:Dillain:Other:Other:Other Julian:Brude:Other:Other:Other Megan:Flikk:Other:Other:Other Samantha:Minot:Other:Other:Other ...
7
votes
2answers
10k views

Compare two file columns

I have long text files with space-delimited fields: cat file1.txt Id leng sal mon 25671 34343 56565 5565 44888 56565 45554 6868 23343 23423 26226 6224 77765 88688 87464 6848 66776 23343 63463 ...
6
votes
5answers
17k views

join : “File 2 not in sorted order”

I've got two files _jeter3.txt and _jeter1.txt I've checked they are both sorted on the 20th column using sort -c sort -t ' ' -c -k20,20 _jeter3.txt sort -t ' ' -c -k20,20 _jeter1.txt #no ...
6
votes
4answers
6k views

Compare two files: lines present in one, not in the other, by one column comparison

I need to compare 2 files. Column 1 is the same in both files. Column 2 is what I want to compare: I want all lines in file 1 that are not in file 2 when comparing column 2. Column 3 is different in ...
6
votes
4answers
1k views

Compare an old file and new file, but ignore lines which only exist in new file?

I have two files: oldlist - This contains a list of files and a md5 hash for each file. This was generated one year ago. newlist - This also contains a list of files and a md5 hash for each file. ...
5
votes
3answers
384 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
5
votes
2answers
88 views

Merging/Combining 2 text files according to numeric field

I have 2 text files file_1: 14595|Age 35|Salary xx|Position ax|2013|Info 1|Info 2|Info 3|Info 4|Info 5 14649|Age 30|Salary xx|Position az|2015|Info 1|Info 2|Info 3|Info 4|Info 5 file_2: ...
4
votes
3answers
5k views

Comparing files line by line in awk with delimiter

file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: abc|123|check def|456|map ijk|789|equator lmn|101112|globe EXPECTED OUTPUT: ijk|789|equator lmn|101112|globe Current ...
4
votes
2answers
102 views

Awk: compare two files

I have a file1.txt USA Joe 123.123.123 Russia Marry 458.786.892 Canada Greg 151.844.165 Latvia Grace 125.895.688 and file2.txt 1 123.123.123 2 151.844.165 3 465.879.515 and I want to create a ...
4
votes
2answers
217 views

Joining two files

I have a 2 files with the following: File1.txt A 1 B 2 C 5 Z 3 File2.txt A 4 B 7 C 10 D 11 What I would like to do is create something like A 1 4 B 2 7 C 5 10 D - 11 Z 3 - Is there a ...
4
votes
1answer
3k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
4
votes
4answers
545 views

Merge two files

I have this file1.txt deiauk 9 kespaul 8 luktol 7 titkur 6 and other file2.txt kespaul b deiauk a And I want to merge both files in one by first value, so my result should be deiauk 9 a ...
4
votes
5answers
520 views

Join two files, matching on a column, with repetitions

How can I get two files A and B, and out put a result like this: File A: 001 Apple, CA 020 Banana, CN 023 Apple, LA 045 Orange, TT 101 Orange, OS 200 Kiwi, AA File B: 01-Dec-2013 01.664 001 ...
4
votes
4answers
3k views

Merge 2 files based on all values of the first column of the first file

I need to merge below 2 files: file1: TABLES REF-IO HEAD-IO DIFF-IO test 200 500 -300 exam 2 3 -1 final 2 1 1 mail 4 2 2 TOTAL 208 506 -298 ...
4
votes
2answers
165 views

Adding column based on matching of second column

I have two files. The first file has the following format: 10D0325 2465 0 0 -9 -9 10D0598 2567 0 0 -9 -9 10D0562 2673 0 0 -9 -9 10D0175 2457 0 0 -9 -9 10D0241 2209 0 0 -9 -9 10D0954 2312 0 0 -9 -9 ...
4
votes
1answer
2k views

intersection of two files according to the first column

I have two files in file A, there are sequence_numbers in the other file B, there are many columns, and the first column is sequnce numbers, I want to get a files with all the lines in the B with the ...
4
votes
1answer
2k views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
4
votes
1answer
66 views

Truncated result returned by JOIN

I'm using join command under linux, but the results vary between different machines. I have two simple files: cat 1.txt a aaa,0.2 b bbb,0.3 c ccc,0.5 cat 2.txt a aaa,0.2 b bbb,0.3 c ccc,0.6 I'm ...
3
votes
3answers
1k views

Using grep/awk/sed to sort and combine 2 files

Ok I am running into a problem. I am taking a wifi log that has MAC addresses listed in it. I want to find out what MAC vendors/manufactors for the devices that are connected to my router. I have 2 ...
3
votes
6answers
496 views

How to combine contents of two files having the same value in a column?

File1: 00:00 274 00:04 476 00:05 450 00:06 499 00:07 373 00:08 206 00:09 471 00:10 154 File2: 00:00 183 00:01 60 00:02 344 00:03 540 00:04 450 00:07 348 00:09 473 00:10 203 Desired output: ...
3
votes
5answers
307 views

Merging Two Files columns in order

I want to merge two files. Files A.txt 001;abc;def;ghi;jkl;pqr 002;abc;def;ghi;jkl;pqr 003;abc;def;ghi;jkl;pqr 004;abc;def;ghi;jkl;pqr 005;abc;def;ghi;jkl;pqr . Second File B.txt 001;mno 002;mno ...
3
votes
2answers
3k views

Merging 2 files with based on field match

I want to create a file that contains columns from two input files. File1 is like: aa 32 bb 15 cc 78 File2 is: fa 19 bc 23 cc 50 de 28 aa 45 bb 31 The task is, read through File1, if the 1st field ...
3
votes
3answers
1k views

Merging content of two files with comparing content of column

I have a problem trying to merge content of two files : file 1 : 873 1.1.1.1 1033 99.99.99.10 1108 78.2.204.174 1257 78.2.8.181 file 2: test1 78.2.8.181 p6 test2 99.99.99.10 p9 test7 1.1.1.1 ...
3
votes
2answers
257 views

add fields from different files

I have two files A and B with the same number of lines. Each of them has multiple columns. I want to add field 6 in file A with field 3 in file B for each line. e.g.: file A: 2769 RUT10EKl3fY 0 ...
3
votes
2answers
360 views

Collate tab-delimited files

I want to compare the information from mapping files to retrieve crosslinked information in a tab-delimited file format. For example, one file contains COG0001 882.DVU3168 COG0002 883.DvMF_2502 ...
3
votes
2answers
347 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
3
votes
2answers
217 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
3
votes
5answers
403 views

source multiple files and output one file

I have a file name "conf1" containing variables like: name='john' last='' custom='1000' and another file name conf2 like this: name='john' last='star' I want to merge between them to one file ...
3
votes
2answers
551 views

Merge some tab-delimited files

I have 100 files with 57,816 rows each.I want to do an outer join of these files by merging on basis of the common column. I am using R programming for this but this is very slow. fileList <- ...
3
votes
3answers
762 views

comparing the first column of two files and printing the entire row of the second file if the first columns match

I have two files in these formats: file1 : air smell hand dude road cat file 2: air,4,21,01,13,3,2 smell,21,4,2,5,6 dude,1,31,42,1 road,1,4,2,1,4 cat,1,5,6,3,1 hand,1,4,2,1,6 mouse,1,3,5,6,2 ...
3
votes
4answers
310 views

Merging Two Files with Difference only

I have two files. file1 contains text as follows: #This Data written by the platform Team id=kingDB file2 contains the text as follows: #This Data written by the platform Team id=kiteDB,mangoDB ...
3
votes
2answers
275 views

join files with numbered index

I am trying this command: join -a1 -11 file1 file2 > file3 file1 looks like: 1 2 3 4 5 6 7 8 9 10 11 file2: 1 lkj klj lkj 2 lkj lkj lkj 3 7 lkj lkj lkj 8 9 11 ...
3
votes
2answers
58 views

Join (large) files on alphanumeric pattern

I have: File 1 like: sting_of_printable_characters*sting_of_printable_characters*sting_of_printable_characters*ALPHANUMERIC_PATTERN File 2 like: sting_of_printable_characters*ALPHANUMERIC_PATTERN ...
3
votes
1answer
48 views

comparing 2 lists with additional data to be printed

I have a file lets say file1.txt ENSG00000223116 AL157931.1 ENSG00000233440 HMGA1P6 ENSG00000207157 RNY3P4 ENSG00000229483 LINC00362 ENSG00000252952 RNU6-58P I have another file say ...
3
votes
1answer
897 views

merging files and getting column values based on id field

bash-3.2$ cat sample.log sample.log.1 sample.log.2 ID COL1 COL2 COL4 1 col1 col2 col4 2 c1 c2 c4 3 co1 co2 co4 ID COL3 COL1 1 col3 col1 2 c3 c1 3 co3 co1 ID COL1 COL2 COL3 1 col1 ...
3
votes
2answers
60 views

Print common lines between two files along with their line numbers in those files

I have two files. I want to compare the content of one file with the other one. If there is a matching line between both files then print the line and its line number in each file. Example: File 1: ...
3
votes
2answers
131 views

Output parameter in joining two matrices?

I have to join two matrices. $cat mat1 : sample gen1 gen2 gen3 gen4 pt1 1 4 7 10 pt3 5 5 8 11 pt4 3 6 9 12 $cat ...
2
votes
2answers
251 views

Merging two tables including multiple ocurrence of column identifiers

I would like to merge two tables: File 1: 1 today 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new Desired output (file 3): 1 today ...
2
votes
3answers
5k views

Find common elements in a given column from two files and output the column values from each file

I have two files with tab-seperated values that look like this: file1: A 1 B 3 C 1 D 4 file2: E 1 B 3 C 2 A 9 I would like to find rows between files 1 and 2 where the ...
2
votes
5answers
142 views

compare data in two files one has 2 column while other has 3 column

I have two files: File 1: Chromosome Position Chr1 5242097 Chr1 4876397 Chr1 9474943 Chr1 8494518 Chr1 39872693 Chr1 9445153 Chr1 23044399 Chr1 8018859 Chr1 9474943 Chr1 ...
2
votes
2answers
189 views

Removing lines with a single common field

I have a tab separated file that looks like this: 123 some text 123 some different text 334 some other text 341 more text and I want to do two things. One is to order everything numerically ...
2
votes
2answers
271 views

Join two files with different number of columns and rows

I have two files. file_1.txt looks like this: R1 C1 C2 C3 C4 C5 R2 C1 C2 C3 C4 C5 R3 C1 C2 C3 C4 C5 R4 C1 C2 C3 C4 C5 ...
2
votes
2answers
94 views

How can I compare two files and create another file that aren't in first file?

file1 contains: 100 200 300 400 file2 contains: 500 600 700 200 300 Expected output file will be: 500 600 700
2
votes
3answers
2k views

compare files line by line and create new one bash programming

I have two text files. File 2 has logs over 1,000,000. File 1 has IP addresses line by line. I want to read file 2 lines and search these lines in file 1, I mean: file 1: 34.123.21.32 45.231.43.21 ...