1

I am trying to join 2 large files(1 file is 699M and other is 20GB). Both the files have only 2 columns. The command I am using is: join -1 2 -2 1 -t , -o 1.2,0,2.2 file1 file2. When I try the command using a smaller file1, the command works fine. I tried printing $? to see if the exit code is non zero, but it is not.

How can I debug to find out the reason?

Some info:
O.S: RHEL5
Arch: x86_64
3
  • is file1 and file2 sorted?
    – h3rrmiller
    May 22, 2013 at 19:41
  • @h3rrmiller: Yes, it is sorted ! May 22, 2013 at 19:46
  • strace -fv join ...
    – ott--
    May 22, 2013 at 21:21

1 Answer 1

1

Both files must be sorted on the join fields, and they must be sorted using the same order that is also used by join. A common problem is that the files are sorted using, say, LC_COLLATE=C, but the current value of LC_COLLATE (that is now used by join) is something else, such as de_DE. (LC_COLLATE=C and LC_COLLATE=de_DE differ when upper and lower case letters are compared, and they also treat whitespace differently.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .