1

I am trying to join 2 large files(1 file is 699M and other is 20GB). Both the files have only 2 columns. The command I am using is: join -1 2 -2 1 -t , -o 1.2,0,2.2 file1 file2. When I try the command using a smaller file1, the command works fine. I tried printing $? to see if the exit code is non zero, but it is not.

How can I debug to find out the reason?

Some info:
O.S: RHEL5
Arch: x86_64
  • is file1 and file2 sorted? – h3rrmiller May 22 '13 at 19:41
  • @h3rrmiller: Yes, it is sorted ! – name_masked May 22 '13 at 19:46
  • strace -fv join ... – ott-- May 22 '13 at 21:21
2

Both files must be sorted on the join fields, and they must be sorted using the same order that is also used by join. A common problem is that the files are sorted using, say, LC_COLLATE=C, but the current value of LC_COLLATE (that is now used by join) is something else, such as de_DE. (LC_COLLATE=C and LC_COLLATE=de_DE differ when upper and lower case letters are compared, and they also treat whitespace differently.)

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.