Since your question misses the important information:
the only way to answer your question is to give a lot of background information.
There are three basic diff
implementations:
The original UNIX diff based on the algorithm from Harold Stone. This usually comes in a herd together with diffh
a simple diff for huge files and bdiff
, a frontend to diff
that calls diff
with segmented parts of both files. This is the oldest diff implementation and a highly portable version of this herd of diff implementations is part of the schilytools.
The programm udiff from the schilytools. This program is nearly 40 years old.
GNU diff an implementation that is approx. 30 years old.
UNIX diff
is from the early 1970s and has been written for tiny machines with very limited memory. It still needs allocated RAM, typically 5*sizeof(int) * sum_of_lines.
The stone algorithm searches for the best resync match after an insertion or deletion and this consumes CPU time that is proportional to the square of the sum of the lines in the files. This limits the typical usability on recent hardware to a few megabytes per file until the diff operation takes too much time.
bdiff
is a frontent for diff
and automatically splits the files into chunks, runs diff
with the chunks and finally combines the diff output. The resulting diff output is no longer optimal in case that a difference spans the position where files ar split. bdiff
has been originally developped for SCCS
in the late 1970s to speed up operation.
diffh
is a separate program that uses CPU time that is approx. proportional to the file size but does an extremely halfhearted job for resyncing.
udiff
This program has been originally written in 1982. It produces an output that in most cases (if called as fdiff
) is identical to the output from UNIX diff, but resyncing does not always works as nice as with the original diff. It consumes as much memory as the original UNIX diff but the consumed CPU time is only linear proportional to the size of the files.
GNU diff
Tries to give the same quality of resyncing as the original UNIX diff, but it consumes more RAM than the sum of the sizes of both files. This makes GNU diff a memory pig as it prevents it from being useful for huge files even though the CPU time consumption only increases linear with file size which makes GNU diff the best choice for mid size files of a few megabytes.
Recent versions of all of these programs have no line length limitation.
With typical file sizes, udiff
(and it's sisters fdiff
and fsdiff
) are the fastest diff implementations. Since it does not try to slurp in both files (if they are larger than 64 MBytes) udiff
is useful for huge files and it seems to be the best choice for your problem.
I recommend you to download a recent version of the schilytools from:
http://sourceforge.net/projects/schilytools/files/
compile, install and then run:
fdiff file1 file2
split
on the files and then doing the diff?man mkswap swapon sudo
.wc
.gdiff
at all in this situation, this will still take a lot of time to finish.