5
votes
1answer
102 views

Merge nonzero blocks of huge (sparse) file A into huge file B

I have two partial disk images from a failing hard drive. File B contains the bulk of the disk's contents, with gaps where sector reads failed. File A is the result of telling ddrescue to retry all ...
2
votes
2answers
13k views

Easy way to copy lines from one file to another [duplicate]

Possible Duplicate: cat line X to line Y on a huge file Very simple issue but can't seem to find a simple resolution! I have a massive text file from which I only need around 150 lines. ...
3
votes
2answers
1k views

how to find offset of one binary file inside another?

I have two binary files. One of few hundreds kilos and other of few gigabytes. I want to know whether the whole, smaller, file is contained within the larger one and if so then what is the offset from ...
37
votes
5answers
20k views

How to remove duplicate lines inside a text file?

A huge (up to 2 GiB) text file of mine contains about 100 exact duplicates of every line in it (useless in my case, as the file is a CSV-like data table). What I need is to remove all the repetitions ...
15
votes
5answers
3k views

How can I edit a large file in place?

I have a few files sized > 1 GB each. I need to remove last few bytes from the files. How can I do it? I prefer to edit file in place to save disk space. I am on HP-UX.