2

I need to grab the first lines of a long text file for some bugfixing on a smaller file (a Python script does not digest the large text file as intended). However, for the bugfixing to make any sense, I really need the lines to be perfect copies, basically byte-by-byte, and pick up any potential problems with character encoding, end-of-line characters, invisible characters or what not in the original txt.

I think it is no problem for a text file to end mid-line, so copying some specific bytesize would accomplish what I intend to do. Both head and dd seem to be able to do this, but man head says to work on text files and man dd to copy the standard input to the standard output (and offer conversion tools), I could not verify that they do the copy bit-to-bit, without any conversion or loss. Thus the following two solutions seem to work in tests, but I am not sure I am testing them on any possible problem characters. A definitive answer or another preferable solution would be most helpful.

Either

dd if=input.txt of=output.txt bs=256 count=1

or

head -c 256 input.txt > output.txt

Also see the original, more specific question here.

  • 2
    Definitely dd is a exact byte by byte copy. You may want to consider adding a trailing new-line to the new file to make the last-part-of-a-line into a line. – user79743 Mar 3 '16 at 12:21
  • 1
    The trouble with dd that I see is that since dd does not know about character encoding, it will happily stop in the middle of a multi-byte utf8 character. – Ulrich Schwarz Mar 3 '16 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.