5

I need to split a .txt file into smaller ones containing 100 lines each, including the header. I don't know if this is relevant, but the original file is delimited like this:

COLUMN1 | COLUMN2 | COLUMN3
1 | 2 | 3
4 | 5 | 6
7 | 8 | 9

I need every file generated from this split to have the header line. Also, they need to be generated in/moved to another directory and follow a name pattern, like file_01.txt, file_02.txt, etc

  • sed '1d' inputfile.txt | split -l99 outputfile will generate the files; you can then slap the header line on the resultant files with a simple for loop wrapped around head and cat. – DopeGhoti Apr 14 '16 at 20:08
  • For a POSIX-only solution, t seems it should be possible with ex to copy the header line to the 101st line, 201st line, etc., and then the buffer could be written to split, but I don't have time at the moment to explore it. :) – Wildcard Jul 11 '17 at 3:27
5

With gnu split you could save the header in a variable then split starting from the 2nd line, using the --filter option to write the header first and then the 99 lines for each piece and also specify the output directory (e.g. path to/output dir/):

header=$(head -n 1 infile.txt)
export header
tail -n +2 infile.txt | split -l 99 -d --additional-suffix=.txt \
--filter='{ printf %s\\n "$header"; cat; } >path\ to/output\ dir/$FILE' - file_

this will create 100-lines pieces as

path to/output dir/file_01.txt
path to/output dir/file_02.txt
path to/output dir/file_03.txt
..............................
  • Note that $FILE is the variable used by split for the piece name (see man split) so don't change it... :) – don_crissti Apr 14 '16 at 20:39
5
awk 'NR==1        {a=$0}
    (NR-1)%100==0 {print a > "d/file_" int(1+(NR-1)/100)}
                  {print   > "d/file_" int(1+(NR-1)/100)}' 
3

Works for me in bash:

lines=100; { read header && sed "1~$((${lines}-1)) s/^/${header}\n/g" | split -l $lines --numeric-suffixes=1 --additional-suffix=.txt - file_ ; } < inputfile.txt
  • Note: to generate files in another directory first cd to it; of course inputfile.txt must then be given as its path, not just filename. – Kamil Maciorowski Apr 14 '16 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.