-2

Suppose I have 100 texts under the same dir, i.e., text1.txt, text2.txt,...,text100.txt. I want to extract certain lines(e.g., first 100 lines)from each text, and save the lines to another new 100 texts respectively,that is, each new text has 100 lines.

I know head -100 text1.txt > text1_new.txt, head -100 text2.txt > text2_new.txt ... can make it. But are there any more efficient methods to extract them simultaneously in the terminal?

Thanks!!

  • 1
    Are the 100 texts all in one directory? If not, do they have a common file name convention to distinguish them? – Jeff H. Apr 20 at 12:39
  • You could use a find command to loop through a directory of files: find /my/directory/of/files/ -type f -exec head -100 '{}' >> ~/consolidated.txt \; If this works for you let me know and I'll leave a detailed answer. – Patrick Apr 20 at 13:23
  • @Patrick my interpretation is that they want 100 new files, not one consolidated one. (text1__100.txt) – Jeff Schaller Apr 20 at 14:00
  • @Patrick Yes, I want to 100 new files with 100 line in each of them, not a consolidated file with 1000 lines. Thanks! – Victor Apr 20 at 14:24
  • @JeffSchaller sorry for the confusing name in my question, you're right, I want another 100 new files from original files, thanks! – Victor Apr 20 at 14:26
0

One way would be

find . -name "text*.txt" -type f -print0 | xargs -0 -I{} sh -c 'f="{}"; head -100 "$f" > "${f%.txt}_new.txt"'
  • find . -name "text*.txt" -type f finds all text files in the directory
  • -print0 prints the file path with a null character to preserve spaces
  • xargs -0 takes the null terminated arguments
  • -I{} is used as placeholder for the argument
  • sh -c executes dash with a command string
  • f="{}" saves the argument in variable f
  • head -100 "$f" the head command
  • "${f%.txt}_new.txt" replaces ".txt" with "_new.txt" in the argument
  • Instead of xargs you could find -exec sh -c 'head -100 "$1" > "${1%.txt}_new.txt"' find-sh {} \;. – pLumo Apr 23 at 14:57
0

You can iterate across the files

for f in *.txt
do
    head -n100 "$f" > "${f%.txt}_new.txt"
done

The caveat is that the *.txt will expand at the moment it's reached, to list all files that match. If you've already run this script (partially or completely) it will match some of the results files too. Solutions are either to delete them before running (rm *_new.txt), or to tighten the pattern.

0

If you want "parallel", use parallel

shopt -s extglob
parallel sh -c 'out="${1%.txt}_new.txt"; head -n 100 "$1" > "$out"' _ -- !(*_new).txt

I'm assuming your shell is bash, and using extended patterns to loop over the text files that are not the *_new.txt ones

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.