I have a long list of urls, and I'm trying to make a shell script to split each paragraph, separated by newline, into its own file. The files would need to be numbered

001 002 003 ...

and so on. Here's what I tried


    # Jakaa pitkä tiedoston kappaleet erilisiin tiedostoihin
    # Ei toimi.

    while read p; do
        if [[ "$p" == "k" ]]; then
            i=$((i + 1))
        if (( $i < 10 )); then
            printf '%s\n' "$p" | tee -a kuvat_vol00"$i"
        elif (( $i > 9 )) && (( $i < 100 )); then
            printf '%s\n' "$p" | tee -a kuvat_vol0"$i"
        elif (( $i > 99 )); then
            printf '%s\n' "$p" | tee -a kuvat_vol"$i"

I didn't know how to use the newline as delimiter, so that's why I use the letter 'k' instead. Stupid propably, I know...

The script works for a test file I made, something like:


This gives me kuvat_001 and kuvat_002 with "123" and kuvat_003 with "sdfsdf" and so on. But when I try it with the urls, it just prints everything to a single file kuvat_001. I figured it has something to do with the slashes and other funny characters in the urls. How do I make it so that the shell doesn't interpret the special characters as special? Or should I use awk or script it in vim directly or something? I just don't know how to do either of those.

I'm a little bit ashamed to ask such a stupid question, but I've struggeled with this for like three days and I can't figure it out. I completely hands down suck at shell scripting, I know, but for some reason I'm excited to learn to use the computer properly. So yeah, I appreciate any help you can give me!

EDIT: The beginning of my input file looks like this. Can it be a problem with that?



This can be done with lots of tools. I would reach for awk as it has the concept of paragraphs.

awk '
   BEGIN { RS="" }
   { F=sprintf("kuvat_%03d", NR) ; print > F ; close(F) }' input_file_name

set the RS variable to the null string to active paragraph mode. For each record (paragraph) print it to a file of the right name and close the file.

  • Thanks, that's way simpler than my solution. It still prints everything to the first file though. It must be a problem with my input file since it seems to work with any other file of similar form, so I couldn't have made a typo. – haarhuur Nov 9 '19 at 1:02
  • Maybe the "blank" lines are not blank? If it comes from a microsoft windows environment there may be CR characters at the end of each line, or maybe there are spaces? If you have the hexdump -C program then 0d is the CR and 0a is the LF which is the unix end of line character (assuming ASCII of course). – icarus Nov 9 '19 at 1:08
  • Aah yes, you're right! There's a CR in front of every LF. But if I type '/\r' in Vim, it doesn't find anything. Why is that? – haarhuur Nov 9 '19 at 1:35
  • Ah, found the solution here. Yes, now everything makes sense, thanks! – haarhuur Nov 9 '19 at 1:40

With GNU csplit utility:

  $ csplit \
      --suppress-matched -s \
       -f kuvat_vol_ -b %03d \
        file '/^$/' '{*}'  \

Using Perl:

$ perl -l -00pe '
    close F if $i;
    open F, ">", sprintf "kuvat_vol_%03d", $i++;
    select F;
 ' file 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.