Extract range of elements from ith element to jth element, up until nth element from a list

Question

In a directory I have a certain number of files. This could be 53 txt-files, but I could also have 123 files. The files have various random names, but all have the file-handle .txt

I can get a list of all files with ls, and put this into a variable.

list_of_txt_files=$(ls *.txt)

But I would like to partition the list in to multiple individual lists, each list with only 10 elements - i.e. a folder with 53 txt-files, should give me 6 lists. That is 5 lists with 10 filenames, and a 6th list with 3 file-names, and my example with 123 txt-files in a directory should give me 12 lists with 10 file-names, and a 13th list with only 3 file-names.

With my example with 53 txt-files: list no. 1 would hold the first file up until the tenth file , and list no. 2 would hold the eleventh file up to the twentieth file, and so on. I titled my question from the ith to jth element in a list, as I suppose other people might want to break down a list differently. Perhaps from the first file to the 100th file in a directory.

The ultimate goal is to be able to use these lists in a for do loop, and use the cat command, to write out the contents of ten files per list to one file per set of ten files - i.e. in my example with 53 files in a directory, this would give me 6 files. Where the first 5 files contains the content of the 50 original txt-files, and the 6th file contains the content of the last remaining 3 txt-files.

I have considered using the head or the tail command, but cannot quite work out how to specify ranges for these two commands.

Answer 1

On a shell with arrays, use them. Say with Bash:

$ touch {01..53}
$ files=(*)
$ echo "${files[@]:0:10}"       
01 02 03 04 05 06 07 08 09 10

$ for ((i = 0 ; i < ${#files[@]} ; i += 10 )) ; do
     echo "${files[@]:i:10}" ; 
     # or
     # cat "${files[@]:i:10}" > set-$(( i / 10 ))
  done
01 02 03 04 05 06 07 08 09 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50
51 52 53

The substring expansion (array slice) "${files[@]:i:10}" expands to a list of words, not a single string, so you can loop over it:

for f in "${files[@]:i:10}" ; do
    somecmd "$f"
done

---

Don't do files=$(ls *.txt), the ls is completely redundant there, it's the shell that evaluates the wildcard anyway. Usually you just save the wildcard pattern to a variable (pat=*.txt) and use it (unquoted) where needed, or if you want to expand it to the actual file names, use echo *.txt instead of ls. For handling lists of filenames, arrays are just better, if you're not constrained to a plain POSIX shell.

---

As for splitting a list with head and tail, you'd need to do something like | head -20 | tail -10 to get lines 11 to 20. Or use sed: | sed -n 11,20p.

Answer 2

With any Bourne-like shell (but the Bourne shell that couldn't access positional elements over$9), you can do:

set -- *.txt
while [ "$#" -gt 0 ]; do
  something with "$1" ${2+"$2"} ${3+"$3"}... ${10+"${10}"}
  [ "$#" -gt 10 ] || break
  shift 10
done

With GNU xargs and shells with support for process substitution:

xargs -n10 -r0a <(printf '%s\0' *.txt) something with

With zsh:

files=(*.txt(N))
while (($#files)) {
  something with $files[1,10]
  files[1,10]=()
}

Or:

autoload -U zargs
xargs -l10 -- *.txt -- something with

Also note that you can use a range in zsh's globs:

something with *.txt([1,10])