Sum the total number of lines of code in all tar files in all subdirectories

Question

I would like to calculate the sum of the total number of lines of all tar files in all subdirectories.

Consider the following example directory, se-example:

se-example
├── directory1
│   ├── archive1.tar
│   └── archive2.tar
└── directory2
    ├── README.txt
    ├── archive3.tar
    └── directory3
        └── archive4.tar

To recreate this example, consider the following:

echo "create and enter example"
mkdir se-example
cd se-example

echo "create example directory 1 (w/ two tar files)"
mkdir directory1
cd directory1

echo "create example tar 1 [+7 lines]"
mkdir archive1
printf "Line 1 \n Line 2" >> archive1/README.txt
printf "Line 1 \n Line 2" >> archive1/code1.py
printf "Line 1 \n Line 2 \n Line 3" >> archive1/code2.py
tar -cf archive1.tar archive1
rm -rf archive1 

echo "create example tar 2 [+5 lines]"
mkdir archive2 
printf "Line 1 \n Line 2 \n Line 3" >> archive2/code1.py
printf "Line 1 \n Line 2" >> archive2/code2.py
tar -cf archive2.tar archive2
rm -rf archive2

cd ..

echo "create example directory 1 (w/ subdirectory, readme, and two tar files)"
mkdir directory2
cd directory2

echo "create example readme [+0 lines]"
printf "Line 1 \n Line 2" >> README.txt

echo "create example tar 3 [+1 line]"
mkdir archive3 
printf "Line 1" >> archive3/code1.py
tar -cf archive3.tar archive3
rm -rf archive3

echo "create example subdirectory (w/ one tar file)"
mkdir directory3
cd directory3

echo "create example tar 4 [+5 lines]"
mkdir archive4
printf "Line 1 \n Line 2 \n Line 3" >> archive4/code1.py
printf "Line 1 \n Line 2" >> archive4/code2.py
tar -cf archive4.tar archive4
rm -rf archive4

cd ..
cd ..
cd ..

echo "done creating example"

In this example, there are four tar files with a total of 18 lines (7 lines in tar 1, 5 lines in tar 2, 1 line in tar 3, and 5 lines in tar 4). The correct answer would give 18.

I know how to count the number of tar files (suggested here):

find . -maxdepth 1 -mindepth 1 -type d | while read dir; do
  printf "%-25.25s : " "$dir"
  find "$dir" -name "*.tar" | wc -l
done

I know how to count the number of lines of code in individual tar files (suggested here):

tar -tf se-example/directory1/archive1.tar | wc -l

How do I combine these two commands, or is there an efficient alternative that will solve this problem?

Answer 1

You're missing the newline \n for the last line in each of your test files, so the count of wc is actually less:

$ find . \( -name "*.py" -o -name "README*" \) -print0 | xargs -0 wc -l
  1 ./directory1/archive1/README.txt
  1 ./directory1/archive1/code1.py
  2 ./directory1/archive1/code2.py
  2 ./directory1/archive2/code1.py
  1 ./directory1/archive2/code2.py
  1 ./directory2/README.txt
  0 ./directory2/archive3/code1.py
  2 ./directory2/directory3/archive4/code1.py
  1 ./directory2/directory3/archive4/code2.py
 11 total

Since /directory2/README.txt is not in an archive, we have 10 lines in total.

1. List the files in the archives:

   $ find . -name "*.tar" -print0 | xargs -0I{} tar tf {}
   archive1/
   archive1/README.txt
   archive1/code1.py
   archive1/code2.py
   archive2/
   archive2/code1.py
   archive2/code2.py
   archive3/
   archive3/code1.py
   archive4/
   archive4/code1.py
   archive4/code2.py
   

2. Extract the files to stdout with the -O / --to-stdout option:

   $ find . -name "*.tar" -print0 | xargs -0I{} tar xOf {}
   Line 1
    Line 2Line 1
    Line 2Line 1
    Line 2
    Line 3Line 1
    Line 2
    Line 3Line 1
    Line 2Line 1Line 1
    Line 2
    Line 3Line 1
    Line 2
   

   (The last line Line 2 has no newline)

3. Count the lines with wc:

   $ find . -name "*.tar" -print0 | xargs -0I{} tar xOf {} | wc -l
   10