I have a git repository full of files, mostly text.

I want to know that there are

  • n lines of code in files ending in .py
  • m lines of code in files ending in .md
  • o lines of code in files ending in .yaml
  • p lines of code in files with no extension
  • etc


  • I want this to be recursive, checking inside folders
  • I want to exclude the .git directory in the top level folder
  • I want to ignore binary files is possible (noting that I have some text files with no extension, and some binary files with no extension)
  • I want to be case insensitive. Group .csv with .CSV
  • I want to ignore empty (or only whitespace) lines
  • If there's a file like myfile.yaml.j2 I do not care whether it's counted in the .j2 group or .yaml.j2.
  • What about empy/blank lines? do they count too?
    – Jetchisel
    May 1, 2020 at 4:52
  • I want to exclude them. The solution I wrote below includes them. I thought there were options to pass to wc for this, but I can't see them. May 1, 2020 at 5:34
  • 3
    Utility for counting lines of code is cloc(linux.die.net/man/1/cloc). Study the manual and see, what it offers. It will probably not fulfill all the requirements you have, but maybe you will find cloc output informative, and you will find out that it is what you need.
    – nobody
    May 1, 2020 at 6:02
  • Could you say that you want to ignore all hidden directories, or just .git? May 1, 2020 at 16:51
  • Ah yes, all hidden directories would be nice May 2, 2020 at 8:29

3 Answers 3


Try this:

find ./ -not -path "./.git/*" -type f -exec wc -l {} + |
    awk '{print tolower($0)}' |
    sed -e '$ d' | 
    sed -e "s#/.*/##g" |
    sed -e "s/\./ \./g" |
    awk '
        { if ( NF <= 2 ) { count["none"] += $1 } else { count[$NF] += $1 } }
        { next }
        END { for (group in count) printf("%d%s%s\n", count[group], OFS, group) }
    ' |
    sort -n

Broken down:

  • find ./ find objects under this directory, recursively
  • -not -path "./.git/*" excluding .git
  • -type f files not directories
  • -exec wc -l {} + for each file, run the word count utility (wc).  This includes empty lines, so doesn't meet all of the question's requirements.
  • awk '{print tolower($0)}' make lowercase
  • sed -e '$ d' delete the last line, which is the sum of lines over all files
  • sed -e "s#/.*/##g" remove the path of the file, e.g., a/something.egg/blah should count as no extension, not .egg/blah extension
  • sed -e "s/\./ \./g" search/replace . with ., so the file extension is its own word
  • awk '{ if ( NF <= 2 ) { count["none"] += $1 } else { count[$NF] += $1 } } { next } END { for (group in count) printf("%d%s%s\n", count[group], OFS, group) }' this is a big one.  awk is powerful, but not super clear
    • count is a dictionary
    • if (NF <= 2) if there are fewer than 3 'words', i.e., there is no extension
    • count["none"] += $1 increment an element in the dictionary, key is string literal none, increment it by adding the number of lines in this file, which is the first word, which is $1
    • count[$NF] += $1 increment an element in the dictionary, key is $NF (the last word in the line), which is the extension, by $1 (the first word in the line), which is the number of lines in this file
    • { next } repeat for all lines
    • for (group in count) a for loop, inline
    • printf(...) format the output string as number extension; e.g., 123 .abc (if there are 123 lines in files ending with .abc)
  • sort -n sort the results in ascending order; -n means sort as a number, not string
  • You don't need {next} in your awk script as that's implied May 1, 2020 at 7:21
  • -not is not an operator supported by find. I guess that you intended to use !.
    – schily
    May 1, 2020 at 7:34
  • I tweaked this a little bit for my use because I found for a large repository the totals were getting printed intermittently (and not only the last line) which messed up the 'none' counts. ` find ./ -not -path "./.git/*" -type f -exec wc -l {} + | awk '{print tolower($0)}' | sed -e "s#/.*/##g" | sed -e "s/\./ \./g" | awk ' { if ( $2 == "total" ) { count["total"] += $1 } else if ( NF <= 2 ) { count["<no-file-extension>"] += $1 } else { count[$NF] += $1 } } { next } END { for (group in count) printf("%d%s%s,", count[group], OFS, group) } ' Dec 6, 2022 at 17:50
  • Please note that for folders with many files the first step will output multiple nnnnn total rows and removing the last row is not enough. This will inflate the number of lines for files with no extension.
    – tmaj
    Mar 8 at 5:47

If I understand you correctly, and my tests are good, I propose this (assuming you want to skip hidden directories and files, tell me if that's not the case):

shopt -s globstar

declare -A arr
for f in test/**; do
  # if a directory, skip
  [[ -d "$f" ]] && continue
  # strip the extension
  # convert it to lowercase
  # if no dot in the name, extension is "empty"
  [[ ! $(basename "$f") =~ \. ]] && ext="empty"
  # count the lines
  lines=$(wc -l "$f"| cut -d' ' -f1)
  # if lines equals to 0, skip
  [[ $lines -eq 0 ]] && continue
  # append the number of line to the array
  lines=$(( "${arr[$ext]}"+$lines ))

# loop over the array
for n in ${!arr[@]}; do
  echo "files $n: total lines ${arr[$n]}"

Output (from my example files):

files yaml: total lines 3
files md: total lines 3
files empty: total lines 4
files csv: total lines 6
files py: total lines 5
  • It looks good, but I get bash: ""+23 : syntax error: operand expected (error token is """+23 "). Are you using bash? Do I have to invoke like . script.sh or just normal ./script.sh? May 2, 2020 at 8:34
  • @falsePockets For now (I deleted al my tests files), try with the line [[ ! -z "${arr[$ext]}" ]] && lines=$(( "${arr[$ext]}"+$lines )) May 2, 2020 at 14:45
  • I will recreate the files and let you know May 2, 2020 at 14:47

I broke things up into functions to try to make it easier to understand:


# For the next two functions, we will use "-print0", which will print out \0 instead of \n.
# This will help prevent whitespace problems when piping the filenames into xargs.

    find "$1" -type f -name "*.$2" -print0 2>/dev/null

    find "$1" -type f -regex '^.*/[^.]+$' -print0 2>/dev/null

    xargs -0 cat

    sed -E '/^[[:space:]]*$/d'

    concat_files | delete_empty_lines | wc -l

    echo "Usage: $0 [EXTENSION]... [SEARCH_DIRECTORY]";

SEARCH_DIR="${*: -1}"

if [ $# -lt 2 ];
    echo "Not enough parameters.";
    exit 1;

if ! [ -d "$SEARCH_DIR" ];
    echo "$SEARCH_DIR does not exist, or is not a directory."
    exit 1;

    printf ".$EXTENSION: %s\n" $(find_extension "$SEARCH_DIR" "$EXTENSION" | line_count_of_files)

printf "No extension: %s\n" $(find_no_extension "$SEARCH_DIR" | line_count_of_files)

This is more of a generic script that lets you specify arbitrary file extensions to search for. It will always search for files with no extension though.

You should save this into a file, give it executable permission, and put it in your PATH. Let's say you name it count_lines.sh. You can call it like this: count_lines.sh py md yaml ~/Code. This will search in the directory ~/Code for files ending in .py, .md, and .yaml, as well as files without an extension at all. You can choose any number of extensions to search for, just make sure you have at least one.

  • 1
    xargs -0 is non-portable, find -exec + however is
    – schily
    May 1, 2020 at 7:36
  • @schily wasn't aware of that. If you know how to fix it to use the portable solution, please make an edit. May 1, 2020 at 7:40
  • Probably by using arguments with the command, given to the functions in case you like to use functions.
    – schily
    May 1, 2020 at 7:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .