0

What I'm trying to do is loop over directories and output information about their child directories in a very specific format so that is suitable for the user and also something that can be used parsing later with tools such as awk, grep, and sed.

This will be run inside of function in a bash script and print out to the users terminal. Much like an ls would, but with specific columns, order and fields. I will build on this to basically run this function again, piping it to something like awk catching just the directories I care about, then running other tasks like diff on them.

~/Parent/2017
Directory Name    Size       Last Modified     Number of Files                 
2017-08-25        1.9 GB     Aug 30   16:39    141
2017-08-26        152 MB     Aug 27   12:57    48
2017-08-29        5.9 GB     Aug 30   7:19     462
2017-08-30        102 MB     Sep 2    14:36    20
2017-08-31        2.0 GB     Sep 1    9:18     148
2017-09-01        1.2 GB     Sep 5    10:21    89

When my Parent directory looks something like this

> ls -l ~/Parent/
2015
2016
2017
Backups
Catalog-file.data
'Temporary Folder DE2B14BB-1052-4FF0-B4E7-B94C0E3CC6F8'
backup-logs
rsync.log

So far I have something like this

  IFS=$'\n';
  for d in `find -maxdepth 1 -type d | grep -P "\d{4}" | grep -v -P "[a-z]+"`; do
    for sd in `find ${d} -maxdepth 1 -type d`; do
      file_count=`ls -lh ${sd} | wc -l`
    done
    ls -lh ${d} | awk -v fc="$file_count" '{printf "%-13s %-8s %-5s %-5s %-8s %-8s \n", $8, $4, $5, $6, $7, fc}'
  done

I know it's not working correct. I'm just putting out the code I have thus far and am stumped. How can I save a variable (file_count) for each row that awk outputs? FYI: I'm probably going to need this so I can replace the ls "Size" column with a du -sh directory_name variable as well.

Normally I could continue to work through this. But I guess it's when I hit this point I thought I'm probably doing this the hard way and there is an easier way to do this. What do you guys think? Two loops like this? Or is there a simpler way.

3
  • It would be useful to describe what you're trying to do, rather than have us guess from your code. Oct 2, 2017 at 6:00
  • Sure, my apologies. I edited the original post to be a bit more clear and add some more context to my intentions with this. Hopefully this provides enough details. If not please let me know parts are still unclear, or what information I could provide to help answer this question. Oct 2, 2017 at 6:11
  • Another tool I've found with awk is getline. I'm trying a new statement like ls -lh ${d} | awk 'BEGIN {"ls -lh| wc -l" | getline count; close("ls -lh | wc -l")}{printf "%-13s %-8s %-5s %-5s %-8s %-8s\n", $8, $4, $5, $6, $7, count}'. I also spent some time searching for ways to do this with perl. I think perl maybe the best for this, but I'm an absolute beginner with perl Oct 2, 2017 at 8:18

2 Answers 2

3

With GNU tools, you could do:

find . \( -path './*/*' -o -type d \) -printf '%TF %TH:%TM/%s/%p\0' |
  gawk -F/ -v 'RS=\0' '
    BEGIN{split("K/M/G/T/P/E/Z/Y", u)}
    function h(n, i) {
      while (n >= 1024) {n /= 1024; i++}
      return sprintf("%.4g%s", n, u[i])
    }
    {s[$4] += $2; n[$4]++}
    NF == 4 {t[$4] = $1}
    END{
      PROCINFO["sorted_in"] = "@ind_str_asc"
      for (d in t) printf "%-13s %-8s %s %s\n", d, h(s[d]), t[d], n[d]
    }'

That counts the number and cumulative apparent size (not disk usage) of all the files in the directories, recursively (add a -maxdepth 2 to find if you don't want to consider files at deeper levels). For disk usage, you'd need to count the block counts and only count hard links once which would become more complicated.

1
  • Beautiful! Works perfect. Thanks @Stéphane Chazelas Oct 2, 2017 at 11:45
0

Addressing one specific part of your script, are you familiar with GNU find command's -printf action? It's there to do customized display of file information along the lines of what you're doing. You might be able to simplify your script if you do file search and display at the same time.

To illustrate the flexibility this command:

find -name '*.sh' -printf 'Script %P last accessed on %D at %r. Owned by %u with permissions set to %M'

...results in something like...

Script foo.sh last accessed on 03/01/17 at 10:15 PM. Owned by joe.blow with permissions set to -rwx-r-x-r-x
Script bar.sh last accessed on 03/22/17 at 02:42 AM. Owned by joe.blow with permissions set to -rw--r---r--

There are about 30 different directives for just about every file attribute you can think of and the date formatting is based on strftime so it's plenty powerful/flexible. The flags have the usual modifiers, too, like left/right justification, minimum number width, etc.

1
  • @B Layer I have seen it before, but I had forgotten about it. Thanks for reminding me. I will explore the options with this. Thank you. Oct 2, 2017 at 8:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .