1

I use the below command to extract the filename, filesize, and date for a big directory with many subdirectories.

find MY_PATH -type f -exec ls -la --block-size=GB --time-style=+%Y-%m-%d_%H:%M:%S {} \; | awk  -F " |/" -v OFS=',' '{print $5,$6,$NF;}' > fileName.csv

What I want is:

  1. It works fine for files with a simple name like test.gz, but if the file has a complex name like [Name] A - A B C_D.zip; It fails and only returns the last part of the file name.
  2. Now the date is --time-style=+%Y-%m-%d_%H:%M:%S, I add _ to avoid splitting the date into two new columns. However I want to use space instead of _ but still return the date as one column.

Below is the name of some files inside one Directory; I have many directories like that.

[www.google.com] Learn - Complete SQL and Databases Bootcamp Zero to Mastery 2022.zip
[www.google.com] Learn - MongoDB - The Complete Developer's Guide 2022.zip
[www.google.com] Learn - SQL - The Complete Developer's Guide (MySQL, PostgreSQL).zip
[www.google.com] Learn - The Complete Oracle SQL Bootcamp (2022).zip

The expected output is CSV file with name,size,date columns.

3
  • edit your question to show a few examples of concise, testable sample input/output to/from the awk command that cover all your needs.
    – Ed Morton
    Feb 6, 2023 at 11:57
  • Is the reason you're using ls instead of the -printf predicate of GNU find with the appropriate format that you want GB units in the output?
    – Kusalananda
    Feb 6, 2023 at 12:25
  • @Kusalananda, Yes My files are big, and I want them as GB units. But using ls is not mandatory, I'm open to any commands that give me this output.
    – Niyaz
    Feb 6, 2023 at 12:33

1 Answer 1

1

I'd use find with printf as suggested and then process the first field to convert bytes to GiB e.g.

find . -type f -printf '%s %TF %.8TT %p\n' | numfmt --to-unit=1073741824 --format='%.2f GiB'

This assumes no newlines embedded in the filenames. It should print mtime per your OP and size in Gib with two decimals... you can always customize the format of both timestamp and size - consult the manual for find (see -printf section) and respectively for numfmt (see --format)


To have them comma-separated or in another order, as I said, adjust the formatting:

find . -type f -printf '%f,%s,%TF %.8TT\n' | \
numfmt --delimiter=, --field=2 --to-unit=1073741824 --format='%.2f GiB'

though this assumes your filenames don't contain commas. If they do, you could probably use a low ascii char as delimiter and run something like

s=$'\002'
find . -type f -printf "%f,${s}%s${s},%TF %.8TT\n" | numfmt -d${s} --field=2 --to-unit=1073741824 --format='%.2f GiB' | tr -d ${s}

Another way with zsh and the zstat module

zmodload zsh/stat
for f in **/*(.D)
do
sz=$(( $(zstat +size $f)/1073741824. ))
mt=$(zstat -F '%F %T' +mtime $f)
printf '%s,%.2f,%s\n' $f:t ${sz} ${mt}
done

Adjust %.2f if you want more (or less) than two decimals...

7
  • Thanks, @don_crissti, But still, there are two problems: 1) The solution returns the full path instead of the file name (I have a multi-level directory). 2) I want a comma between columns to convert them to a CSV file as the final destiny.
    – Niyaz
    Feb 6, 2023 at 13:45
  • Also, I updated my question to give you a better understanding.
    – Niyaz
    Feb 6, 2023 at 13:57
  • @Niyaz - so, you want to create a csv file but as I see your filenames may contain commas? You understand those need to be quoted but then, if the filenames may also contain quotes those might need to be escaped as well and this is a whole different question then? Feb 6, 2023 at 14:03
  • Since the files come from a third party, I should expect all special characters inside the file name. I can handle the problem with python, But I wonder if there is a handy way to do it by Linux command. Do you Think It is possible to do that by Linux command?
    – Niyaz
    Feb 6, 2023 at 14:10
  • 1
    I'm not much into csv, I know there are dedicated tools for parsing arbitrary output and creating csv (including escaping) but since I don't use them I can't answer your question (I would have mentioned them in my answer) Feb 6, 2023 at 15:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .