0

Is it possible to use xargs to run more than one command? The only related information I found was this question, but the answer was not related to xargs.

I'm running a simple

du / -ah | sort -r -n | head -n 10 |  awk '{print $2}'

to find the files that occupy most disk space. However, it is returning also directories, and I want to delete only files in order to keep any structure.

I am weary of using find command, because I am not sure if I can search by disk usage, and not file size.

Any input would be super appreciated!

5
  • See: unix.stackexchange.com/q/22432/216907 all you need is find to filter only regular files towards du.
    – thanasisp
    Commented May 9, 2022 at 21:48
  • Also be sure to sort before head, like into the post linked to your previous question, in order to get the files you need.
    – thanasisp
    Commented May 9, 2022 at 21:52
  • Are you using GNU/LInux?
    – thanasisp
    Commented May 9, 2022 at 22:00
  • Hello @thanasisp! Thanks for your reply. Yeah, I'm using Debian. You're right about the sort before the head, I don't know what the heck was I thinking, my bad! Thanks for pointing it out. I was gonna be scratching my head for a while there. Thanks! Commented May 10, 2022 at 20:28
  • Welcome, see also linked posts.
    – thanasisp
    Commented May 10, 2022 at 20:37

2 Answers 2

1

You can use find to get only regular files and run du for them. Standard error is supressed for the command below, because if you do it for your root directory, there could be some complaints for some special dirs. Here -h is used for both du and sort, which is good for sorting displaying file sizes.

find / -type f -exec du -h {} + 2>/dev/null | sort -hr | head | cut -f2-

Also the newline is used as the file separator for the above command. The safe way into a GNU shell is to use the null byte to separate the arguments like:

find / -type f -exec du -h0 {} + 2>/dev/null | sort -zhr | head -z | cut -zf2- | xargs -r0 <some command to run for the 10 biggest files>

For your case (searching / as any user) just print the files, together with their size, and decide what to do. Don't send the result directly to rm --.

See also:

1
  • Thank you again @thanasisp! This is a very concise and straightforward answer, I will definitely look into it and try it out. I've learnt something very important, thanks! Commented May 10, 2022 at 20:35
0

directories are one of many types of files (regular, directory, symlink, fifo, device...). Maybe you meant regular files as that's the other type of files beside directories that have disk space allocated for them¹.

du / -ah | head -n 10 | sort -r -n | awk '{print $2}'

Is wrong because with -h, you get KMGT... suffixes which means sort -n won't work. The GNU implementation of sort has a -h option to decode those suffixes but because of rounding, the order may still be wrong.

Also beware that hard links are counted only once and that file paths may contain newline characters and not be made of text. Not to mention that your awk returning the second whitespace delimited field will choke on file paths containing whitespace. 0 is the only byte value that can't occur in a file path, so you want to use NUL-delimited records to represent file lists.

-h is a GNU extension. The GNU implementation of find can report disk usage with -printf %b or -printf %k, so you could do something like:

find . -type f -printf '%b\t%p\0' |
  sort -zrn |
  numfmt -d $'\t' -z --to=iec --from-unit=512 |
  tr '\0' '\n'

Note that the tr is only to output it for human consumption, if you wanted to post-process that list, you'd want to keep the format NUL-delimited. For instance, to delete the 10 largest (in terms of disk usage) files:

find . -type f -printf '%b\t%p\0' |
  sort -zrn |
  head -zn10 |
  cut -zf 2- |
  xargs -r0 rm -f

(-printf, -z, -r, -0 are all GNU extensions, though xargs -0 is now supported by a few other implementations; -r as well to a lesser extent).

Technically, rm unlinks the files from their parent directory. You might find that that pipeline returns the same file linked to 10 different directories; you'd then unlink it from them, but if that file is still linked to some more directories, you won't reclaim its space.

If you wanted to delete all the entries (links) in the directories under . of the 10 largest files, you could do:

find . -type f -printf '%b\t%D:%i\t%p\0' |
  sort -zrn |
  gawk -F'\t' -v RS='\0' -v ORS='\0' '!seen[$2]++ && ++n > 10 {exit}1' |
  cut -zf 3- |
  xargs -r0 rm -f

¹ beware that for files of type directory, du reports the disk usage of the directory file itself, plus that of any unique file that it lists and those of its subdirectories recursively.

1
  • Hello @Stéphane! Thanks a bunch for your reply. It is indeed a very elegant and precise way of achieving what I want. I have to reckon, for a rookie like me, it looks a bit daunting, but surely it's food for thought, and hopefully I will understand a bit more the realms of gawk. This reply is indeed extremely complete and detailed. I hope I get to understand it in it's entirety; mostly the gawk part. Thanks a lot for sharing your knowledge! Commented May 10, 2022 at 20:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .