I'm sorting a 25 GB file containing 200M lines on Fedora 18 using:
sort -S 10G -T /bigdisk bigfile
and I am getting process killed due to no memory. The process isn't maxing out my RAM before death (12 GB free) and disk space is ok.
Any suggestions as to the cause appreciated. I'm sure sort can manage this due to its partitioning and merge.
More info from dmesg
[87278.935572] Out of memory: Kill process 1971 (sort) score 258 or sacrifice child
[87278.935574] Killed process 1971 (sort) total-vm:4512168kB, anon-rss:4237040kB, file-rss:584kB
/run/shm
as my/bigdisk
for storing sorts temp files. It seems the kernel considers the ram disk in/run/shm
as more important, so when sort was writing to it and memory ran out, the kernel killed sort. Using a sluggish VM hard disk instead solved it. – drevicko Apr 22 '15 at 2:14