I have a perplexing problem. I have a library which uses sg for executing customized CDBs. There are a couple of systems which routinely have issues with memory allocation in sg. Usually, the sg driver has a hard limit of around 4mb, but we're seeing it on these few systems with ~2.3mb requests. That is, the CDBs are preparing to allocate for a 2.3mb transfer. There shouldn't be any issue here: 2.3 < 4.0.
Now, the profile of the machine. It is a 64 bit CPU but runs CentOS 6.0 32-bit (I didn't build them nor do I have anything to do with this decision). The kernel version for this CentOS distro is 2.6.32. They have 16gb of RAM.
Here is what the memory usage looks like on the system (though, because this error occurs during automated testing, I have not verified yet if this reflects the state when this errno is returned from sg).
top - 00:54:46 up 5 days, 22:05, 1 user, load average: 0.00, 0.01, 0.21 Tasks: 297 total, 1 running, 296 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 15888480k total, 9460408k used, 6428072k free, 258280k buffers Swap: 4194296k total, 0k used, 4194296k free, 8497424k cached
I found this article from Linux Journal which is about allocating memory in the kernel. The article is dated but does seem to pertain to 2.6 (some comments about the author at the head). The article mentions that the kernel is limited to about 1gb of memory (though it's not entirely clear from the text if that 1gb each for physical and virtual or total). I'm wondering if this is an accurate statement for 2.6.32. Ultimately, I'm wondering if these systems are hitting this limit.
Though this isn't really an answer to my problem, I'm wondering about the veracity of the claim for 2.6.32. So then, what is the actual limit of memory for the kernel? This may need to be a consideration for troubleshooting. Any other suggestions are welcome. What makes this so baffling is that these systems are identical to many others which do not show this same problem.