5

I am trying to find the cause of some OS wonkiness, and am concerned about environment variables.

The software I have been handed puts an incredibly high amount of environment variables into the user's .profile.

When I type set | wc the result is 9571 bytes long! There are 184 entries. To me, this seems inordinately large, but I don't have anything explicit to point at and say "this is wrong because xyz".

I see nothing in the ulimit documentation that speaks about environment variable total size, but am concerned. I'm not worried about overall memory utilization (there's enough to do what I need), but I am worried about overrunning some internal limit and causing odd behavior in the OS (probably not germane to the question I'm asking, but the "odd behavior" is that shared memory queues are not giving me back all the data I put in on the other end. I'm getting around 5% out).

Every single script that starts up, every single shell that is run, and every single binary that runs gets a full, individual copy of the environment variables, and I think 9k is too big. Should this be a worry, or am I concerned over nothing?

I'm running on an embedded x86 QNX 6.4.1 Neutrino system with half a gig of RAM.

  • 500,000,000 - 9,571 = 499,990,429 – mikeserv Apr 11 '14 at 18:36
  • @mikeserv Yes, absolutely. I was wondering if there was some sort of reserved space per process that was being overfilled, not concerned with overall memory usage. – kmort Apr 11 '14 at 19:00
  • I think goldilocks is dead on. One way to get a little perspective is to cd into a directory with a ton of of little files then do ( set -- * ; echo $# ) – mikeserv Apr 11 '14 at 19:00
  • 1
    set does not give you the size of the environment, it also includes shell variables (and functions as well in some shells). Run env | wc -c to see the size of the environment. – Gilles Apr 11 '14 at 22:56
  • @Gilles You are correct, there is a difference there. I get 9364 bytes with env. Not a big one, but certainly more accurate. Thanks. – kmort Apr 12 '14 at 12:57
6

When I type set | wc the result is 9571 bytes long!

Assuming you got that number correct, it is in fact quite small, probably because you are using QNX. On a normal desktop system, it is much larger. Here's what I get on fedora 20:

> set | wc --bytes
133195

133 kB. I did not count the entries as many of them are sourced functions (git seems to install a lot of these), but I did skim them over and there doesn't seem to be anything untoward. A few kB at most is from custom stuff by me.

I am worried about overrunning some internal limit and causing odd behavior in the OS

I very much doubt this is possible because a lack of bounds checking there would indicate a very significant bug in the implementation -- preventing someone from just writing a long variable to inject data into memory has to have been a basic concern. Beyond that, as you say, 9kB is nothing memory wise. I would presume lookups on these are done with a hash table, so the number of entries would not impair performance.

  • Oh. Wow. 133kB is huge!! :-) Then again, I'm used to dealing with an embedded system. Interesting, my Win 7 box is at about 3.5kB. But, as you said, this should have been a fundamental concern. The thing that got me going on this is the shared memory issue mentioned above and a file descriptor numbering 24770 when ulimit was 1000 per process. I think I'll mark "lots of environment variables" off my list of potential problems. Thanks. – kmort Apr 11 '14 at 19:01
  • Out of sheer curiosity, what do you get when you run env | wc -c as Gilles suggests above. I'm just curious how big the difference is for you. Thanks. – kmort Apr 12 '14 at 12:58
  • env | wc- c = 3849. The difference is mostly function definitions; if I write set to a file and remove those, I'm left with ~4800 bytes. I have 70 env variables. So you do have a significant set of variables (which may, or may not represent someone's poor design choices), but again, I can't see it being anything to worry about. If there were an issue there, it would be a sort of "heartbleed" scale vulnerability. – goldilocks Apr 12 '14 at 14:49
  • Awesome. Thanks for the input. I think it is a poor design choice and have been lobbying to change it. We'll see if I ever win. :-) – kmort Apr 12 '14 at 15:05
3

set reports not only environment variables but also shell variables and, on some shells, functions. Use env | wc -c to get the size of the environment.

The environment size is indirectly limited by the maximum size of the arguments to execve (which consist of the environment plus the command line arguments). You can get the applicable value of this limit with getconf ARG_MAX.

3849 bytes doesn't seem particularly high. POSIX says that the limit can be as low as 4096 bytes, but with 512MB of RAM, your system isn't low-end and is likely set up to allow a lot more. In any case, if that limit was hit, then execve would fail; this has nothing to do with shared memory queues.

  • Great advice. getconf ARG_MAX reveals 61440, way more than the 9364 bytes env | wc shows me using. I agree it shouldn't be related to shared memory queues nor file descriptors being weird. (Also, umask was not being applied to new files properly... It was weird.). It was just a suspicion that something was going very strange with the OS. Of course, it was a wrong suspicion. In fact, the QNX documentation says I'll get E2BIG when I try to execve something if the environment is too big. See qnx.com/developers/docs/6.4.1/neutrino/lib_ref/e/execve.html. Thanks again. – kmort Apr 12 '14 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.