Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Base on this I'm running the command

< /dev/urandom hexdump -v -e '/1 "%u\n"' |
awk '{ split("0,2,4,5,7,9,11,12",a,",");
       for (i = 0; i < 1; i+= 0.0001)
         printf("%08X\n", 100*sin(1382*exp((a[$1 % 8]/12)*log(2))*i)) }' |
xxd -r -p |
sox -traw -r44100 -b16 -e unsigned-integer - -tcoreaudio

I notice that the memory used by awk continually grows while this command is running, for example consuming over 500MB of memory by the time 75MB of raw audio data has been played. All of the other commands in the pipeline maintain a constant amount of memory.

What is awk using this memory for and is there an alternative that does the intended stream processing using only a constant amount of memory?


in case the awk version matters:

⑆ awk --version
awk version 20070501

Here's the command I tested based on Thomas Dickey's answer:

< /dev/urandom hexdump -v -e '/1 "%u\n"' |
awk 'BEGIN { split("0,2,4,5,7,9,11,12",a,",") }
           { for (i = 0; i < 1; i+= 0.0001)
               printf("%08X\n", 100*sin(1382*exp((a[$1 % 8]/12)*log(2))*i)) }' |
xxd -r -p |
sox -traw -r44100 -b16 -e unsigned-integer - -tcoreaudio
share|improve this question
    
I also see a memory leak on my BSD-Darwin (Mac) system. – Otheus Mar 5 at 22:31
2  
+1 just for posting a music generator – Celada Mar 5 at 22:53
up vote 11 down vote accepted

This statement is odd:

split("0,2,4,5,7,9,11,12",a,",");

It repetitively splits a constant string to create an array a. If you move that into a BEGIN section, the program should work the same — without allocating a new copy of the a array for each input-record.

Addressing comments: the for-loop and expression do not allocate memory in a simple manner. A quick comparison of mawk, gawk and awk shows that there is no problem with the first two, but /usr/bin/awk on OSX does leak rapidly. If Apple had a bug-reporting system, that would be the place to go.

share|improve this answer
1  
I did as you suggested on my Mac (I'm not the OP). I still see a memory leak with awk. – Otheus Mar 5 at 22:33
    
Somehow, simply referencing the a hash uses memory. – Otheus Mar 5 at 22:38
    
Same here; I still see the memory growth. I also did a rough comparison and the memory usage seems to be growing at the same rate with this change. – bames53 Mar 5 at 22:38
    
Even this will cause a memory leak: awk 'BEGIN { split("0,2,4,5,7,9,11,12",a,","); } { for (i = 0; i < 1; i+= 0.0001) a[1]; }' – Otheus Mar 5 at 22:43
    
You could switch to mawk or gawk. Apple's base system includes some real antiques. – Thomas Dickey Mar 5 at 22:44

Here's a perl equivalent that doesn't leak:

perl -lne 'BEGIN { @a=(0,2,4,5,7,9,11,12);}
   for ($i = 0; $i < 1; $i+= 0.0001) {
     printf("%08X\n", 100*sin(1382*exp($a[$F[0] % 8]/12)*log(2))*$i) }'

It's almost identical. $1 gets replaced by $F[0] and i is replaced with $i. The hash a is replaced with an actual array, @a.

You would be wise to generate some input and compare the output and note differences between the two. There are often nuances as to how interpretive languages deal with floating point.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.