3

So I ran the following command and I accidentally saw the monitor and I wondered what is exactly happening. can someone explain to me what is exactly happening here?

echo "`seq 100000000 -1 1`" > file

system monitor screen-shot while running the command

Since it happens many times that the images don't load for me I try to describe what I see in it. The memory usage grows linearly and the CPU1 usage is fixed at 100% then at the times t1(barely noticeable) t2 t3 t4 t5 the memory usage suddenly grows much faster for about half a second and CPU1 usage decreases and CPU2 usage increases all at the same time then memory growth rate comes back to it's linear form with the exact same slope and CPU1 usage grows towards 100% again and CPU2 usage drops to it's prior state in another half a second. The interesting points are that:

t(i)-t(i-1) = 2*(t(i+1)-t(i))     for every i
The amount of decrease in CPU1 usage doubles for each i
The time interval in which these things take seems to be the same for all i
The amount of sudden growth in memory usage also doubles for all i
The sum of the loads of CPU1 and CPU2 seem to be constant in time.

So I'm asking multiple questions in one question but I think this is the best way to do it. Rather than posting the same picture 5 times with almost the same explanations.

Can someone please explain to me Why exactly are these observations taking place?

Memory info:

$ sudo lshw -class memory
  *-firmware              
       description: BIOS
       vendor: LENOVO
       physical id: 0
       version: 9ECN31WW(V1.14)
       date: 08/18/2014
       size: 128KiB
       capacity: 6592KiB
       capabilities: pci upgrade shadowing cdboot bootselect edd int13floppynec int13floppytoshiba int13floppy360 int13floppy1200 int13floppy720 int13floppy2880 int9keyboard int10video acpi usb biosbootspecification uefi
  *-cache:0
       description: L1 cache
       physical id: b
       slot: L1 Cache
       size: 32KiB
       capacity: 32KiB
       capabilities: synchronous internal write-back instruction
       configuration: level=1
  *-cache:1
       description: L2 cache
       physical id: c
       slot: L2 Cache
       size: 256KiB
       capacity: 256KiB
       capabilities: synchronous internal write-back unified
       configuration: level=2
  *-cache:2
       description: L3 cache
       physical id: d
       slot: L3 Cache
       size: 6MiB
       capacity: 6MiB
       capabilities: synchronous internal write-back unified
       configuration: level=3
  *-cache
       description: L1 cache
       physical id: a
       slot: L1 Cache
       size: 32KiB
       capacity: 32KiB
       capabilities: synchronous internal write-back data
       configuration: level=1
  *-memory
       description: System Memory
       physical id: 2a
       slot: System board or motherboard
       size: 8GiB
     *-bank:0
          description: SODIMM DDR3 Synchronous 1600 MHz (0.6 ns)
          product: HMT451S6BFR8A-PB
          vendor: Unknown
          physical id: 0
          serial: 1E52E0EA
          slot: DIMM0
          size: 4GiB
          width: 64 bits
          clock: 1600MHz (0.6ns)
     *-bank:1
          description: DIMM [empty]
          product: Empty
          vendor: Empty
          physical id: 1
          serial: Empty
          slot: DIMM1
     *-bank:2
          description: SODIMM DDR3 Synchronous 1600 MHz (0.6 ns)
          product: HMT451S6BFR8A-PB
          vendor: Unknown
          physical id: 2
          serial: 1E82E0B8
          slot: DIMM2
          size: 4GiB
          width: 64 bits
          clock: 1600MHz (0.6ns)
     *-bank:3
          description: DIMM [empty]
          product: Empty
          vendor: Empty
          physical id: 3
          serial: Empty
          slot: DIMM3

CPU info:

$ sudo lshw -class processor
  *-cpu                   
       description: CPU
       product: Intel(R) Core(TM) i7-4700HQ CPU @ 2.40GHz
       vendor: Intel Corp.
       physical id: 4
       bus info: cpu@0
       version: Intel(R) Core(TM) i7-4700HQ CPU @ 2.40GHz
       serial: To Be Filled By O.E.M.
       slot: U3E1
       size: 3285MHz
       capacity: 3400MHz
       width: 64 bits
       clock: 100MHz
       capabilities: x86-64 fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts cpufreq
       configuration: cores=4 enabledcores=4 threads=8

Kernel and OS info

$ uname -a
Linux <Machine name> 4.10.0-42-generic #46~16.04.1-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux

Shell info:

$ $SHELL --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)

Swap info:

$ lsblk | grep SWAP
├─sda2   8:2    0   935M  0 part [SWAP]
4

You're running a command that generates ~848MB of output, trying to interpolate that into a shell command line argument for echo and then redirecting that to a file. (BTW, modern shells on modern systems allow command lines up to a few megabytes long, but none will allow nearly a gigabyte).

Is there any reason to be surprised that this uses an enormous amount of RAM and CPU power?

Even running seq 100000000 -1 1 > file is going to take a fair amount of time, but nowhere near as much memory usage because without the command substitution, the output doesn't need to be kept in memory.

e.g. on my AMD Phenom II 1090T, it takes 1 minute and 2 seconds to run the seq command and redirect its output to a file:

$ time seq 100000000 -1 1 > /tmp/seq.test

real 1m2.136s   user 1m0.215s   sys 0m1.405s
$ ls -lh /tmp/seq.test 
-rw-r--r-- 1 cas cas 848M Jan  8 12:06 /tmp/seq.test
  • I think you have completely got my question wrong. I didn't wonder about 100% CPU usage or the memory usage at all. my question was about the trends in them. – yukashima huksay Jan 8 '18 at 11:43
  • 1
    as the mem & swap part of the graph is unlabelled, I'll assume the red line is swap activity. The CPU spikes and drops seem to correspond with swap activity...and it looks like the kernel is switching the process from one core to another at ~ the same time. The only way to be sure what is happening is to profile the kernel while it's happening. Posting a graph without relevant info about your system (RAM, CPU, kernel & sh versions, etc) to a q&a site is only going to get you educated guesses at best. as mentioned, this is hardly surprising with a process that rapidly uses lots of RAM. – cas Jan 8 '18 at 12:58
  • Please let me know if you need any more information. About my system. – yukashima huksay Jan 8 '18 at 13:36
  • No it's not swap it's memory: As I've mentioned three times: The memory usage grows linearly, the memory usage suddenly grows much faster for about half a second, then memory growth rate comes back to it's linear form – yukashima huksay Jan 8 '18 at 13:39
  • @yukashimahuksay It is likely that the shell allocate memory in increasingly large chunks at the times when you see the rapid increases memory/swap and that the kernel takes over for a short while while doing so (hence the dips in the CPU graph). My guess is that the shell does something like increasing the buffer by a fixed factor every time it needs to be enlarged. – Kusalananda Jan 10 '18 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.