0

I have a customized Linux Kernel which prints to dmesg and syslog (using printk) some additional information about each received MP-TCP packet in the system. Things work perfectly, however I am generating a huge amount of data (using iperf tool) around 2 Gbps of data which correspond to extremely high number of packets. Because of the printk, all my CPUs are 100% loaded in my system (I have 4 CPUs). I am worried that this might affect the packet processing in my system and causes additional problems and latyency. So my question is there any alternative method for storing my data in the Liunx Kernel without increasing the load on my CPUs?

To give an example of the size of the data, for 60 seconds of an iperf session, the size of the `/var/log/syslog is around 1.2 Gbytes.

Let me add one more thing, I do not need to process the reported data by the Linux Kernel in an online fashion. I will process them in offline mode.

  • 1
    printk prints to the kernel ring buffer, which is usually highly limited. See: unix.stackexchange.com/questions/332422/… If your system is additionally configured to log to /var/log/syslog, wich is usually an ordinary file, probably the file open and write operations will cause a huge I/O load on your system. – Richard Neumann Sep 5 '17 at 13:11
  • Yes, I am already aware about the limitation of kernel ring buffer. I have already increased it to 16 MBytes which is the maximum value but it is still small. This is why I am using /var/log/syslog to check my data. But as you said and I reported in my question it causes huge I/O. So do you have any alternative solution? – IoT Sep 5 '17 at 13:16
  • A faster solution might be storing it in RAM i.e. a sufficiently large tmpfs. – Richard Neumann Sep 5 '17 at 13:18
  • Any good example? – IoT Sep 5 '17 at 13:19
  • 2
    Just mount a tmpfs in some directory where you'd like to log your stuff. You can even mount it to /var/log. E.g. mount -t tmpfs -o size=4G tmpfs /var/log But be reminded that you'll need sufficient RAM and that using SWAP might slow down your system again. – Richard Neumann Sep 5 '17 at 13:25
2

You should probably look into ftrace, the built-in Linux kernel tracing tool. For example, this slightly out-of-date LWN article says

Ftrace introduces a new form of printk() called trace_printk(). It can be used just like printk(), and can also be used in any context (interrupt code, NMI code, and scheduler code). What is nice about trace_printk() is that it does not output to the console. Instead it writes to the Ftrace ring buffer and can be read via the trace file. Writing into the ring buffer with trace_printk() only takes around a tenth of a microsecond or so.

For example you can add something like this to the kernel or module:

trace_printk("read foo %d out of bar %p\n", bar->foo, bar);

The kernel doc ftrace.txt has the details. In directory /sys/kernel/debug/tracing/ you can set the trace ring buffer size in pseudo-file buffer_size_kb.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.