Questions tagged [buffer]

This tag is for questions about kernel buffer caches, including pipe buffers. These are used to store the recently accessed files and/or frequently accessed files.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

How to trigger two commands when pressing a key bind

So I wanted to pass the text stored in my buffer through a socket to other machine, for that I manage to get a socket done, I just need to fire up the following command when in copy-mode-vi bind -T ...
user avatar
0 votes
0 answers
18 views

Why are packets being dropped (kfree_skb)?

I am currently sending ~9mpps, each packet is 72bytes large, and is a UDP DNS request. The packets are being sent to a Dell server I have using an intel i40e interface. The server is using all of its ...
user avatar
  • 294
0 votes
2 answers
71 views

Explanation about free, available and buff/cache and how to find what process is buff/cache saved for in a server in the command "free -mh "

I see explanation for free -mh as: free displays the total amount of free and used physical and swap memory in the system, as well as the buffers and caches used by the kernel. The information is ...
user avatar
  • 1
1 vote
2 answers
69 views

why do we need to pass buffers to system calls in order to have information returned? Why can't system calls allocate the buffer internally?

I'm reading TLPI, and in section 4.4: System calls don’t allocate memory for buffers that are used to return information to the caller. Instead, we must pass a pointer to a previously allocated ...
user avatar
0 votes
0 answers
65 views

Clear socket buffer of CAN adapter

I have a USB/CAN adapter, from lsusb: OpenMoko, Inc. Geschwister Schneider CAN adapter it provides a socket interface, from ip addr: 5: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 16 qdisc pfifo_fast ...
user avatar
  • 567
0 votes
0 answers
98 views

How to write bytes to serial port's input buffer

I'm working on creating an emulator for a Captron button that is connected to a serial port /dev/ttyS0. I'm trying to mimic the on-press action by writing a byte array to the serial port by printf /...
user avatar
0 votes
0 answers
26 views

linux notification of disk write

In linux, during normal (buffered) writes, the data first goes to page cache and then goes to disk at a later point in time. so for example, when write() returns, we know that data is in page_cache. ...
user avatar
  • 101
0 votes
1 answer
47 views

When is it beneficial to add -u to tr?

One of the arguments to the tr command is: -u Guarantee that any output is unbuffered. Under what circumstances is it a good idea to not buffer the output? e.g. should I use -u if I expect each ...
user avatar
0 votes
0 answers
63 views

Are direct writes to block devices buffered? How do I ensure it's written to disk before removing it?

Say I do something like dd if=file of=/dev/sdc (simplified for illustration), is this buffered? Can I sync it like I would any other filesystem write, or do I need to do something else to make sure ...
user avatar
  • 293
5 votes
1 answer
292 views

Why is mawk's output (STDOUT) buffered even though it is the terminal?

I am aware that STDOUT is usually buffered by commands like mawk (but not gawk), grep, sed, and so on, unless used with the appropriate options (i.e. mawk --Winteractive, or grep --line-buffered, or ...
user avatar
  • 1,030
0 votes
1 answer
3k views

Buffer I/O error on dev sda6, logical block 44, async page read

I don't know why this happens... I was just doing my work (development) and suddenly one of my drive start lagging then I just unmount it by right click on it. After it is still lagging, so I restart ...
user avatar
1 vote
1 answer
319 views

configure vim to use both tmux and system clipboard

I'm fairly new to vim (I'm using nvim) so forgive my lack of knowledge, many questions are similar to mine, but not quite the same. I would like this: vim by default should cut/copy to a buffer, not ...
user avatar
  • 61
0 votes
1 answer
189 views

How does buffering for TCP packets work?

If I write a few times in a quick succession on a socket (with the POSIX function write), usually all the data I wrote gets sent in a single TCP packets. Unless I write too much or unless I wait too ...
user avatar
1 vote
0 answers
40 views

linux handling of files

I want to perform updates on a file via multiple processes in parallel. These processes all open this file for write in parallel. Abbreviations used: f : file, p[i] : process i, b[i] : buffer for FD ...
user avatar
  • 150
1 vote
1 answer
143 views

Pipe from xwd to ImageMagick's convert produces cropped image

I'm trying to take a screenshot of a remote host (using xwd) and convert it to png (using convert). I need it to be an unattended oneliner, so this is what I'm doing: sshpass -p THE_PASSWORD ssh user@...
user avatar
1 vote
0 answers
354 views

ping: sendmsg: No buffer space available

I am running a ping from one of the terminals: ping www.google.com This runs without any issues indefinitely. Now on another terminal, I run: for i in {1..5}; do speedtest; done Now after sometime on ...
user avatar
  • 201
0 votes
1 answer
230 views

Do sync and writing to /proc/sys/vm/drop_caches really empty all the disk cache/buffers?

When you want to empty disk caches/buffers, you usually issue the following command: # sync && echo 3 > /proc/sys/vm/drop_caches But after typing that command, free doesn't really report 0 ...
user avatar
3 votes
2 answers
280 views

How do I feed data to a pipe until it's full, no more no less?

I want to use pipes on Linux as a synchronization primitive between a master process and a slave process. The classic way is to create two pipes, but I believe there's a way to use a single fd instead....
user avatar
  • 41
1 vote
0 answers
81 views

Buffer Memory Drops to Zero

We have been having problems with a system that becomes inaccessible. Characteristics are 1) sudden spike in blocked and run queues and 2) buffer memory drops to zero. The symptoms are 1) existing ...
user avatar
  • 11
3 votes
1 answer
429 views

broken pipe when tar extracted from stdin buffer

I am restoring files from a tar archive on an LTO-7 tape to a locally mounted network share. If I directly restore to the share it runs very slowly (90 MB/s). When I use an additional buffer I get the ...
user avatar
  • 205
0 votes
1 answer
77 views

Bright Color Escape Sequences not saved in Tmux capture-pane?

I'm trying to use tmux to create a pty for me that I can read the contents of, including color. I'm using tmux capture-pane -e -t paneName and tmux save-buffer tmuxbuffer.txt and I get normal colors, ...
user avatar
  • 63
0 votes
0 answers
277 views

Effect of Buffer Size in File I/O in Unix

I am trying to understand the inner workings of Unix based OSs. I was reading on buffered I/O, and how the buffer size affects the number of system calls made, which in turn affects the total time ...
user avatar
4 votes
2 answers
205 views

Buffering stdout separately from stderr

I am running a utility, which emits the following: Progress to its Standard Error Data/Yield/Output to its Standard Output I did not build the utility, nor can I easily modify it. I wish to do the ...
user avatar
2 votes
1 answer
135 views

Does it have 3 levels of buffering, or 2 levels of buffering, when using stdio library?

From The Linux Programming Interface: (I've read the related sections. I pasted this sum-up diagram just for illustration.) I know that for direct IO which uses read and write system calls. There is ...
user avatar
  • 961
1 vote
1 answer
207 views

File order on FAT/FAT32/VFAT file systems

I have several audio devices (car radio, portable radio, MP3 player) that take SD cards and USB sticks with a FAT file system on it. Because these devices have limited intelligence they do not sort ...
user avatar
2 votes
2 answers
501 views

How to send output of `pv` to python script, and output progress one line at a time?

I use tar to quickly copy thousands of small files which I pipe to pv to show the progress and speed of the copying. $ sudo tar c --files-from /tmp/camfilenames | pv --width 40 | sudo tar x -C /...
user avatar
0 votes
1 answer
444 views

Disable buffering for file descriptors in bash

I may be misunderstanding what's going on here, but I feel it's got to do with pipe buffering. I have a script which uses several file descriptors (# 3 and above) for various logging levels. Depending ...
user avatar
-1 votes
1 answer
108 views

Which hardware is used by buffer

I know that the cache is used to store data so that future requests for that data can be served faster. Normally we have three level cache: L1, L2 and L3 in the current computer architecture. We can ...
user avatar
  • 2,819
0 votes
1 answer
454 views

Linux FIFO buffering depends on invocation order of readers and writers

I'm trying to write a simple python script that reads from a fifo and then writes to another fifo. I have created two FIFOs using the following command: $ mkfifo input $ mkfifo output I invoke the ...
user avatar
  • 150
9 votes
1 answer
3k views

Do I need to take action regarding my Microarchitectural Data Sampling (MDS) status?

My dmesg output contains the following line: [ 0.265021] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more ...
user avatar
  • 323
1 vote
1 answer
489 views

avoid bash buffering problem in log processing

I have bash buffering a problem similar to what can be found here: Turn off buffering in pipe The socat solution in the above question is quite interesting as I have access to this command in my ...
user avatar
14 votes
3 answers
2k views

What happens when a file that is 100% paged in to the page cache gets modified by another process

I know that when a page cache page is modified, it is marked dirty and requires a writeback, but what happens when: Scenario: The file /apps/EXE, which is an executable file, is paged in to the page ...
user avatar
0 votes
0 answers
145 views

background process with "&" and problem with buffering

I've written this script in the ~/.bashrc file and I've sourced the file, but it's not working properly. It's purpose is to open the link provided by the jupyter notebook in the Chrome browser in a ...
user avatar
  • 491
1 vote
0 answers
30 views

Why doesn't sync guarantee that buffers are flushed?

As far as I can see, the only use of sync is to ensure that there's no incomplete information, which is supposed to be safely written on disk but may actually be still only in a volatile buffer. ...
user avatar
0 votes
0 answers
69 views

Displaying a changing buffer in realtime in linux

Sometimes when I am searching for a song on my machine, I use the following: locate *artist*.mp3 | sort | less The thing about this is that the sort will buffer until all files have been located, ...
user avatar
  • 203
4 votes
2 answers
1k views

Flush the pipe/printf buffers externally for already running process with known PID

I am writing a data logging app, all programs are started like: ./program > out.bin The data collector periodically pools the stdout output files and reads the data. The issue is that the IO ...
user avatar
  • 179
2 votes
0 answers
92 views

Unbuffer converting all characters to bell?

When I pipe a large-ish output through unbuffer, all I end up getting is an endless stream of the bell character (u0007). See here, a 1631-character string of arbitrary content piped through unbuffer ...
user avatar
1 vote
1 answer
951 views

Disable buff/cache linux for fixing ubuntu hangs on full ram

I have 8 gb of ram, ssd and very greedy for ram Android studio/gradle. and sometimes when gradle builds project(eats ram) it hangs all ubuntu(and xubuntu). ui becomes so laggish - mouse cursor moves 1 ...
user avatar
  • 11
3 votes
1 answer
2k views

What controls the buffering of stdout stderr?

What part of the system sets up the buffering of the three standard streams when a program is started? Is this part of linux, or glibc, or maybe bash? Does POSIX define the behaviour, or is it part ...
user avatar
  • 438
1 vote
0 answers
3k views

UDP send buffer error in netstat

I we are running voice application which stream UDP (RTP) packets for voice, I am investigation some strange issue where my ifconfig ens4 | grep error counter going up specifically overrun If you see ...
user avatar
  • 1,530
3 votes
1 answer
594 views

Using multiple copy buffers in Bash

Ctrlu cuts text from current location to the beginning of line. Now, I can paste this with Ctrly whenever I want until the next time I hit Ctrlu to copy another line in the buffer. Now the problem I ...
user avatar
1 vote
0 answers
3k views

flush tty device buffer from terminal

I have a serial device on /dev/ttyAMA0. I have program written in C that communicates with this device. But apart from the C program I sometimes need to input some configuration data from terminal, ...
user avatar
  • 155
0 votes
0 answers
101 views

Sizing of Samba Linux Server

Do you know any performance of sizing recommendations about Samba 4.7.1 ? For few informations : 2vCPUs / 1 Core 8GB of RAM 140 simultaneous Windows users / 250 at most / 20 NFS volumes mounted via ...
user avatar
5 votes
0 answers
330 views

Tailing file using less +F/shift-F, turn off line buffering?

When using tail -f, it prints each character as it is written to the file, but less +F won't print until it sees a linefeed. And if the data being written to the file does not contain linefeeds, must ...
user avatar
  • 93
2 votes
0 answers
141 views

fq_codel not working on saturated link?

I'm setting up a test to saturate a link between two machines with a gig router in between them, then run fq_codel on Debian Buster and see if my ping times can improve (what I think fq_codel is ...
user avatar
3 votes
1 answer
2k views

Why do terminal emulators still have screen flicker?

Why are there still visual artifacts when terminal emulators draw text-based applications? This on recent computers that render 3D games and GUI windows including anti-aliased vector fonts with no ...
user avatar
  • 749
1 vote
2 answers
847 views

why /proc/net/udp is showing wrong rx_queue size in CentOS server?

I have a java server application. while(true) { serverSocket.receive(receivePacket); process(receivePacket); serverSocket.send(sendPacket); try { ...
user avatar
  • 111
3 votes
1 answer
237 views

Page cache behavior guarantees and log files

Are there any guarantees as to when and in what order dirty pages are written out to disk? In the specific case of log structured files (files that are only ever appended to), is it possible for dirty ...
user avatar
  • 151
1 vote
0 answers
127 views

How to get the network buffer usage of some process

I have an Ubuntu server, which contains some user-processes. It always crashes after starting-up several hours. "crashes" means that I cannot neither ssh to it nor ping it anymore. I'm doubting that ...
user avatar
  • 2,819
1 vote
1 answer
436 views

How to pipe to less unbuffered?

I'm trying to display progress, and make it usable under less: while true; do echo -n .; sleep 1; done | less Is there a way to make it work?
user avatar
  • 2,913