Questions tagged [buffer]

This tag is for questions about kernel buffer caches, including pipe buffers. These are used to store the recently accessed files and/or frequently accessed files.

Filter by
Sorted by
Tagged with
3
votes
1answer
249 views

How do I ensure all data to an SSH pipe is sent, despite lengthy disconnects?

I'm on a laptop with intermittent internet connectivity. (i.e. I sometimes don't have network for a week.) I want the output of a process on my laptop to end up on my server. It all needs to get ...
5
votes
3answers
2k views

How to forward between processes with named pipes?

The /tmp/in, /tmp/out and /tmp/err are named pipes, already created and opened by some process (for reading, writing and writing, respectively). I would like to create a new process that pipes its ...
1
vote
1answer
52 views

Why combining two unbuffers give this error?

I am reading this answer here and run test code unbuffer xxd -ps /usr/bin/telnet | unbuffer -p less but get Missing filename ("less --help" for help). Why combining two unbuffers give this error?
5
votes
3answers
673 views

Why does this one-liner fail to produce output? [duplicate]

The following produces nothing in bash: while true ; do upower -d ; sleep 1 ; done | grep percentage | uniq I've discovered that it doesn't matter what the last- or even the second to last- program ...
0
votes
0answers
117 views

Backup server with tar, encrypt and directly transfer the result, AFTER BUFFERING IT

So I've been investigative how to make full backups with tar, and I found that tar is a good tool if the backup is to be made once, and if the server has enough disk space to handle the output file. ...
4
votes
1answer
5k views

Formatting grep output with awk. Simple case and background case

I'm struggling with this problem. I have this line: mplayer *.* 2>/dev/null | grep Playing which just launchs mplayer and only the line matching Playing filename is printed on screen. This works ...
1
vote
1answer
175 views

Function tracing per thread

I have a complex multi threaded application running on a Cent OS 5.8 the application is coded using C and C++ I am searching for a FUNCTION TRACING tool which can help me do the following. Trace all ...
1
vote
0answers
675 views

How can I find which process is eating my memory? [duplicate]

My Linux system sometimes reaches 90% memory usage. We are running some Java processes, and top shows overall usage as 45%. If I stop the running processes, usage reduces to 5-10%. I want to list ...
4
votes
1answer
154 views

Disable buffering in a VM configuration

I am executing some benchmark using a DBMS and virtual machines. And I am using this command free && sync && echo 3 > /proc/sys/vm/drop_caches && free on each Guest, and ...
22
votes
2answers
3k views

Pipes, how do data flow in a pipeline?

I don't understand how the data flows in the pipeline and hope someone could clarify what is going on there. I thought a pipeline of commands processes files (text, arrays of strings) in line by line ...
8
votes
3answers
2k views

Piping commands, modify stdin write to stdout

echo Hello World | nano - | less I want to modify Hello -> Goodbye while in the the text editor. It does not have to be with nano. I am stuck with nano reading stdin but requiring me to write to a ...
8
votes
2answers
5k views

Buffering (named) pipe in GNU OS

In the GNU OS a process can only write data to a pipe if another process reads the same data (from the same pipe) at the same time. Is there something like a pipe which lets the 1st process write and ...
1
vote
1answer
3k views

How to enable a non-root user to empty the linux buffer cache

I'm required to empty the linux buffer cache in a python script, that runs on a Debian wheezy VM. As root I run sync; echo 3 | sudo tee /proc/sys/vm/drop_caches, but the script is run by a user, ...
4
votes
1answer
512 views

Meaning of fsync() in sshfs+LUKS setup

Background: I'm investigating methods for encrypted storage on untrusted machines. My current setup uses sshfs to access a LUKS-encrypted image on the remote machine, which is decrypted locally and ...
0
votes
1answer
88 views

:bnext not cycling through all buffers in vim

For some reason :bnext is not cycling though all the buffers shown by :buffers I have 5 buffers open but only 3 buffers are visited when I used :bnext repeatedly. How can I cycle through all the ...
1
vote
1answer
93 views

Use saved input for scripts

I have a few scripts that I constantly enter in input as below: [support@ccfcore-3 exporttool]$ ./sendExamToRepo.sh Enter the Study Instance UID: 2.25.98472613238667427310842065102689842379 The ...
2
votes
2answers
315 views

Specifying height for vertical splits in vim

Usually I am editing two files, A and B, split horizontally. I set it up, so that one buffer would only occupy the first 2 lines at the top of the screen, and the rest is for the other buffer. ...
7
votes
2answers
5k views

What should be the buffer size for the sort command?

I have a machine with 2 TB of RAM and I am running a sort command on a file of size 150G where I have specified the buffer-size as 1000G, after doing my bit of reasearch on google, I got this piece of ...
3
votes
1answer
276 views

Artificially prime buffer cache?

Is it beneficial to artificially prime the buffer cache when dealing with larger files? Here's a scenario: A large file needs to be processed, line by line. Conceptually, it is easy to parallelize ...
0
votes
1answer
541 views

Preventing a new instance of a program to open when calling it

My problem is with FBI (frame buffer image viewer) for Linux, but my question and any answers are perhaps applicable to more than only that program. I don't know. Anyway… While things are loading on ...
5
votes
1answer
2k views

Difference between “blockdev --flushbufs” and “sync” on Linux?

Is there a difference in practice between running blockdev --flushbufs and sync(1) on Linux? (apart from blockdev flushing for a specific device, and sync being system-wide). sync(1) manpage says it ...
2
votes
1answer
401 views

Why don't I see a file being copied right away with ls?

Is it due to the buffering in memory? I did cp /network/large_file . & and I kept doing ls -lh to monitor progress, but the file didn't appear until it was about 400MB. Linux 2.6 kernel, Bash 4, ...
1
vote
1answer
5k views

very high buffer memory usage [closed]

Is this normal - I've never seen buffers so high? $ cat /proc/meminfo MemTotal: 16332808 kB MemFree: 1112264 kB Buffers: 10630124 kB Cached: 699400 kB SwapCached: 0 kB Active: 3262924 kB Inactive: ...
3
votes
0answers
141 views

xdelta3 fails to decode on block device

I'm trying to use xdelta3 to bring a remote block device into sync with a local one. I'm able to easily generate a xdelta3 patch file using the command: xdelta3 -e -B 33554432 -v -9 -I 0 -s /dev/...
2
votes
1answer
3k views

Daemontools multilog loses log line time information. How to fix it?

I am using daemontools to monitor a process and its output log. I am using multilog to write the logs to disk. The run script for the log is: #!/bin/bash PATH=/usr/local/bin:/usr/bin:/bin cd /usr/...
6
votes
1answer
1k views

Ksh loses data after piping 16K bytes

I recently found that ksh may lose some data after printing more than 16K bytes to the stdout if it is blocked for a couple of seconds. This test.sh script prints out 257*64 (16448) bytes: #!/usr/...
6
votes
1answer
3k views

Switching between split buffers in vim

recently starting trying to use VIM as my full-time text editor since I spent a lot of time SSH'ed anyways. Recently installed NERDTree so I can quickly swap between files in a project. Easy question ...
6
votes
2answers
3k views

linux stdbuf - line-buffered stdin option does not exist

The man page for the stdbuf command states that line-buffered mode is invalid as a standard input buffering option. What is the reason for this? tail -f access.log | stdbuf -iL cut -d' ' -f1 | uniq ...
5
votes
2answers
9k views

Alternate FIFO device for Linux with a (way) bigger buffer while still having filesystem semantics?

I am developing something that consists of a program I made generating output, and a program I did not make (and would very much like not to have to alter it) consuming it. I am using a named FIFO ...
1
vote
5answers
2k views

Display output to console while grep is used

In vim I use the following command to compile a tex file: pdflatex\ \-file\-line\-error\ \-shell\-escape\ \-interaction=nonstopmode\ $*\\\|\ grep\ \-P\ ':\\d{1,5}:\ ' this works in terms of getting ...
3
votes
1answer
355 views

Ex and Vi: buffers and regexes

I want to put regex matches to buffer but cannot get even simpler example y :.put myNewBuffer return msg [line] [put] [buffer] working ie how to put something to buffer? So to regular expressions ...