Questions tagged [pv]

Pipe Viewer, or pv, is a Unix tool for monitoring pipelines and the progress of data through them.

Filter by
Sorted by
Tagged with
0
votes
0answers
30 views

Clear a tmp directory files only and watch progress using find and pv

I have a giant tmp directory and I don't want to delete the subdirectories. Instead, I just want to find the files in the directories and delete them. My thinking is to find the top most directories,...
1
vote
0answers
39 views

Invoke pv to monitor files when moving directories

I'm trying to move files (movies) between directories. I have a script which renames my movies and then a separate script which I use to open a new terminal and monitor the movies being moved from one ...
0
votes
2answers
27 views

Continue copying file

I was copying a very big file and I accidentally stopped it. Can I resume copying data without need to delete copy and copy data again? Command I used: pv original.data > copy.data
1
vote
1answer
49 views

How can I estimate the whole time of a write process including sync?

Progress and estimated time to write without and with sync I have found no tool (or straightforward method) that will include flushing the buffers when showing the progress and estimating the ...
2
votes
2answers
96 views

How to send output of `pv` to python script, and output progress one line at a time?

I use tar to quickly copy thousands of small files which I pipe to pv to show the progress and speed of the copying. $ sudo tar c --files-from /tmp/camfilenames | pv --width 40 | sudo tar x -C /...
0
votes
1answer
375 views

Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?

I'm excited to learn that xz now supports multithreading: xz --threads=0 But now I want to utilise this as much as possible. For example, to recompress gzips as xz: gzip -d -k -c myfile.gz | pv | ...
1
vote
1answer
57 views

Why should I set the source device with < > when using pv?

I have a question out of curiosity ;-) I find that I could accomplish cloning a disk with the command pv instead of dd too. For that I used what I found in the internet pv < /dev/sda1 > /dev/...
1
vote
3answers
89 views

Writing to 2 SD cards at the same time

I would like to write an image at two SD cards at the same time. I envision at least two concurrent writing scenarios: Is it possible to write the image with a tee or similar bifurcating mechanism? ...
0
votes
1answer
25 views

Pipe viewer logging

Using the pipe viewer (pv) utility, I can visually monitor the data flow going through a pipe. I would however like to be able to log the data flow in a file instead of dynamically viewing it in a ...
0
votes
1answer
173 views

Total Progress over entire bash shell script

EDIT FOR CLARITY : say I have the following script (let's posit pv and curl are already installed) : (that currently runs under ubuntu but that I plan to make POSIX-compliant so that it may run on ...
0
votes
2answers
446 views

df -h results different than VGdisplay / LVdisplay

Im trying to remove the added disk to the VM, as the disk space is not needed anymore. However, it looks to be that the VM has in the VG almost all the data used what is available there. However, i'...
6
votes
5answers
860 views

Using pv with md5sum

I used md5sum with pv to check 4 GiB of files that are in the same directory: md5sum dir/* | pv -s 4g | sort The command completes successfully in about 28 seconds, but pv's output is all wrong. ...
2
votes
1answer
238 views

Use pv with find -exec

I use the following command to verify ~700 GiB of backed-up files: $ find -type f -exec md5sum {} + | sort > ~/checksums This takes many hours, so I would like to integrate pv into the command to ...
0
votes
2answers
564 views

How can I run multiple pv commands in parallel?

I want to run a sequence of command pipelines with pv on each one. Here's an example: for p in 1 2 3 do cat /dev/zero | pv -N $p | dd of=/dev/null & done The actual commands in the pipe don't ...
6
votes
3answers
1k views

With the command pv it is possible to clone a drive, how do I mount it? [duplicate]

I was able to backup a drive using the following command. pv -EE /dev/sda > disk-image.img This is all well and good, but now I have no way of seeing the files unless I use this command pv disk-...
1
vote
1answer
273 views

watch netcat transfer dump from android to pc

how can i monitor netcat transferring from android to my linux machine i used this command on android device ( sender ) to make a full dump for my device : dd if=/dev/block/mmcblk0 | busybox nc -...
2
votes
3answers
248 views

Pipe-Viewer problem with changing Rate-Limit

I'm using pv for sending files via ssh. I can change "active pv" the limit at under 100M without any problem. When i set active pv process to 100M or 1G or higher I cant change rate anymore... BUT! ...
0
votes
1answer
280 views

How to get PID when start bash script

I'm running bash files for zfs send jobs and this my bash file example: zfs send -Rc tank/test@snap | pv -fs datasize -F "%p***%t***%e***%r***%b" | mbuffer -q -s 128k -m 1G -O ip:port When I start ...
3
votes
1answer
918 views

What is the best way pause “zfs send via PV” and resume

I'm using PV for my ZFS send-recv replication. I use ZFS resume token too but i want to pause and resume like sigstop, sigcontinue. Because using resume token means sending same thing again. So how ...
3
votes
2answers
994 views

How to use pv to show progress of openssl encryption / decryption?

I need to encrypt and be able to decrypt files with openssl, currently I do this simply with: openssl enc -aes-256-cbc -salt -in "$input_filename" -out "$output_filename" and the decryption with: ...
1
vote
2answers
2k views

lvm and a partitioning question

So when I do (as a root) fdisk -l I see /dev/sda1 and /dev/sda2 Now I am practicing creating logical volumes, when I tried partitioning /dev/sda2 I got two new partition /dev/sda2p1 and /dev/...
0
votes
1answer
497 views

Bash redirect 'interactive' output to special file

I want to redirect e.g. the output i.e. the progress bar of pv to /dev/tty1. ... | pv -perts 100G | ... 2> /dev/tty1 But what I get is whenever the progress bar updates it does so in an new ...
5
votes
1answer
2k views

Pipe a command to pv but hide all the original command's output

I'm trying to use pv, but I want to hide the command I piped in's output while still be able to see pv's output. Using command &> /dev/null | pv doesn't work (as in, pv doesn't receive any data)...
1
vote
0answers
117 views

How can I buffer a pipe with minimal added latency?

I am connecting to a debian system over a serial connection and streaming real time data from a pressure sensor. Originally I used: python3.6 pressure_streamer.py The data is being produced at 32....
1
vote
0answers
68 views

What's wrong with the way I'm using PV?

I'm using PV to insert a buffer of size 50M between my video recording command and my video streaming command: /usr/bin/raspivid -b 8000000 -w 1280 -h 720 -rot 270 -v -fps 40 -o - -t 0 | pv -B 50M | ...
1
vote
1answer
1k views

How does pv work?

How does the following command work? pv file.tar.gz | tar -xz From my understanding the pipe operator | creates a pipe and stdout of pv is mapped to the O_WRONLY end of the pipe and tar's stdin is ...
1
vote
0answers
73 views

pv working with existing PID doesn't work with -s option

I would like to both use -d and -s option while calling pv. I tried as below: $ cat -n /tmp/x.py 1 import time 2 import threading 3 import sys 4 5 def w(): 6 ...
1
vote
0answers
521 views

Pipe viewer (PV) not working?

I'm trying to use Pipe Viewer on a video pipe that I'm both writing and reading from. The below command works perfectly when I pass temp_video.h264 as an -i input to ffmpeg. However, when I attempt ...
3
votes
2answers
130 views

Which command produces more data per second?

Which command produces more data per second? This could be useful to quickly fill a file with garbage data or to test data transfer rates. So far, I found that "/dev/zero" is the quickest one. $ cat /...
5
votes
1answer
1k views

How track progress of a command in a pipe if only the size of its input is known in advance?

I would like to track progress of a slow operation using pv. The size of the input of this operation is known in advance, but the size of its output is not. This forced me to put pv to the left of the ...
16
votes
1answer
890 views

How to get an average pipe flow speed

If myfile is increasing over time, I can get the number of line per second using tail -f | pv -lr > /dev/null It gives instantaneous speed, not average. How can I get the average speed (i.e the ...
1
vote
1answer
339 views

Can the unit displayed in the transfer rate meter in pv be changed?

In pv, the rate meter is displayed as 47.5MiB 0:00:00 [ 165MiB/s] [================================>] 100% where the unit used for the transfer stats is MiB (1024 bytes). Is it possible to change ...
0
votes
0answers
377 views

LVM Logical Volume (linear) best performance

As a system administrator, one thing I do very often is increase LV's size. In most cases I have to extend first the VG, adding a new disk (we are talking about virtual machines). I have got a lot of ...
0
votes
1answer
127 views

What on earth is pv -bartpes

Many pages advise using a switch for pv called -bartpes. The following command from this page is an example: # openssl enc -aes-256-ctr -pass pass:"$(dd if=/dev/urandom bs=128 count=1 2>/dev/null |...
1
vote
2answers
2k views

How to redirect dd to pv? [duplicate]

This is my dd command which I need to modify: dd if=/tmp/nfs/image.dd of=/dev/sda bs=16k Now I would like to use pv to limit the speed of copying from NFS server. How can I achieve that? I know that ...
23
votes
2answers
12k views

Is it better to use cat, dd, pv or another procedure to copy a CD/DVD?

Background I'm copying some data CDs/DVDs to ISO files to use them later without the need of them in the drive. I'm looking on the Net for procedures and I found a lot: Use of cat to copy a medium: ...
0
votes
1answer
735 views

Volume Group fails to recover

For the last 10 days I've been living a nightmare trying to recover my data. I'm trying to recover VG and mount it to access the data here's what I get [root@]# pvscan Found duplicate PV ...
5
votes
2answers
2k views

Define physical volume inside non-partitioned disk

I've been able to define a physical volume (LVM) in two ways: Creating a 8e (Linux LVM type) partition and then # pvcreate /dev/sdb1 Usign pvcreate directly using a non-partitioned disk and then # ...
5
votes
1answer
746 views

Why doesn't sed exit immediately after writing the output?

I ran sed on a large file, and used the pv utility to see how quickly it's reading input and writing output. Although pv showed that sed read the input and wrote the output within about 5 seconds, sed ...
3
votes
1answer
5k views

Syntax When Combining dd and pv

In : sudo dd if=/dev/sda bs=64k | pv --size 1.5t | dd of=/dev/sdb Does the block size for dd go on the left side of this after the input as shown or on the right after the output? With the pipe ...
3
votes
2answers
807 views

Pipe Viewer - Progress monitor performance consequence

I am writing a batch script to sort through gigs and gigs of data. All of the data is text but the script will take a long long time to execute. I would like to give some visual indication that the ...
2
votes
1answer
113 views

Tar using progress bar and handling multi-volumes

I'm trying to create a tar of a large LVM volume onto multiple external drives. That part is solved. Now, because it is a LONG process, I'd like a progress bar. From the excellent articles here, I'...
3
votes
1answer
2k views

How to use pv with dd?

I'm trying to use dd with pv to write an iso to a usb drive. I did: iso="myiso.iso"; dd if="$iso" bs=1M | pv --eta --size $(ls -l "$iso" | awk '{print $5}') --progress --bytes --rate --wait > /dev/...
2
votes
2answers
98 views

Backed up over wrong hard drive

I've just written over the wrong hard drive using the command: sudo sh -c 'pv /dev/sdb >/dev/sdc' How do I go about undoing this? I was creating the first even backup of the drive, and I backed ...
1
vote
2answers
109 views

Why does it temporarily look like the pv command transfer limit is no longer enforced when I come out of suspend to ram?

I have this perl script and I discovered the pv command and decided to use it to get some feedback into what is going on with the randomness in terms of throughput. After a few tests1 I decided to ...
11
votes
1answer
10k views

Creating a bootable Linux installation USB without unetbootin

I want to install Scientific Linux from USB. I don't know why unetbootin doesn't work but I am not curious to find out: after all, I transferred to Linux from Windows to see and learn the underlying ...