1

I have the following script which has been running for over 3 days:

seq -w 1 1000000 | while read i; do (./myscript.pl $i >> output.txt); done

Is it possible to find out how far it has got with the sequence?

0
1

Since you're calling a command with a different argument for each element, check what argument that command is currently processing.

ps w -C myscript.pl

You can get an approximate value by checking how many write system calls the seq process has performed. Find the process ID with pgrep seq (or some other command such as ps uw -C seq if there are multiple seq processes and you need to identify which one you want). If the PID is 1234, then

grep syscw /proc/1234/io

reads some IO statistics of the process from /proc¹. Add one to this value² then multiply by getconf PIPE_BUF /, which is 4096, to get the number of bytes that seq has written. This is the total length of all the numbers that it's written so far. Since you're printing constant-width lines, divide by 8 to get the number of numbers written so far.

A more convenient way to see how far seq has gone is to trace its system calls with strace. On Ubuntu, by default, you need to be root to do this.

$ strace -p1234
strace: Process 1234 attached
write(1, "0008193\n0008194\n0008195\n0008196\n"..., 4096^Cstrace: Process 1234 detached

In the example above, 8193 is the next number that seq will write to the pipe, when the reading side has finished processing 8192 and is ready for more.

¹ If seq was writing to a regular file, then cat /proc/1234/1/fdinfo or lsof -p1234 would reveal the offset in the file, which is the number of bytes that seq has written. But for a pipe you don't get this information.
² Determined experimentally. Offhand I don't know why the first write operation writes twice PIPE_BUF.

0

I don't know what your script does, but could you watch cat output.txt in another terminal to monitor what your output file looks like?

Alternatively (and I know this isn't what you're asking for) perhaps run your command like this:

seq -w 1 1000000 | while read i; do (./myscript.pl $i >> output.txt & echo i); done

or

seq -w 1 1000000 | while read i; do (./myscript.pl $i tee -a output.txt); done

0

Use the pv pipe view util:

seq -w 1 1000000 | pv -lcs 1M | 
while read i; do (./myscript.pl $i >> output.txt); done

...which will output to STDERR a constantly updated progress bar giving both the number of lines read, and the percentage of lines done at the moment.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.