0

I want to monitor/follow changes of a CSV file (some logging output of a robot). It is a huge file with <TAB> as delimiter and the first row with header information.

So far, I am using the output of tail - f <file.csv>, but it is not formatted per column, and looks like:

yAccelRaw       zAccelRaw       xGyroRaw    ... ... ...
3       256     1   ... ... ...
4       255     3   ... ... ... 
4       255     -6  ... ... ...
3       253     -1  ... ... ...
4       254     2   ... ... ...
5       255     0   ... ... ...
4       255     3   ... ... ...
5       254     3   ... ... ...
5       253     -1  ... ... ...
4       255     3   ... ... ...

With a CSV with 30 or more columns, understanding which value belongs to which column is not really easy. I was wandering if there is a general solution for printing the output formatted as table?

So far, I use tail -f <file.csv> | cut -f5,6 to cut out specific columns and observe their output, but I would prefer the full overview. Also, I tried piping the result to column which doesn't get updated.

1

You could use, for example, awk to capture the headers on the first line, and repeat them every 10 lines, and reduce the column size to say max 5 characters with something like this:

tail -n +1 -f file.csv |
awk  'NR==1 { n=split($0,hdr) }
NR%10==1    { for(i=1;i<=n;i++)printf "%5.5s ",hdr[i];printf "\n" }
            { for(i=1;i<=n;i++)printf "%5.5s ",$i;printf "\n" }
'

This would give something like

yAcce zAcce xGyro ...
    3   256     1 ...
    4   255     3 ... 
    4   255    -6 ...
    3   253    -1 ...
    4   254     2 ...
    5   255     0 ...
    4   255     3 ...
    5   254     3 ...
    5   253    -1 ...
    4   255     3 ...
yAcce zAcce xGyro ...
    3   256     1 ...
...

If the width is still too wide to fit in your terminal, you can pipe the result into less -S +F which makes it act like tail -f, but allows you to scroll left and right with the arrows if you pause the tail with Control-C. To continue tailing type F.

  • It helps a lot! Extra points for the header repetition :) However, can you expand on the less -S -F idea? I cannot scroll when piping the result to less. Also, the header gets eaten. – Marcus Jun 29 '17 at 22:00
  • It is +F not -F. It makes less do a tail -f initially. You cannot scroll at this point. You have to interrupt with control-c to stop the tailing, but you stay inside less. you can then scroll: check your keybindings in less if you cannot. When you have the view of the columns you want you can type F again to continue tailing with these columns. – meuh Jun 30 '17 at 5:35
  • If I pause less, then scroll, and try to continue, less is "Waiting for data... (interrupt to abort)" indefinetly. My work around is piping to less without +F, scroll and then initiate following with shift+F (that also is non-pausable of course in terms of "continue following after pausing"). I assume there is an issue with following a piped input. – Marcus Jul 4 '17 at 17:36
  • There will be a problem when piping a command into less because the control-c can also be delivered to the command as a signal which will probably stop it. The simplest answer is to put the result in a file and use less on that file, so that a control-c will only be received by less. Or you can pipe from a shell script if you get it to ignore signals with trap '' int. You will have difficulty stopping the script: a useful trick is to use control-z to put it in the background then use a kill command, or use the quit signal (control-\). – meuh Jul 4 '17 at 18:06
  • I found a similar answer in unix.stackexchange.com/questions/26826/… ... anyhow, doesn't it make the last part of your answer a bit misleading? – Marcus Jul 5 '17 at 9:36
0

Column does exactly what you're asking for. You can identify tab as the separator with an argument. Why is piping to column not an option here? cat <file.csv> | column -s "\t"

  • 1
    It's not an option because I want to monitor file changes. I know column for that task, but, as far as I know, I have to take care of updating the file input. – Marcus Jun 29 '17 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.