38

If have a long text file and I want to display all the lines in which a given pattern occurs, I do:

grep -n form innsmouth.txt | cut -d : -f1

Now, I have a sequence of numbers (one number per line)

I would like to make a 2D graphical representation with the occurrence on the x-axis and the line number on the y-axis. How can I achieve this?

enter image description here

  • 1
    Could you explain what you mean by occurrence? Do you mean how many times a particular number is found in the file? Or do you just want the actual value of the number on the x-axis and the line number that number was found on on the y-axis? – terdon Mar 15 '15 at 20:05
  • I mean by occurence simply in which order a pattern was found. e.g.: first time in line 400, second time in line 410 , third time in line 412 ... – Abdul Al Hazred Mar 15 '15 at 20:38
44

You could use gnuplot for this:

 primes 1 100 |gnuplot -p -e 'plot "/dev/stdin"'

produces something like

enter image description here

You can configure the appearance of the graph to your heart's delight, output in various image formats, etc.

  • 2
    I have downloaded gnuplot and tried to test it by entering: seq 100 | gnuplot -p -e 'plot "/dev/stdin"' . strangely no graph appeared, but the exit code (echo $?) was 0 , so no error appeared either. – Abdul Al Hazred Mar 15 '15 at 22:18
  • @AbdulAlHazred did you install gnuplot or gnuplot-x11? if the former, afaik it only provides file output (i.e. generating pdf, png etc. files) rather than interactive plots direct to the screen. – steeldriver Mar 15 '15 at 22:55
  • @AbdulAlHazred: What happens if you just do seq 100 >seq.dat, then run gnuplot interactively and at the prompt type plot "seq.dat" ? – Nate Eldredge Mar 15 '15 at 23:02
  • @steeldriver I have an error Failed to initialize wxWidgets. with gnuplot-x11... Do I need to have one or the other? or can both gnuplot and gnuplot-x11be installed? – 3kstc Mar 17 '15 at 3:27
  • 1
    Very nice; append notitle to plot without the title. – Victoria Stuart Aug 9 '18 at 23:00
13

I would do this in R. You'll have to install it but it shouold be available in your distributions repositories. For Debian-based systems, run

sudo apt-get install r-base

That should also bring in r-base-core but if it doesn't, run sudo apt-get install r-base-core as well. Once you have R installed, you could write a simple R script for this:

#!/usr/bin/env Rscript
args <- commandArgs(TRUE)
## Read the input data
a<-read.table(args[1])
## Set the output file name/type
pdf(file="output.pdf")
## Plot your data
plot(a$V2,a$V1,ylab="line number",xlab="value")
## Close the graphics device (write to the output file)
dev.off()

The script above will create a file called output.pdf. I tested as follows:

## Create a file with 100 random numbers and add line numbers (cat -n)
for i in {1..100}; do echo $RANDOM; done | cat -n > file 
## Run the R script
./foo.R file

On the random data I used, that produces:

enter image description here

I am not entirely sure what you want to plot but that should at least point you in the right direction.

  • My Rscript v3.4.4 generates plots.pdf by default, regardless if using ggplot or plot. – Vorac May 27 '18 at 10:06
  • @Vorac did you mean to comment on another answer? What does ggplot have to do with it? And why is the default output filename relevant? – terdon May 27 '18 at 10:10
  • On my debian system this subset of your script is enough #!/usr/bin/env Rscript; args <- commandArgs(TRUE); a<-read.table(args[1]); plot(a$V2,a$V1,ylab="line number",xlab="value"); to generate an Rplots.pdf in the same directory. – Vorac May 27 '18 at 10:25
  • 1
    @Vorac yes, of course. But I want to choose the output file name. And, more importantly show how it can be done so it can be scripted. Otherwise, each time you run an RScript, it will use the same name and overwrite the output of a precious run. – terdon May 27 '18 at 10:30
11

If it might be that a very simple terminal printout would suffice, and that you could be satisfied by inverted axes, consider the following:

seq 1000   |
grep -n 11 |
while IFS=: read -r n match
do  printf "%0$((n/10))s\n" "$match"
done

The above charts an inverted trend on a 10% scale for every occurrence of the pattern 11 in the output of seq 1000.

Like this:

11
        110
        111
        112
        113
        114
        115
        116
        117
        118
        119
                  211
                            311
                                      411
                                                511
                                                          611
                                                                    711
                                                                              811
                                                                                        911

With dots and occurrence count it could be:

seq 1000    |
grep -n 11  | {
i=0
while IFS=: read -r n match
do    printf "%02d%0$((n/10))s\n" "$((i+=1))" .
done; }

...which prints...

01 .
02           .
03           .
04           .
05           .
06           .
07           .
08           .
09           .
10           .
11           .
12                     .
13                               .
14                                         .
15                                                   .
16                                                             .
17                                                                       .
18                                                                                 .
19                                                                                           .

You could get the axes like your example with a lot more work and tput - you'd need to do the \033[A escape (or its equivalent as is compatible with your terminal emulator) to move the cursor up a line for each occurrence.

If awk's printf supports space-padding like the POSIX-shell printf does, then you can use it to do the same - and likely far more efficiently as well. I, however, do not know how to use awk.

1

Enhancing Nate's answer to have PDF output and to plot lines (requires the rsvg-convert):

| gnuplot -p -e 'set term svg; set output "|rsvg-convert -f pdf -o out.pdf /dev/stdin"; plot "/dev/stdin" with lines'
0

Or you can redirect the stdout data through pipe to a custom python script. This will allow you immense amount of customization and flexibility in parsing, pre-processing and visualizing the data.

Here is a tutorial on this I wrote to do exactly as you intend. link

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.