I have .txt files with the following structure:


My goal is to search for lie events represented in the last column. This I realized with

sed -n '13,$s/lie/&/p' $subj > $SUBJECTS_DIR/temp.txt

where I started the search from line13 and saves all lines with 'lie' to my temp-file. But is it possible to collect data during the sed-command? The first column contains a timestamp: so when lie=TRUE I want to collect the value from column_1; also, when the last lie of this sequence (in total 8 'lie' occurs the corresponding value of column_1 is needed to calculate the time-difference. The search should be continued until file ends. Has someone any idea how to collect these informations during sed?

  • 1
    Wrong tool. sed can't do calculations itself. So you need to involve external tools (like using the e option to the s command in GNU sed) or implement it yourself (which can be fun, but is obviously nonsense). Switch to a tool like python or perl.
    – Philippos
    Jul 13 '17 at 11:30

You can use awk for this.

awk -F"," '/lie/{a[++i>2?2:i]=$1}END{print a[2] - a[1] }' input .txt

The awk command will store the first and the most recent timestamp that matches "lie" in an array. Once the file has been read, awk will subtract the first match from the last match.

  • Thanks for this line, it calculates exactly the difference I need! But is it possible to get multiple timestamp differences from one file? In my data example, there starts condition 'lie' in line3 and ends in line10; it starts again in line14 and ends in e.g. line22 following again a gap and 'lie' starts again in line34... and so on. How can awk automatically detect this? Thnaks a lot!!
    – osteocyt
    Jul 17 '17 at 7:21

It is not clear for me what is "time difference", but the first column with the "lie" can be simply expanded with this command:

cat data.txt | grep lie | sed "s/,.*,lie//g"

grep discards all lines without "lie" and sed deletes all except first column.

Then, you can f.e. transform it into array and work with needed numbers in bash

  • "time difference" means that in the above example I need to substract the value in first column of line 10 (describes a time) the the value in first column of line 3. This sequence of 'lie' is followed by a gap and the next sequence 'lie' starts again and so on. I want to have the length of this particular events...
    – osteocyt
    Jul 17 '17 at 7:33

Probably Too Late to Be Posting This:

sed  -nE '13,$ s/([^,]*).*lie$/\1/p' in.txt


  • -nE:

    • -n for --quiet, --silent : no automatic printing of pattern space.
    • -E for --regexp-extended : use extended regular expressions
  • ([^,]*) : captures the timestamp

  • .*lie$ : ensures that the line ends with a 'lie'

Thus: timestamps for "lie" starting from line 13 would be returned

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.