0

Here is the situation:

a.csv

Contains values from some sensors captured every 1 second, the last column is the epoch (they actually looks like 1453998477.044).
(*s are the sensors values).

Example:

* , * , * , 2.356
* , * , * , 3.356
* , * , * , 4.356
* , * , * , 5.356
* , * , * , 6.356
* , * , * , 7.356
* , * , * , 8.356

b.csv

Have a fixed number of epochs (one per row) indicating the moment a certain part of a program was started/finished, so the epochs are sorted ascendantly.

In the next example, the program have two parts, it started at 2.421 and finished the first part at 5.500, then the other part was finished at 8.012.

2.421
5.500
8.012

The problem

The idea is to add a column to a.csv with this properties:

  1. By default it is filled with zeros.
  2. In each row where the epoch is the closest to some staring/finishing epoch in b.csv it must have a 1.

Following the above examples, the desire output is this:

* , * , * , 2.356 , 1
* , * , * , 3.356 , 0
* , * , * , 4.356 , 0
* , * , * , 5.356 , 1
* , * , * , 6.356 , 0
* , * , * , 7.356 , 0
* , * , * , 8.356 , 1

PS: The problem itself is a general issue, but it was difficult to me to explain it in general terms, that's why I used a specific case

Thanks guys

  • I don't understand where the values of the new column come from. Especially it is strange that the files have different numbers of lines. – Hauke Laging Feb 3 '16 at 14:46
  • The numbers come from the logic compassion, if the difference is less or equal than a certain delta it is added some flag, if not, other flag. I will edit the question to show why the files have different length. – onlycparra Feb 3 '16 at 14:57
  • What comparison? Do you want to compare the last field of a.csv to the 1st field of b.csv? Which line of b.csv? What are "upper and lower fields"? Please edit your question and clarify what you need. – terdon Feb 3 '16 at 15:11
  • the comarison is between the substraction and a delta: (a.csv, current row, $4) - (some value from b.csv) < delta ? if yes, add 1 to current row in a.csv, else add 0 – onlycparra Feb 3 '16 at 15:14
  • OK, what value from b.csv? The first? The third? All of them? How can we know? You need to give us enough data so we can reproduce your test files and process them to get your desired output. So, please edit your question and add the necessary information. – terdon Feb 3 '16 at 15:27
0

Command:

awk 'NR==FNR {values[NR]=$1; next;}
    { if (values[1]>values[2]) newvalue=0; else newvalue=1;
      print $0 "," newvalue; }' b.csv a.csv

Output:

*,*,*,1,1
*,*,*,2,1
*,*,*,3,1
  • Can you please explain me a little bit your answer? I don't get it and the output is different – onlycparra Feb 3 '16 at 15:40
  • @onlycparra Your "question" is very abstract, so is the answer naturally. That is a very gerneral example how you can add a column to a CSV file based on calculations from another file. – Hauke Laging Feb 3 '16 at 16:00
  • Thanks, I have made some editions, anyways, I am trying to understand your answer (I'm not very skilled with awk). Does values[NR] have the entire b.csv, right?. How can I put the 4th column of a.csv in an array othervalues[NR of a.csv] and do for each elements in othervalues[], if they are equal to some value in values[] then newvalue=1 – onlycparra Feb 3 '16 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.