Tell me more ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 1 down vote favorite

I would like to calculate the difference between two data sets at any given moment in time. The problem is that the "timestamps" at which both data sets were taken differ. For example consider these data sets

time-1   state-1    time-2    state-2
6.164012 1          6.164015  1
6.290051 2          6.290055  2
6.454675 3          6.454678  3
7.006175 4          7.006178  4
7.080418 5          7.016555  5
7.193570 6          7.016595  4
7.351850 7          7.016595  3
7.465638 8          7.016595  2
7.492480 9          7.080421  3
7.524426 10         7.193572  4
8.006629 11         7.203975  5

It would be even sufficient to have the difference between state-1 and state-2 calculated at each 0.000001 seconds (brute force). Does anybody know how to solve this issue?

EDIT: To clearify my question: the first data set contains a time stamp (time-1) and a corresponding value (state-1); the second data set contains also a time stamp (time-2) and a corresponding value (state-2). I would like to compare the "state" variables at every moment in time to find the maximum difference between both. However, since the time stamps are not synchronized between both data sets I cannot simply compare line 1 with line 1 etc. The data sets can be read as follows: data set 1 has 1 state at time 6.164012 seconds and after 2 states after 6.290051 seconds. Data set 2 has 1 state after 6.164015 seconds and 2 states after 6.290055 seconds.

EDIT2: You can think of both data sets as step functions (x-axis is the time, y-axis the state variable). Then you end up with something like this:

^
|    +---------+
|    |         | +---+
| +--+   +---+ | |   |
| |      |   | +-+   + ... f1
|--------+   +-------- ... f2
+------------------------------->

And I want to know at which time (x value) the difference between f1 and f2 (y_1 - y_2) is at its peak. In this case I don't have "functions" but two data sets belonging to the same experiment being run with different parameters.

share|improve this question
I am not sure I understand. Do you have the same number of observations in each dataset? I mean, do you just want to compare t-1(0)<=>t-2(0), t-1(1)<=>t-2(1), t-1(N)<=>t-2(N) ? – terdon Oct 8 '12 at 14:44
What values is it supposed to use if only one of the sets has a state for a given time? – Michael Mrozek♦ Oct 8 '12 at 15:00
1  
You need to define what type of interpolation to do between timestamps: linear, sample-and-hold, etc. – Jim Paris Oct 8 '12 at 16:01
What is the specific question you are trying to answer? At what time point are the two values most different? In any case, I am getting the feeling this belongs in SO or math.SE. – terdon Oct 8 '12 at 19:16
This kind of data processing looks like a job for R. You'll find a lot of R experts on Cross Validated; I don't know if your question is suitable for that site in its current form, however. – Gilles Oct 19 '12 at 16:07

closed as off topic by derobert, jasonwryan, warl0ck, phunehehe, jw013 Oct 9 '12 at 16:28

Questions on Unix & Linux Stack Exchange are expected to relate to Unix or Linux within the scope defined in the FAQ. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about closed questions here.

1 Answer

up vote 1 down vote

Your question is not very clear. If what you are trying to do is compare the 2nd and 4th fields of one file do this:

awk '{print $2-$4}' file.txt

I suspect this is not what you are trying to do, if so, please update your question with more information so we can help you better.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.