Scientific Data processing(Graph comparison and interpretation)
- by pinkynobrain
Hi stackoverflow friends, I'm trying to write a program to automate one of my more boring and repetative work tasks. I have some programming experience but none with processing or interpreting large volumes of data so i am seeking your advice(both suggestions of techneques to try and also things to read to learn more about doing this stuff).
I have a piece of equipment that monitors an experiment by taking repeated samples and displays the readings on its screen as a graph. The input of experiment can be altered and one of these changes should produce a change in a section of the graph which i currently identify by eye and is what im looking for in the experiment. I want to automate it so that a computer looks at a set of results and spots the experiment input that causes the change.
I can already extract the results from the machine. Currently they results for a run are in the form of an integer array with the index being the sample number and the corresponding value being the measurement.
The overall shape of the graph will be similar for each experiment run. The change im looking for will be roughly the same and will occur in approximatly the same place every time for the correct experiment input. Unfortunatly there are a few gotcha's that make this problem more difficult.
There is some noise in the measuring process which mean there is some random variation in the measured values between different runs. Although the overall shape of the graph remains the same.
The time the experiment takes varies slightly each run causing two effects. First, the a whole graph may be shifted slightly on the x axis relative to another runs graph. Second, individual features may appear slightly wider or narrower in different runs.
In both these cases the variation isn't particularly large and you can assume that the only non random variation is caused by the correct input being found.
Thank you for your time,
Pinky