matlab - Algorithm to detect a linear behaviour in a data set -


i have posted question algorithm make polynomial fit of part of data set time ago , received propositions wanted. face problem try apply ideas suggested in answers. goal find best linear fit of data set, in part of linear.

here example of must :data sets

we have these 2 data sets, , must make linear trend of linear part of data @ left of dashed line. in red, have ideal data set, has linear part beginning until dashed line. in blue, have 'problematic' data set, has plateau. bold part part have use linear fit of data.

my problem tried mentionned in question linked above : found second order derivative of smoothed data , looked when not 'close enough' of 0. here results problematic data set (first image) , ideal data set (second image) :

enter image description here enter image description here

(sorry quality, don't know why blurred) on both images, plotted first order derivative , in red, second order derivative. on first image, see peaks of second derivative values. problem peaks not 'high', making difficult establish threshold tell if set linear or not... on contrary, peak of first derivative quite high, making easy see visually.

i thought calculate mean of values of first derivative , when value differ mean value enough... when take mean of values of first derivative in order see values differ mean value, there sort of offset due peak.

how remove offset in order take mean value of data at right (the data @ left of discontinuity seen on image 1 non linear or linear have different value values @ right!) of peak efficiently ?

the mean operator (as have noticed) very sensitive outliers (peaks). may wish use more robust estimators, such median or x-percentile of values (which should more appropriate case).


Comments