Noise in measurements

11 posts / 0 new
Last post
Bizjak
Noise in measurements

I am new and do not have my scale up and running. I notice noise spikes in the data on some systems. One thing to consider is to use a mean instead of an average to avoid noise spikes. With an average calculation very large values or small values can drastically change the average. The mean is just the center value and so may be a good way to reduce spikes.

Adam
I've noticed some spurious

I've noticed some spurious readings since I installed my hive scale the other day - I wonder if it might be a good idea to re-read the sensors that are polled every time hive.sh is run if the values the sensors read are significantly different from the previous or average values.

Even plotting a trendline based on a moving window average might be useful.

Either way, those data points can be ignored after the fact. I'm going to be messing with my data and try to develop some useful functions for massaging the data. My schedule is packed full of bee removals right now - but hopefully I will have time to play around with some code.

Paul
Paul's picture
Fixed or greatly improved

You may be seeing two kinds of noise. I just mention wind noise as I know that is not what you are talking about.

Bad readings from sensors have always been a concern and focus. At the data collection level, I don't like to filter or hide them -I like to eliminate them. Reading serial sensors from user space on the Pi running non real time OS will result in bad reads.

One approach Adrian and I have discussed is what I call a Data Acquisition Daemon (dad).
http://hivetool.org/w/index.php?title=Change:_Data_Acquisition It is just all the drivers rolled into one c program that runs in a loop in the background. The start of it will be in the next distribution.

Just before this forum started we had an email storm on fixing the bad reads. The HX711, DHT22 and hive.sh programs have been improved and the bad reads should be almost eliminated. I will post the fixes to the forum.

Adam
Seems like there is a wealth

Seems like there is a wealth of info still hidden deep in the e-mails.

I'd love to start contributing in some way but I don't want to re-invent the wheel!

Paul
Paul's picture
Keep asking questions until

Keep asking questions until we get all the "insider information" published!

Paul
Paul's picture
to fix HX711

If you are running the hivetool image 0.7x and you like to compile, edit /home/downloads/hx711/hx711wp.c

around line 192 replace
delayMicroseconds(.2);
with
delayMicroseconds(1);

The gcc command to compile is at the top of the file or see: http://hivetool.org/w/index.php?title=HX711
Guess it needs to be fixed there, too.

Paul
Paul's picture
mean or median

Bizjak,

You say use the mean instead of the average. I think the average and mean are the same and you probably meant median. It took me a while to think it through, but you are right. That would get rid of the NO DATA TO CONSIDER error caused by one huge outlier.

I found this code to calculate the median http://stackoverflow.com/questions/1961173/median-function-in-c-math-lib... :

int compare (const void * a, const void * b)
{
  return ( *(int*)a - *(int*)b );
}

int main ()
{
  int n;
  qsort (values, ELEMENTS, sizeof(int), compare);
  for (n=0; n

I'm going to give it a try in hx711wp.c

Thanks!

Bizjak
Yes, median was the process I

Yes, median was the process I was thinking of, another advantage of it is that is is a point in the data set and down not require as many cycles to calculate the average (if cycle count is an issue)

Adam
So I've not been able to look

So I've not been able to look at my hive data too much since the Pi is powering down every now and then, but I've noticed that for almost every other reading of the hive weight, a messed up weight is recorded - not really 'noise' since the differences are so large. I tried the re-compile with:

delayMicroseconds(1);

but the differences weren't apparent from a visual inspection.

Paul
Paul's picture
did you copy it to /usr/local/bin and rename

Dumb question but I have to make sure. After you edited and compiled the HX711 code, did you copy it to /usr/local/bin and rename it:

sudo cp hx711wp /usr/local/bin/hx711

Adam
In your earlier post, you

In your earlier post, you referenced:

around line 192 replace... and make the change from .2 to 1 second.

It seemed like there were delayMicroseconds(.2); statements all over the place, not just line 192.

When I compared the hx711wp.c to the one included in the zip files you provided for the hivetool 0.73 image, I saw that the statements were all delayMicroseconds(1);

I made that change and recompiled -> Last few hours of readings appear stable thanks.

Log in to post comments