View previous topic :: View next topic 
Author 
Message 
Per Svensson
Joined: 03 Oct 2004 Posts: 207 Location: Gothenburg, Sweden

Posted: Tue Feb 16, 2021 12:38 pm Post subject: Moving Average code for Bascom and a Windows simulator 


Many years ago I wrote a Bascom filter for moving average of integer data. Mostly used to smoothing noisy sensor data.
It is probably one of the most useful routines I have ever written, as I use it so often for all kind of applications where noise is involved.
I hope you will use it
It is a very compact and fast.
There is a small INCfile attached. It contains routines for Integer, Long and Single data types. The Single version also have a special feature where a negative averaging factor will force the filter to
instantly jump to the current input value. This can be useful in some cases where integration factor is very slow.
The Single version have no limitations of input data and averaging factor, whilst the Integer and long versions are limited by what their respective accumulators can swallow.
They are limited to (2**F)*FinalAverage < 7FFF_FFFF
This mean that the more smoothing you want the less size the final value can have. But observe that the limitation is only for the accumulator. The input data can have full swing as long as the accumulator does not overflow.
so for wildly transient data like spikes it will still work fine
This is the iterative math: DataNew = [DataOld*(2^Factor  1) + Newval] / (Factor)
If you are worried about accumulator overflow, then you can simply include a test of the size, but of course it will slow down the speed somewhat.
There is also a simulator for this type of filter that I wrote in VB5. You can install it in windows and use it to see what the filter output would be after a certain number of iterations.
It also permit simulation of noise in the input stream.
/Per 

Back to top 


albertsm
Joined: 09 Apr 2004 Posts: 5157 Location: Holland

Posted: Wed Feb 17, 2021 10:04 am Post subject: 


Thanks for sharing Per.
Maybe you can add a small bascom example that uses the functions.
I read about IIR_FILT_INT but i see no reference to it. _________________ Mark 

Back to top 


Per Svensson
Joined: 03 Oct 2004 Posts: 207 Location: Gothenburg, Sweden

Posted: Wed Feb 17, 2021 11:53 am Post subject: 


Ok Mark,
Here comes two test examples and an update of the include file (to cover some print formatting that I use)
RunAvg.bas is for integer data
RunAvg_sing.bas is for single float data
I have no separate example for the alternative IIRfilter that you mention, but it is now added to the include file so
you can try that call in a copy of the integer example if you like.
The effectiveness of all these filters rely on the speed of the shift function.
It could be even faster if the shift function was more optimized for word and longwords.
If the shift is a multiple of 8 for instance, then bytes can be shifted rather than bits. Etc.
/Per 

Back to top 


Per Svensson
Joined: 03 Oct 2004 Posts: 207 Location: Gothenburg, Sweden

Posted: Wed Feb 17, 2021 11:55 am Post subject: 


I forgot to mention that the two test examples runs nicely in the internal Bascom simulator.
Just compile and type F2 to execute it 

Back to top 


OFamily
Joined: 23 May 2010 Posts: 221 Location: Japan

Posted: Wed Feb 17, 2021 2:51 pm Post subject: 


Is it a type of exponential moving average (EMA) that subtracts the previous mean from the total value, not the simple moving average (SMA)?
I think the exponential moving average (EMA) needs a factor, where is it done?
Please tell me the principle of calculation.
best regards 

Back to top 


Per Svensson
Joined: 03 Oct 2004 Posts: 207 Location: Gothenburg, Sweden

Posted: Wed Feb 17, 2021 3:33 pm Post subject: 


Yes it works pretty much like you describe.
The standard way would have been to capture a certain (N) number of inputs and calculate the arithmetic mean. Then discarding the oldest each time a ned data comes in.
This requires a lot of memory and is time consuming if N is large.
The nice thing with this presented method is that there is no need to remember a series of input data values. We simply add them to an accumulator , and as
there is no discrete "oldest data" to discard we simply subtract the average from the sum.
Miraculously this works and as the iteration goes on and the accumulator divided by N approaches the average.
To avoid the slow division operator we use rightshift instead. The only drawback is that we now only can use a smoothing factor of N=2**k, where k is an integer.
But this is a small price to pay in most cases.
Note also what I wrote about the potential accumulator overflow. For large k the accumulator need to be mush wider than the input data type.
/Per 

Back to top 


OFamily
Joined: 23 May 2010 Posts: 221 Location: Japan

Posted: Thu Feb 18, 2021 12:14 am Post subject: 


Thank you for your answer.
I think it's a mathematically unfounded calculation method.
I also often use a lowpass filter that uses a simple moving average to remove noise from sensors.
It's similar to your calculation method, but it stores the data in a buffer, subtracts the oldest data from the total value, and then adds the latest data.
As for the lowpass filter of the sensor, the buffer is as small as 16 to 32, so memory consumption is not a problem.
Code:  '
' * Subroutine for moving average A/D conversion data of HX711 *
'
Hx711movavg:
Avgsum = Avgsum  Average(avgpoi) 'Subtract the oldest data.
Average(avgpoi) = Hx711ad 'Store new data in the moving average buffer.
Avgsum = Avgsum + Hx711ad 'Add up the new data.
'
Avgpoi = Avgpoi + 1 'Update the moving average pointer.
If Avgpoi > 16 Then 'Is the pointer the upper limit?
Avgpoi = 1
End If
'
Hx711avg = Avgsum
Shift Hx711avg , Right , 4 , Signed 'Take 16 moving averages.
Return 
I'm looking forward to seeing how it differs from your calculation method. 

Back to top 


enniom
Joined: 20 Oct 2009 Posts: 482

Posted: Thu Feb 18, 2021 1:04 am Post subject: 


.
I've used the following code to estimate Simple Moving Averages (SMA) for several years in a production environment with limited processing capability (and limited awake time to minimize power consumption). It works very well and compares favorably to SMAs calculated using the conventional method with large arrays (which need lots of RAM). The resulting values are extremely close.
Code:  Dim Sma(10) as Single 'Keep the calculated SMAs here
Declare Sub Movingaverage(byref Xx As Single , Byref Yy As Single , Byval Num As Word)
Sub Movingaverage(byref Xx As Single , Byref Yy As Single , Byval Num As Word)
Local Jnum As Word
Jnum = Num  1
Xx = Xx * Jnum
Xx = Xx + Yy
Xx = Xx / Num
End Sub
Call Movingaverage(Sma(2), 1.25, 300) 'Approximates a 300point moving average (or 5minute SMA if called every second)
'This can easily be converted to a Function as well

Also, if needed, the value of Num can be increased from 1 to nn at the start of the calculations until the desired number of points is reached.
E 

Back to top 


