We continue our basic signal-processing posts with one on the moving-average, or smoothing, filter. The moving-average filter is a linear time-invariant operation that is widely used to mitigate the effects of additive noise and other random disturbances from a presumably well-behaved signal. For example, a physical phenomenon may be producing a signal that increases monotonically over time, but our measurement of that signal is corrupted by noise, interference, or flaws in the measurement process. The moving-average filter can reveal the sought-after trend by suppressing the effects of the unwanted disturbances.
In this post we will use discrete time exclusively. So this post serves two purposes: continuing on with simple idealized filtering concepts and solidifying the discrete-time signal-processing notation and analysis.
The moving average filter is a procedure that involves a long time series and a short averaging window. The window is slid along the data (it ‘moves’ along the data, as all convolutions do), and we take an average of whichever elements of the time series are currently within the window. It is also called the running averaging, rolling average, moving mean, and rolling mean. Wikipedia says, “A moving average is commonly used with time series data to smooth out short-term fluctuations and highlight longer-term trends.” In this post I describe how that works in terms of our linear time-invariant signal processing machinery: impulse responses and frequency responses (transfer functions).
The Discrete-Time Moving-Average Filter
Suppose we have some collected sampled data . The graph of versus reveals an erratic function that is difficult to interpret visually, as in Figure 1.
Intuitively, we would like to average out, somehow, the random fluctuations and leave only the true trends produced by the physical process that gave rise to the collected data: When is the data truly increasing? Decreasing? What is the rate of increase or decrease? When is it constant or zero?
To find answers to these questions, we can conceive of applying an averaging operation, knowing that if the additive noise at each sample is independent of the additive noise at the others, and the additive noise has a mean of zero, then such averaging will tend to suppress the effects of the noise while maintaining the presence of the non-noise component of the signal. If the signal component of the data is itself not independent sample-to-sample, then it will not be averaged away by the averaging process. The output of such an averaging operation is . We’ll define the value of to be the arithmetic average of the preceding points of and itself:
This is illustrated in Figure 2.
This averaging operation moves along with time, and so it is understandably called a moving average, as we mentioned at the top of the post.
Is this moving average operation a linear time-invariant system? First, let’s check for linearity. Let and . Then what is the output for ? If the system is linear, it must be . We have the output
So that the defined moving-average operation is linear.
Next we test for time-invariance (here, in discrete time, this is often called shift invariance). If the system is time invariant, then the output for a shifted version of an input is just the shifted output for the original unshifted input. Let . Then look at the output for input . If this is equal to , then the moving-average system is time-invariant. Let’s take a look.
Is ? Let’s perform a substitution for to find the answer. Let . Then if , . Similarly, if , then . Our output signal becomes
But the right side of (7) is, by definition, . So we have
and therefore the defined moving-average operation is time-invariant. Since the moving-average operation is equivalent to a linear time-invariant system, it can be represented by an impulse-response function or a frequency-response function (transfer function). Let’s find expressions for each of these key system input-output functions.
Impulse Response Function
To find the impulse-response function, we simply apply an input that is equal to the discrete-time impulse , which is shown in Figure 3, and determine the value of for each possible . In this situation, we rename as , the conventional symbol used for impulse-response functions in both continuous and discrete time.
implies that the impulse response is given by
Since is zero except at , where it is equal to 1, will be zero whenever the summation interval does not contain . Thus, for all , we have the interval which is entirely to the left of zero, provided that (which we assume is true). Therefore, the impulse response is zero for all negative values of .
When , the summation interval is , which does contain zero at the right edge, and so we have
For , we have
which is equal to provided that , as already assumed. This pattern continues, with the value of being until . For , the summation interval is , which no longer contains zero, and so . For , the interval continues to slide to the right of zero, and will never again contain zero, so that . In summary, we have the impulse-response function
The impulse-response function for the moving-average filter with length is simply a rectangle with width and height . The impulse response is graphed in Figure 4. This is consistent with the intended operation of the filter: sum the previous signal values and the current signal value, and divide by the number of values you summed, which is .
The frequency-response function is defined as the response of the system to a sine-wave input, and can be computed, as we’ve shown in a previous post, using the Fourier transform applied to the impulse response.
This version of uses a continuous frequency variable . It is periodic with period equal to one. Let’s evaluate the transform in (14), then think about discretizing frequency in anticipation of using a computer to analyze the situation in both discrete time and discrete frequency.
where . This last summation is an example of the general geometric series
So the frequency response for the moving-average filter is given by
Notes on the Frequency Response
We make the following observations about the frequency response.
- The derived frequency-response function is similar to the Fourier transform of a continuous-time rectangle function, which is a sinc function, but it is not identical to a sinc. In particular, the denominator contains the function instead of being proportional to .
- The derived frequency response is periodic with period . This periodicity follows from the periodicity of the discrete-time Fourier transform.
- For small values of , , and so . Therefore, the bulk of the energy in is concentrated near . The moving-average filter passes lower frequencies and attenuates or rejects higher frequencies.
- since .
- for . For example, , which implies that the moving-average filter output is zero for an input sine wave with frequency . (Can you see this from the time-domain operation of the filter?)
Sampled Frequency Response and Zero-Padding
The frequency response we’ve derived is valid for any , but it is also periodic with period 1. So the important values of reside in a single period, say . We can look at samples of the frequency response by simply substituting into our general formula. Say we want to look at the frequency values
where . Then we have the sampled frequency-response function
for . How can we compute this set of values using a transform? We have
To use the power and efficiency of the FFT, we can extend the sum from to , and simply define the missing values of to be zero. That is, pad with zeros and compute
This is called zero-padding, and it allows us to flesh out the details of the shape of the frequency-response function. When we use, say, , we pad with zeros, and the effect (not proven here) is that the original values of are preserved, but a new value is inserted between each pair of them. Later in the post we will provide a plot of that uses and .
Let’s switch gears and go back to the time domain to look at the response of the moving-average filter to a unit-step input. Recall that the discrete-time unit-step function is denoted by , and is equal to one for and zero otherwise. Let’s evaluate the convolution sum to determine the step response.
For , has no non-zero samples for . That is, and have no overlap where is nonzero. Therefore, the step-response for negative values of .
For and , the overlap between and is partial. We obtain
Finally, when , the overlap is complete, and we have
The step response function is shown in Figure 5.
Interconnection of Multiple MA Filters
Here we look at connecting multiple moving-average filters in series, which means that the equivalent linear time-invariant system has a frequency-response function that is the multiplication of the individual moving-average frequency responses. Assume that we connect identical moving-average filters, each having parameter and frequency-response function . Then the frequency response for the equivalent system is ,
On the other hand, the impulse response for the equivalent system is the convolution of the individual impulse-response functions,
What do the impulse responses and frequency responses look like as a function of ? We’ve already encountered the idea of convolving a rectangle with itself multiple times, so we know, for example, that for , the function will be a triangle. More importantly, the frequency response as a function of becomes more and more selective. That is, the attenuation of input frequency components far away from gets very large as grows. The impulse and frequency-response functions for various are shown in Figure 6. In this figure, the frequency responses were calculated by Fourier transforming a zero-padded version of the corresponding impulse responses. In terms of the variables defined above, we have and . This permits a relatively fine sampling of the frequency-response functions.
A Look at the Residual of the MA Filter
The moving-average filter output reveals the trends, if any, exhibited by the underlying noise-free data, and so is useful as a way to “de-noise” arbitrary datasets. But it may also be of interest to determine the properties of the noise that is rejected by the filter. In this case we want to find the trends with the moving-average filter, then remove them from the input, leaving only the high-frequency fluctuations and no trending signals to contaminate them.
Mathematically, the signal we desire is the input minus the moving-average output,
This can be expressed as the following function of time
which isn’t particularly revealing. Instead, we can attempt to use previous results and approaches to simplify the analysis. The signal can be obtained by convolving itself with an impulse, and is obtained by convolving with as in Figure 7.
Since we know is a rectangle with height and width , we easily obtain as in Figure 8.
The frequency response is
The impulse response and corresponding frequency response are plotted in Figure 9. It is clear that this filter passes only relatively high frequencies, providing the desired inverse operation relative to a moving average.
Significance of Moving-Average Filters in CSP
The main use I have for a moving-average filter is as a smoothing filter in the frequency-smoothing method (FSM) of spectral correlation function estimation. Certain kinds of specialized signal-to-noise ratio calculations can also benefit from the use of a fast smoothing operation, and the computational cost of a moving-average filtering operation is quite low because of the unique shape of the impulse response–a rectangle. This enables the use of a head-tail running sum to implement the convolution, avoiding having to do multiplications for each output point of the filter.