We continue our progression of Signal-Processing ToolKit posts by looking at the frequency-domain behavior of linear time-invariant (LTI) systems. In the previous post, we established that the time-domain output of an LTI system is completely determined by the input and by the response of the system to an impulse input applied at time zero. This response is called the impulse response and is typically denoted by .
The impulse response is useful and mathematically elegant, but looking at it as a graph doesn’t provide a lot of insight into the particular linear system under study. At least for me, anyway. For example, here are several impulse responses graphed on a single set of axes:
It is difficult to view the magnitude and phase of the impulse response and quickly surmise how input signals are transformed into output signals, even for the simple (but important) case of inputs that are sinusoidal. So let’s take a close look at just how a linear time-invariant system transforms a sine-wave input.
LTI Input-Output Relationship for Sine-Wave Input in Continuous Time
Suppose our input signal is ,
where is the real-valued amplitude, is the frequency, and is the phase parameter. Now recall that the output of any linear time-invariant system is the convolution of the input and the impulse-response function ,
Using our sine-wave in (1), we can calculate the output in the following way
More compactly, we use the symbol to denote the Fourier transform of to obtain the input-output relationship
We interpret this result to mean that an LTI system passes a sine-wave input to the output after scaling it by the complex number , where is the frequency of the input sine wave and is the Fourier transform of the impulse response. That is, “LTI System” implies “Sine-wave In, Sine-wave Out.” Linear systems do not create new frequency components, they can only scale each frequency component at the input to produce a contribution to the output.
The number is the response to the sinusoidal input with frequency , and so the function describes the full frequency response for all possible sine-wave inputs.
So we can guess that the response to an arbitrary input will consist of the sum of the weighted sine-wave components of the input signal, where each sine-wave component with frequency is weighted by the complex number . We’ll show this is in fact true later in the post.
I note in passing that this analysis of linear time-invariant systems’ input-output behavior can be viewed as deriving the Fourier transform (see (5)).
LTI Input-Output Relationship for Sine-Wave Input in Discrete Time
Suppose we input a discrete-time sine wave to a linear shift-invariant system with impulse response ,
The output is the input convolved with the impulse response
which is the same result we found in continuous time. If is causal, and has only a finite number of non-zero values (it is an FIR filter), then for and , and we have
We can sample this function of frequency over the interval times by using the frequencies :
which we recognize as the discrete Fourier transform.
Returning to the example of the three impulse responses in Figure 1, if we instead compute and plot the frequency responses, we obtain the graphs in Figure 2:
Focusing on the upper plot, which shows the magnitude of the frequency response for each linear time-invariant system (filter), we see that these three filters are piecewise constant. They are either zero or, on some intervals, they are equal to a non-zero constant. For any frequency for which the frequency response function is zero, an input sine wave with frequency will produce a zero-valued output. That is, the sine-wave will be blocked and will not appear at the output (more mathematically, it appears at the output with zero amplitude).
Readers having some familiarity with filter terminology will recognize that the three filters in Figure 2 are ideal filters, and that is a low-pass filter, is a bandpass filter, and is a high-pass filter.
General Input-Output Relationship for LTI Systems
Now let’s apply the Fourier transform to the time-domain input-output relationship for LTI systems, which is the convolution integral. That is, we have the output for the input of an LTI system with impulse-response function ,
We want to find the Fourier transform of . Can you guess what it will be in terms of the Fourier transforms of and ? Let’s use the symbol to denote the Fourier transform operator:
Let’s go through the exercise.
So, a convolution in the time domain is a multiplication in the frequency domain. A truly difficult-to-evaluate convolution may be quite easy in the frequency domain, provided the individual transforms and can be found. Moreover, this basic frequency-domain input-output characterization is consistent with our original encounter with as the complex number that scales the amplitude of an input sine wave with frequency : describes the scaling of each and every sine-wave component of the input.
Because describes how the linear time-invariant system transfers an arbitrary frequency-domain input to its frequency-domain output, is also called the transfer function of the filter.
Let’s circle back to the meaning of the impulse response. Suppose we have an input described by the Fourier transform
that is, the transform of the input is equal to one for all frequencies . Then our frequency-domain input-output relation reduces to
which means the output of the system is equal to the transfer function. But what time-domain has for all ? Well, we already encountered this signal, which is the impulse function
So, if we manufacture an impulse function, and apply it to our system, we will obtain the impulse response by definition, and this corresponds to an input that simultaneously applies all possible unit-magnitude sine waves to the system. This explains the origin of the formerly common behavior called ‘kicking the tires.’ When inspecting a car, it is stereotypical that the inspector (customer) will give a sharp kick to a tire to see what happens with the car. That is, if something falls off, the tire moves around, or some unsettling noises emit from under the hood. The inspector here is simultaneously applying all sinusoidal forcing functions to the mechanical system, hoping to observe a tell-tale resonance.
Significance of the Frequency Response in CSP
The frequency response, or transfer function, of a linear time-invariant system comes up in various places throughout the theory of CSP. The most prominent example is when we want to find the spectral correlation function for a random signal that has passed through an LTI system: what is the spectral correlation function for the output as a function of the spectral correlation of the input signal and the transfer function of the filter? As a concrete example, the input signal might be a communication signal and the LTI system might be a multipath propagation channel. We’ve already shown that the result (for the non-conjugate SCF) is
See the post on signal processing operations in CSP for more details.