# SPTK: Frequency Response of LTI Systems

The frequency response of a filter tells you how it scales each and every input sine-wave or spectral component.

We continue our progression of Signal-Processing ToolKit posts by looking at the frequency-domain behavior of linear time-invariant (LTI) systems. In the previous post, we established that the time-domain output of an LTI system is completely determined by the input and by the response of the system to an impulse input applied at time zero. This response is called the impulse response and is typically denoted by $h(t)$.

[Jump straight to ‘Significance of Frequency Response in CSP’ below.]

The impulse response is useful and mathematically elegant, but looking at it as a graph doesn’t provide a lot of insight into the particular linear system under study. At least for me, anyway. For example, here are several impulse responses graphed on a single set of axes: Figure 1. Three impulse response functions. What can we infer from these graphs? Which linear time-invariant filter will produce an output sine wave with frequency 0.25 with a non-zero output amplitude?

It is difficult to view the magnitude and phase of the impulse response and quickly surmise how input signals are transformed into output signals, even for the simple (but important) case of inputs that are sinusoidal. So let’s take a close look at just how a linear time-invariant system transforms a sine-wave input.

### LTI Input-Output Relationship for Sine-Wave Input in Continuous Time

Suppose our input signal is $x(t)$, $\displaystyle x(t) = A e^{i 2 \pi f_0 t + i \phi} \hfill (1)$

where $A$ is the real-valued amplitude, $f_0$ is the frequency, and $\phi$ is the phase parameter. Now recall that the output $y(t)$ of any linear time-invariant system is the convolution of the input $x(t)$ and the impulse-response function $h(t)$, $\displaystyle y(t) = \int_{-\infty}^\infty x(u) h(t-u)\, du \hfill (2)$ $\displaystyle = \int_{-\infty}^\infty x(t-u) h(u) \, du \hfill (3)$

Using our sine-wave $x(t)$ in (1), we can calculate the output in the following way $\displaystyle y(t) = \int_{-\infty}^\infty \left[ A e^{i 2 \pi f_0 (t-u) + i\phi} \right] h(u) \, du \hfill (4)$ $\displaystyle = \underbrace{\left[ Ae^{i 2 \pi f_0 t + i\phi} \right]}_{Independent\ of\ u} \underbrace{\int_{-\infty}^\infty h(u) e^{-i 2 \pi f_0 u} \, du}_{Fourier\ Transform\ of\ h(u)} \hfill (5)$

More compactly, we use the symbol $H(f)$ to denote the Fourier transform of $h(t)$ to obtain the input-output relationship $\displaystyle y(t) = H(f_0) \left[ A e^{i 2 \pi f_0 t + i \phi} \right] \hfill (6)$

or $\displaystyle y(t) = H(f_0) x(t) \hfill (\mbox{\rm Sinusoidal\ x(t)}) \hfill \ \hfill (7)$

We interpret this result to mean that an LTI system passes a sine-wave input to the output after scaling it by the complex number $H(f_0)$, where $f_0$ is the frequency of the input sine wave and $H(f)$ is the Fourier transform of the impulse response. That is, “LTI System” implies “Sine-wave In, Sine-wave Out.” Linear systems do not create new frequency components, they can only scale each frequency component at the input to produce a contribution to the output.

The number $H(f_0)$ is the response to the sinusoidal input with frequency $f_0$, and so the function $H(f)$ describes the full frequency response for all possible sine-wave inputs.

Now recall that all energy signals can be represented by their Fourier transform (and some power signals too), $\displaystyle x(t) = \int_{-\infty}^\infty X(f) e^{i 2 \pi f t} \, df \hfill (8)$ $\displaystyle = \int_{-\infty}^\infty \underbrace{\left[ X(f) df \right]}_{\substack{Complex\\ Amplitude \\ Includes \\ Mag\ \&\ Phase}} \underbrace{e^{i 2 \pi f t}}_{\substack{Sine\ Wave \\ With\ Freq\ f}} \hfill (9)$

So we can guess that the response to an arbitrary input will consist of the sum of the weighted sine-wave components of the input signal, where each sine-wave component with frequency $f$ is weighted by the complex number $H(f)$. We’ll show this is in fact true later in the post.

I note in passing that this analysis of linear time-invariant systems’ input-output behavior can be viewed as deriving the Fourier transform (see (5)).

### LTI Input-Output Relationship for Sine-Wave Input in Discrete Time

Suppose we input a discrete-time sine wave to a linear shift-invariant system with impulse response $h(k)$, $\displaystyle x(k) = A e^{i 2 \pi f_o k T_s +i \phi} \hfill (10)$ $\displaystyle = A e^{i 2 \pi f_1 k + i \phi} \ \ \ (f_1 = f_0T_s = f_0/f_s) \hfill (11)$

The output $y(k)$ is the input convolved with the impulse response $\displaystyle y(k) = x(k) \otimes h(k) \hfill (12)$ $\displaystyle = \sum_{j=-\infty}^\infty x(j) h(k-j) = \sum_{j=-\infty}^\infty x(k-j) h(j) \hfill (13)$ $\displaystyle = \sum_{j=-\infty}^\infty \left[ A e^{i 2 \pi f_1(k-j) + i\phi} \right] h(j) \hfill (14)$ $\displaystyle = A e^{i 2 \pi f_1 k + i \phi} \sum_{j=-\infty}^\infty h(j) e^{-i 2 \pi f_1 j} \hfill (15)$ $\displaystyle = A e^{i 2 \pi f_1 k + i \phi} H(f_1) \hfill (16)$ $\displaystyle = x(k) H(f_1) \hfill (17)$

which is the same result we found in continuous time. If $h(j)$ is causal, and has only a finite number of non-zero values (it is an FIR filter), then $h(j) = 0$ for $j<0$ and $j \ge N$, and we have $\displaystyle H(f) = \sum_{j=-\infty}^\infty h(j) e^{-i 2 \pi f j} = \sum_{j=0}^{N-1} h(j) e^{-i 2 \pi f j} \hfill (18)$

We can sample this function of frequency over the interval $f \in [0, 1]$ $N$ times by using the frequencies $k/N, k = 0, 1, \ldots, N-1$: $\displaystyle H(k/N) = \sum_{j=0}^{N-1} h(j) e^{-i 2 \pi j k /N}, \ \ k = 0, 1, \ldots, N-1 \hfill (19)$

which we recognize as the discrete Fourier transform.

Returning to the example of the three impulse responses in Figure 1, if we instead compute and plot the frequency responses, we obtain the graphs in Figure 2: Figure 2. Frequency response functions corresponding to the impulse response functions graphed in Figure 1. Which filter can pass an input sine wave with frequency 0.25 to its output with a non-zero amplitude?

Focusing on the upper plot, which shows the magnitude of the frequency response $|H(f)|$ for each linear time-invariant system (filter), we see that these three filters are piecewise constant. They are either zero or, on some intervals, they are equal to a non-zero constant. For any frequency $f$ for which the frequency response function $H(f)$ is zero, an input sine wave with frequency $f$ will produce a zero-valued output. That is, the sine-wave will be blocked and will not appear at the output (more mathematically, it appears at the output with zero amplitude).

Readers having some familiarity with filter terminology will recognize that the three filters in Figure 2 are ideal filters, and that $H_1(f)$ is a low-pass filter, $H_2(f)$ is a bandpass filter, and $H_3(f)$ is a high-pass filter.

### General Input-Output Relationship for LTI Systems

Now let’s apply the Fourier transform to the time-domain input-output relationship for LTI systems, which is the convolution integral. That is, we have the output $y(t)$ for the input $x(t)$ of an LTI system with impulse-response function $h(t)$, $\displaystyle y(t) = \int_{-\infty}^\infty x(u) h(t-u)\, du \hfill (20)$

We want to find the Fourier transform of $y(t)$. Can you guess what it will be in terms of the Fourier transforms of $x(t)$ and $h(t)$? Let’s use the symbol ${\mathcal{F}}$ to denote the Fourier transform: $\displaystyle Y(f) = {\mathcal{F}} [y(t)]. \hfill (21)$

Let’s go through the exercise. $\displaystyle Y(f) = {\mathcal{F}} \left[\int_{-\infty}^\infty x(u) h(t-u)\, du \right] \hfill (22)$ $\displaystyle = \int_{-\infty}^\infty \left[ \int_{-\infty}^\infty x(u) h(t-u) \, du \right] e^{-i 2 \pi f t} \, dt \hfill (23)$ $\displaystyle = \int_{-\infty}^\infty \int_{-\infty}^\infty x(u)h(t-u) e^{-i 2 \pi f t} \, du \, dt \hfill (24)$ $\displaystyle = \int_{-\infty}^\infty x(u) \left[ \int_{-\infty}^\infty h(t-u) e^{-i 2 \pi f t} \, dt \right] du \hfill (25)$ $\displaystyle = \int_{-\infty}^\infty x(u) \left[ \int_{-\infty}^\infty h(v) e^{-i 2 \pi f (v + u)} \, dv \right] du \hfill (26)$ $\displaystyle = \int_{-\infty}^\infty x(u) e^{-i 2 \pi f u} \, du \left[\int_{-\infty}^\infty h(v) e^{-i 2 \pi f v} \, dv \right] \hfill (27)$ $\displaystyle = \int_{-\infty}^\infty x(u) e^{-i 2 \pi f u} H(f) \, du \hfill (28)$ $\displaystyle = H(f) \int_{-\infty}^\infty x(u) e^{-i 2 \pi f u}\, du \hfill (29)$

or $\displaystyle Y(f) = H(f) X(f) \hfill (30)$

So, a convolution in the time domain is a multiplication in the frequency domain. A truly difficult-to-evaluate convolution may be quite easy in the frequency domain, provided the individual transforms $H(f)$ and $X(f)$ can be found. Moreover, this basic frequency-domain input-output characterization is consistent with our original encounter with $H(f_0)$ as the complex number that scales the amplitude of an input sine wave with frequency $f_0$: $H(f)$ describes the scaling of each and every sine-wave component of the input.

Because $H(f)$ describes how the linear time-invariant system transfers an arbitrary frequency-domain input to its frequency-domain output, $H(f)$ is also called the transfer function of the filter.

***

Let’s circle back to the meaning of the impulse response. Suppose we have an input described by the Fourier transform $\displaystyle X(f) = 1, \ \ \ \forall f, \hfill (31)$

that is, the transform of the input $x(t)$ is equal to one for all frequencies $f$. Then our frequency-domain input-output relation reduces to $Y(f) = H(f) \hfill (32)$

which means the output of the system is equal to the transfer function. But what time-domain $x(t)$ has $X(f) = 1$ for all $f$? Well, we already encountered this signal, which is the impulse function $\displaystyle x(t) = \delta(t) \hfill (33)$

So, if we manufacture an impulse function, and apply it to our system, we will obtain the impulse response by definition, and this corresponds to an input that simultaneously applies all possible unit-magnitude sine waves to the system. This explains the origin of the formerly common behavior called ‘kicking the tires.’ When inspecting a car, it is stereotypical that the inspector (customer) will give a sharp kick to a tire to see what happens with the car. That is, if something falls off, the tire moves around, or some unsettling noises emit from under the hood. The inspector here is simultaneously applying all sinusoidal forcing functions to the mechanical system, hoping to observe a tell-tale resonance.

### Significance of the Frequency Response in CSP

The frequency response, or transfer function, of a linear time-invariant system comes up in various places throughout the theory of CSP. The most prominent example is when we want to find the spectral correlation function for a random signal that has passed through an LTI system: what is the spectral correlation function for the output as a function of the spectral correlation of the input signal and the transfer function of the filter? As a concrete example, the input signal might be a communication signal and the LTI system might be a multipath propagation channel. We’ve already shown that the result (for the non-conjugate SCF) is $S_y^\alpha(f) = H(f+\alpha/2)H^*(f-\alpha/2) S_x^\alpha (f). \hfill (34)$

See the post on signal processing operations in CSP for more details. 