SPTK: Interconnection of Linear Systems – Cyclostationary Signal Processing

Previous Post: Frequency Response Next Post: Convolution

It is often the case that linear time-invariant (or for discrete-time systems, linear shift-invariant) systems are connected together in various ways, so that the output of one may be the input to another, or two or more systems may share the same input. In such cases we can often find an equivalent system impulse response that takes into account all the component systems. In this post we focus on the serial and parallel connections of LTI systems in both the time and frequency domains. Much more complex interconnections can be constructed from these two basic kinds of connections.

[Jump straight to ‘Significance of Connected LTI Systems in CSP’ below.]

Time-Domain Analysis

Let’s first look at a serial connection of two systems, then we’ll move on to a parallel connection. A serial connection of two systems just means that the output of the first system forms the input to the second system.

Figure 1. The serial connection of two linear time-invariant systems. Here the systems are denoted by their impulse-response functions $h_j(t)$ . The signal $x(t)$ is the input to the first system, $y(t)$ is the output of the first system and the input to the second system, and $z(t)$ is the output of the second system.

Figure 1. The serial connection of two linear time-invariant systems. Here the systems are denoted by their impulse-response functions $h_j(t)$ . The signal $x(t)$ is the input to the first system, $y(t)$ is the output of the first system and the input to the second system, and $z(t)$ is the output of the second system.

Let’s go through the mathematics that describes an equivalent system for the serially connected systems in Figure 1. As we progress we’ll present results with a bit less detail, but for this first example of interconnected systems, we’ll look at all the details.

The basic mathematical problem is to find an expression for the response $z(t)$ in terms of the input $x(t)$ and the two impulse response functions $h_j(t)$ . Using what we’ve already established about linear time-invariant systems in the Signal Processing ToolKit posts, we can write the following two expressions

$\displaystyle y(t) = h_1(t) \otimes x(t), \hfill (1)$

and

$\displaystyle z(t) = h_2(t) \otimes y(t), \hfill (2)$

where $\otimes$ denotes convolution. We are looking for an expression that does not explicitly contain $y(t)$ . Let’s do the obvious and substitute $y(t)$ from (1) into (2).

$\displaystyle z(t) = \int_{-\infty}^\infty y(v) h_2(t-v)\, dv \hfill (3)$

$\displaystyle = \int_{-\infty}^\infty \left[ \int_{-\infty}^\infty x(u) h_1(v-u)\, du \right] h_2(t-v)\, dv \hfill (4)$

$\displaystyle = \int_{-\infty}^\infty \int_{-\infty}^\infty x(u) h_1(v-u) h_2(t-v)\, du dv \hfill (5)$

$\displaystyle = \int_{-\infty}^\infty \int_{-\infty}^\infty x(u) h_1(v^\prime) h_2(t-v^\prime -u)\, dv^\prime du \hfill (6)$

$\displaystyle = \int_{-\infty}^\infty x(u) \left[ \int_{-\infty}^\infty h_1(v^\prime) h_2((t-u) - v^\prime) \, dv^\prime \right] \, du. \hfill (7)$

and we notice that the inner integral is the function

$\displaystyle g(t^\prime) = h_1(t^\prime) \otimes h_2(t^\prime)$

evaluated at $t^\prime = t-u$ . So our expression becomes

$\displaystyle z(t) = \int_{-\infty}^\infty x(u) g(t-u) \, du \hfill (8)$

$\displaystyle = x(t) \otimes g(t), \hfill (9)$

where

$\displaystyle g(t) = h_1(t) \otimes h_2(t). \hfill (10)$

The equivalent system has an impulse-response function that is the convolution of the two individual impulse-response functions $h_1(t)$ and $h_2(t)$ , as illustrated in Figure 2.

Figure 2. Equivalent linear time-invariant system for the serial connection of the two systems shown in Figure 1.

We can use induction to prove that the serial connection of $N$ LTI systems has an impulse-response function that is equal to the convolution of the $N$ component-system impulse-response functions, as shown in Figure 3.

Figure 3. The serial connection of $N$ LTI systems is equivalent to a single LTI system with impulse response given by the convolution of the $N$ individual impulse-response functions.

Now let’s turn to the parallel connection of two linear time-invariant systems, as depicted in Figure 4.

Figure 4. The parallel connection of two linear time-invariant systems with impulse-response functions $h_1(t)$ and $h_2(t)$ . The input is $x(t)$ , and the output is $z(t) = y_1(t) + y_2(t)$ .

Figure 4. The parallel connection of two linear time-invariant systems with impulse-response functions $h_1(t)$ and $h_2(t)$ . The input is $x(t)$ , and the output is $z(t) = y_1(t) + y_2(t)$ .

The math is simpler here:

$\displaystyle y_1(t) = \int_{-\infty}^\infty x(u) h_1(t-u) \, du \hfill (11)$

$\displaystyle y_2(t) = \int_{-\infty}^\infty x(u) h_2(t-u) \, du \hfill (12)$

$\displaystyle z(t) = y_1(t) + y_2(t). \hfill (13)$

The equivalent system is found by adding $y_1(t)$ to $y_2(t)$ ,

$\displaystyle z(t) = \int_{-\infty}^\infty x(u) h_1(t-u) \, du + \int_{-\infty}^\infty x(u) h_2(t-u) \, du \hfill (14)$

$\displaystyle = \int_{-\infty}^\infty x(u) \left[ h_1(t-u) + h_2(t-u) \right] \, du \hfill (15)$

$\displaystyle = \int_{-\infty}^\infty x(u) g(t-u) \, du, \hfill (16)$

where

$\displaystyle g(t) = h_1(t) + h_2(t). \hfill (17)$

So the equivalent system for a parallel connection is simply the sum of the two component systems’ impulse response functions. For the parallel connection of $N$ linear time-invariant systems $h_k(t)$ , we have the equivalent impulse response

$\displaystyle g(t) = \sum_{k=1}^N h_k (t), \hfill (18)$

which is illustrated in Figure 5.

Figure 5. The equivalent system for the parallel connection of $N$ linear time-invariant systems has an impulse response that is the sum of the $N$ impulse-response functions for the component systems.

The equivalent systems for serial and parallel connections of discrete-time linear shift-invariant systems follow the same rules as those above for continuous-time linear time-invariant systems. The mathematics is nearly identical, with convolution integrals replaced by convolution sums.

Frequency-Domain Analysis

We can gain more insight into the nature of the output of an interconnected set of systems if we look at the input-output relations in the frequency domain. Starting again with the serial connection of two linear time-invariant systems (Figure 1), we have the temporal input-output relation

$\displaystyle z(t) = h_1(t) \otimes h_2(t) \otimes x(t), \hfill (19)$

where $\otimes$ denotes convolution. We’ve already established in the Signal Processing Toolkit that the Fourier transform of a signal that is equal to the convolution of two functions is the product of the Fourier transforms of those two functions: convolution in the time domain is multiplication in the frequency domain. This means that (19) can be easily transformed to yield the input-output relationship for the serial connection of two linear time-invariant systems given by

$\displaystyle Z(f) = H_1(f)H_2(f)X(f) = G(f)X(f), \hfill (20)$

which is illustrated in Figure 6.

Figure 6. The equivalent transfer function for the serial connection of two LTI systems with transfer functions $H_1(f)$ and $H_2(f)$ is their product $H_1(f)H_2(f)$ .

Using the result shown in Figure 3, we find that the equivalent transfer function for the serial connection of $N$ LTI systems with transfer functions $H_k(f)$ is simply their product,

$\displaystyle G(f) = \prod_{k=1}^N H_k(f), \hfill (21)$

as illustrated in Figure 7.

Figure 7. The transfer function of the equivalent system for the serial connection of $N$ linear time-invariant systems is the product of the $N$ individual transfer functions.

It should now come as no surprise (Equation (18)) that the parallel connection of $N$ linear time-invariant systems has an equivalent transfer function that is simply the sum of the individual-system transfer functions, as shown in Figure 8.

Figure 8. The transfer function of the equivalent system for the parallel connection of $N$ linear time-invariant systems is the sum of the individual transfer functions.

Interpretations

Serial ConnECTiON

A linear time-invariant system is completely characterized (in terms of determining the output for any input) by its impulse-response function. The Fourier transform of the impulse-response function also, therefore, completely characterizes the system, and that function is called the transfer function or frequency response. The output $Y(f)$ is related to the input $X(f)$ through multiplication by the transfer function,

$\displaystyle Y(f) = H(f) X(f). \hfill (22)$

So a spectral component of the input, $X(f_0)$ , is scaled by the transfer function evaluated at the frequency of the spectral component, $H(f_0)$ , to produce the spectral component at the output $Y(f_0)$ . Since $H(f_0)$ can be a complex number, both the magnitude and phase of the spectral component in $x(t)$ can be modified by the system, delivering (‘transferring’) the spectral component $H(f_0)X(f_0)$ to $y(t)$ .

In the case of the serial connection of $N$ linear time-invariant systems, the scaling of the input spectral component $X(f_0)$ is simply the multiplication of all the individual scaling functions from the connected systems, or $\displaystyle \prod_{k=1}^N H_k(f_0)$ . To annihilate the spectral component at $f=f_0$ in the output, for example, all that is required is that at least one of the component systems has a transfer function that is zero at $f_0$ .

Parallel Connection

In the case of the parallel connection of $N$ linear time-invariant systems, the equivalent impulse-response function is the sum of the component-system impulse-response functions, and so the equivalent transfer function is the sum of the component-system transfer functions. The spectral component in the input $x(t)$ with frequency $f_0$ is scaled by the sum of the component-system transfer functions, or $\displaystyle \sum_{k=1}^N H_k(f_0)$ . To annihilate a spectral component of the input so that it does not appear in the output requires that this sum be zero. A sufficient, but not necessary, way to do that is to make sure each $H_k(f_0)$ is zero.

Examples

Bandpass Filter

A bandpass filter is a linear time-invariant system that has a transfer function that is zero (or very small) for all input frequencies $f$ except those in the two intervals

$\displaystyle [-f_{h}, -f_{l}]$

and

$\displaystyle [f_l, f_h],$

where $f_h > f_l$ . Only a contiguous subset, or band, of frequencies is passed through the system, not all frequencies. We can construct a bandpass filter from a lowpass filter and a highpass filter in serial connection. A lowpass filter passes all frequencies in the interval $[-f_{lpf}, f_{lpf}]$ and a highpass filter passes all frequencies in the two intervals

$\displaystyle [-\infty, -f_{hpf}]$

and

$\displaystyle [f_{hpf}, \infty].$

Consider the serial connection of a lowpass filter $h_3(t)$ and a highpass filter $h_4(t)$ shown in Figure 9. The maximum frequency that is passed by the lowpass filter $h_3(t)$ (the `cutoff frequency’) is $f_3$ , and the minimum frequency that is passed by the highpass filter $h_4(t)$ (its cutoff frequency) is $f_4$ . Crucially, $f_3 > f_4$ . When we connect these two linear time-invariant systems serially, we multiply their transfer functions, obtaining the transfer function $G(f) = H_3(f)H_4(f)$ shown in the figure. Here, $f_l = f_4$ and $f_h = f_3$ .

Figure 9. A bandpass filter can be created by the serial concatenation of a lowpass filter and a highpass filter, provided the cutoff frequency of the lowpass filter is greater than the cutoff frequency of the highpass filter.

Notch Filter

The inverse of a bandpass filter is called a notch filter or bandstop filter. Such a linear time-invariant system passes all frequencies except those in the two intervals

$\displaystyle [-f_{h}, -f_{l}]$

and

$\displaystyle [f_l, f_h].$

The notch filter can be created by the parallel connection of a suitable lowpass filter with a highpass filter, as shown in Figure 10. Here we must have $f_1 < f_2$ . The equivalent system is the sum of the two transfer functions, or $G(f) = H_1(f) + H_2(f)$ .

Figure 10. A notch filter can be created by the parallel connection of a lowpass filter and a highpass filter, provided the cutoff frequency for the lowpass filter is smaller than the cutoff frequency for the highpass filter.

Complex Arrangement of Filters

In this final example of the interconnection of linear time-invariant systems, consider the arrangement of seven different systems shown in Figure 11. Just think if we had to figure out the equivalent system by dealing with the time-domain convolutions. It would be a mess of integrals.

Instead, let’s look at it through the frequency-domain lens of the transfer function.

$\displaystyle Y_1(f) = X(f) H_1(f)H_2(f) \hfill (23)$

$\displaystyle Y_2(f) = [H_3(f) + H_4(f)]Y_1(f) \hfill (24)$

$\displaystyle \Rightarrow Y_2(f) = X(f) \left[ H_1(f)H_2(f)\left[ H_3(f) H_4(f)\right] \right] \hfill (25)$

$\displaystyle Y_3(f) = H_1(f)H_5(f)H_6(f) \hfill (26)$

$\displaystyle Y_4(f) = Y_3(f) + X(f)H_1(f)H_7(f) \hfill (27)$

$\displaystyle \Rightarrow Y_4(f) = X(f)H_1(f)H_5(f)H_6(f) + X(f)H_1(f)H_7(f) \hfill (28)$

$\displaystyle Z(f) = Y_4(f) + Y_2(f) \hfill (29)$

$\displaystyle = X(f) \left( H_1(f) H_2(f) [H_3(f) + H_4(f)] \right)$

$\displaystyle + X(f) \left( H_1(f) H_5(f) H_6(f) + H_1(f)H_7(f) \right) \hfill (30)$

$\displaystyle \Rightarrow G(f) = Z(f)/X(f) = H_1(f) \left[ H_2(f)(H_3(f) + H_4(f)) + H_5(f) H_6(f) + H_7(f) \right]. \hfill (31)$

Figure 11. A complex arrangement of linear time-invariant systems (filters). The equivalent system is most easily characterized in terms of its transfer function, which is a relatively simple function of the transfer functions of the connected systems.

Significance of Connected LTI Systems in CSP

CSP is mostly about nonlinear, as opposed to linear, mathematical operations, so that the role of LTI systems and their interconnected versions mostly has to do with their effects on a signal prior to application of CSP. A canonical example involves a pulse-amplitude-modulated signal (such as PSK and QAM), with a pulse-shaping LTI system (filter) $P(f)$ , a propagation-channel LTI system $H(f)$ , and a reception LTI system $R(f)$ . These systems are connected in series (the output of one forms the input of the next), so that the equivalent LTI system that connects the modulated impulse train at the transmitter to the received signal is $G(f) = P(f)H(f)R(f)$ . This equivalent system can then be used to determine the spectral correlation function or the cyclic cumulants.

Previous Post: Frequency Response Next Post: Convolution

Author: Chad Spooner

I'm a signal processing researcher specializing in cyclostationary signal processing (CSP) for communication signals. I hope to use this blog to help others with their cyclo-projects and to learn more about how CSP is being used and extended worldwide. View all posts by Chad Spooner

Chad Spooner on Automatic Spectral SegmentationNovember 21, 2023
The value of $latex S_{min}$ is just the minimum value of the PSD estimate, excluding the edges if using the…
Maggiee on Automatic Spectral SegmentationNovember 21, 2023
Hi Chad! As per your suggestion, I was browsing through your paper “Multi-Resolution White-Space Detection for Cognitive Radio” and this…
John Snoap on The Next Logical Step in CSP+ML for Modulation Recognition: Snoap’s MILCOM ’23 Paper [Preview]November 18, 2023
Hi Mehdi, Thanks for the kind words and the good questions! The dimensions listed in the Activations column actually refer…
John Snoap on Introducing Dr. John A. SnoapNovember 17, 2023
Thank you, Chad!!! It has definitely been quite the journey learning everything. I could not have done it without you…
Mansoor on Introducing Dr. John A. SnoapNovember 17, 2023
Congrats to John. Who knew you could make great friends through a blog?
Michael on Introducing Dr. John A. SnoapNovember 17, 2023
Always exciting to see someone blending ML and DSP! Will there be a public release of the dissertation?
Martin Ryba on Introducing Dr. John A. SnoapNovember 17, 2023
Is his dissertation available online anywhere?
Mehdi Akeddar on The Next Logical Step in CSP+ML for Modulation Recognition: Snoap’s MILCOM ’23 Paper [Preview]October 20, 2023
Hello Chad, Thanks for the amazing work and Congratulation for the paper ! After some reading, I had some questions…
Chad Spooner on SPTK: The Z TransformOctober 16, 2023
Yes indeed! I've fixed the captions. Your reference is my The Literature [R192], which I cite in the post. Thanks…
Todd Reinking on SPTK: The Z TransformOctober 16, 2023
Chad, aren't Figure 7 and 8 labeled incorrectly? Each represents an analysis and synthesis MDFT filterbank, but the second one…