Previous SPTK Post: The Fourier Transform Next SPTK Post: Frequency Response
In this Signal Processing Toolkit post, we’ll take a first look at arguably the most important class of system models: linear time-invariant (LTI) systems.
What do signal processors and engineers mean by system? Most generally, a system is a rule or mapping that associates one or more input signals to one or more output signals. As we did with signals, we discuss here various useful dichotomies that break up the set of all systems into different subsets with important properties–important to mathematical analysis as well as to design and implementation. Then we’ll look at time-domain input/output relationships for linear systems. In a future post we’ll look at the properties of linear systems in the frequency domain.
[Jump straight to ‘Significance of Linear Systems in CSP’ below.]
Types of Systems
Linear versus Nonlinear
Let’s use a generic notation at first. Suppose we have a system with a single input and a single output . The system is the mathematical model for the mapping from to which we will denote by :
If the system is linear, then the output for a sum of inputs is the sum of the outputs:
which applies to all inputs and for which the system output is defined.
Let . Then we immediately have
which generalizes easily to
and to the most common statement of system linearity
If (7) is not obeyed by the system for all valid inputs and all complex numbers and , the system is not linear.
Time-Varying versus Time-Invariant
A system is time-invariant if the shape of the response to input is the same as that for , but shifted in time. That is
Stable versus Unstable
A system is bounded-input/bounded-output (BIBO) stable if the output signal is bounded for every bounded input. In other words, if is the system input, then
where both and are finite non-negative numbers.
Causal versus Noncausal
For causal systems, the output of the system is always zero prior to the application of the input. If for , the the output for the input is also zero for :
Because of the arrow of time, physical systems have to be causal, but mathematical models of systems do not. In practice, this means that if you develop a mathematical system design and it lacks causality, you have to do some more work before the system can be implemented with a physical device (often called realizing the system).
We will also use the alternative notation to denote action of a system on an input:
Linear Time-Invariant Systems: Filters
Recall our signal representation that involved an infinite set of shifted and weighted impulse (or delta) functions:
What is the output of a linear, time-invariant system for that can be represented by (13)?
Let’s start off by considering a simpler input than the general one in (13):
which is a finite sum of shifted and scaled impulses. If our system is linear, then the output (also called the response) to input (also called the excitation) is the sum of the outputs for the shifted and scaled impulses. Let the response to an impulse at be denoted by . Then by linearity,
Now if the linear system is also time-invariant, then the forms (shapes) of the responses are identical–they are merely time-shifted versions of each other:
where is the system’s response to an impulse at . This particular response is called the system impulse response, or just impulse response, and is often denoted by :
We see that if we know the response to a single delta function at , we can find the total system response to any input that is the sum of time-shifted and scaled impulses.
Turning back to our representation (13) for general , we have a continuum of impulses, each one with strength and temporal location . By linearity and time-invariance, we have the response given by
Rearranging this result into a more conventional format yields
This particular integral is called a convolution integral, or just a convolution. It is the mathematical tool that describes the output of a linear, time-invariant system for an arbitrary input. Linear time-invariant systems are also commonly called filters in engineering. The key thing to understand is that the output for any input can be computed by knowing only the input and the response of the system to an impulse applied at time .
For discrete-time signals and systems, the situation is similar. Linearity is the same, and time-invariance is replaced by shift-invariance, since time is quantized into discrete chunks or shifts.
An arbitrary discrete-time signal can be represented by the sum of scaled and shifted discrete-time impulses,
where the impulse is defined by
Let a discrete-time system be linear and shift-invariant, and denote the response to a discrete-time impulse at by . Then
which is called a convolution sum, or just convolution.
We’ll look in detail at convolution integrals and sums in another Signal Processing ToolKit post. Let’s finish this post by assessing some of the generic system properties in the specific context of linear time-invariant systems.
System Properties for LTI and LSI Systems
Causality implies that the output of a system cannot anticipate the input-it has to wait to respond until, at the earliest, the instant the input is applied. Let’s see what this means for LTI systems.
The general expression for the output of an LTI system with input and impulse response is
By using the change of variables , this convolution can be re-expressed as
Either form may be used, and one or the other may be more convenient in a particular setting.
Suppose the input is zero until the time instant . Causality says that the output must also be zero for all . Let’s impose this condition on in the computation of the convolution integral:
Because is zero for ,
Now for , we require to be zero, and for this range of , the upper limit for on the integral of (27) ranges from to zero. Therefore, to enforce causality for any imaginable , we require that the impulse-response function be zero for . In this case, the output of the system is
So causality requires the impulse response to be zero for , which makes sense because after all the impulse response is the system’s response to an impulse at . The same requirement for causality is levied on the discrete-time impulse-response function .
As we mentioned in the first part of this post, a system is bounded-input/bounded-output (BIBO) stable if every bounded input produces a bounded output. So for a bounded input , there is some finite number such that
Another way of saying this is
Assuming we have a linear time-invariant system with impulse-response function and output , what is the condition on for BIBO stability?
By elementary calculus, the magnitude of an integral is less than or equal to the integral of the integrand magnitude,
For to be bounded, then, we require
Equation (39) means that the impulse response is absolutely integrable. For discrete-time systems, the impulse-response function must be absolutely summable.
For a BIBO stable and causal linear time-invariant system, the requirement is
and a similar condition holds for the discrete-time impulse response .
For LTI systems, the memory of the system is the length of the non-zero portion of the impulse response. If there is some finite number for which for all , then the system is said to have finite memory.
In discrete time, a finite-memory linear shift-invariant system is also called a finite impulse response (FIR) filter. Otherwise, it is an infinite impulse response (IIR) filter. You can use MATLAB’s filter.m to implement FIR and IIR filters. Note that FIR filters are BIBO stable, IIR filters may or may not be. Why?
Significance of Linear Systems in CSP
We haven’t yet talked about the frequency-domain representation of a linear system (hint: it is the Fourier transform of the impulse-response function), so connecting LTI systems to CSP is a bit awkward at this point. However, LTI system models are prevalent in the modeling of communication-signal channels (for example, multipath propagation), and so we need to be able to understand how an LTI system that is applied to a signal affects the various CSP parameters of the filtered signal. We’ll be able to describe this mathematically more easily after we look at the frequency response of LTI systems. (For the impatient, go ahead and look at the post on signal processing operations Signal Processing Operations and CSP.)
When we talk about the spectral correlation function, we often speak of the temporal correlation between two “narrowband spectral components” of the signal. These narrowband spectral components are actually the outputs of two LTI systems (bandpass filters). We’ll get to the notions of lowpass, highpass, and bandpass filters in subsequent SPTK posts.
One application of CSP is source separation, in which the processed data contains two or more signals that overlap in both time and frequency, and the goal is to extract one or more of the signals from the noisy sum. A kind of signal separator that exploits CSP is known as the frequency-shift (FRESH) filter, which is a linear system, but it is not time-invariant. A FRESH filter is a multi-branch system where each branch consists of a frequency shifter followed by a FIR filter. We’ll eventualy have a post on FRESH filters at the CSP Blog.
Previous SPTK Post: The Fourier Transform Next SPTK Post: Frequency Response