SPTK: The Characteristic Function – Cyclostationary Signal Processing

Previous SPTK Post: I and Q Next SPTK Post: The Matched Filter

Let’s return to the probability section of the Signal Processing ToolKit posts with a look at the characteristic function, which is the Fourier transform of the probability density function. We will see it has a deep connection to the central mathematical entities of CSP, which are moments and cumulants.

Jump straight to the Significance of the Characteristic Function in CSP.

Basic Definitions

The characteristic function for a real-valued random variable $X$ is the expected value of the function $e^{iuX}$ . The variable $u$ is a dummy variable; it has no intrinsic physical meaning. Recalling that the expected value of a function of a random variable is the integral of the product of that function and the probability density function for the variable, we can write the definition mathematically

$\displaystyle \Phi_X(u) = E\left[e^{iuX} \right] \hfill (1)$

$\displaystyle = \int_{-\infty}^\infty e^{iux} f_X(x) \, dx. \hfill (2)$

Reexpressing this in a more familiar form leads to a Fourier-transform-like expression

$\displaystyle \Phi_X(u) = \int_{-\infty}^\infty f_X(x) e^{-i(-u)x} \, dx \hfill (3)$

$\displaystyle f_X(x) \Longleftrightarrow G_X(u) \hfill (4)$

are a Fourier-transform pair, then the characteristic function is

$\displaystyle \Phi_X(u) = G_X(-u) \hfill (5)$

That is, the characteristic function is the Fourier transform of the random variable’s distribution function, with a change in the sign of the transform’s variable.

Connection to the Moment-Generating Function

The moments of the random variable can be easily obtained from the characteristic function by differentiation. Let’s take it slowly and take the first derivative of $\Phi_X(t)$ with respect to $t$ ,

$\displaystyle \frac{d}{du} \Phi_X(u) = \frac{d}{du} \left[ \int_{-\infty}^\infty e^{iux} f_X(x) \, dx \right] \hfill (6)$

Assuming (with good reason) that we can pass the derivative operation through the integral, we obtain

$\displaystyle \frac{d}{du} \Phi_X(u) = \int_{-\infty}^\infty \left[ \frac{d}{du} e^{iux} \right] f_X(x) \, dx \hfill (7)$

The derivative of the complex exponential is $\displaystyle \frac{d}{du} e^{Au} = A e^{Au}$ , so that the derivative of the characteristic function is simply

$\displaystyle \frac{d}{du} \Phi_X(u) = \int_{-\infty}^\infty e^{iux} \left[ ix f_X(x )\right] \, dx \hfill (8)$

If we evaluate this derivative at $u=0$ , we obtain a scaled version of the first moment of the random variable $X$ ,

$\displaystyle \left. \frac{d}{du} \Phi_X(u)\right|_{u=0} = i \int_{-\infty}^\infty x f_X(x) \, dx = iE[X]. \hfill (9)$

Now let’s look at the second derivative of the characteristic function–simply take the derivative of the first derivative (8),

$\displaystyle \frac{d^2}{du^2} \Phi_X(u) = \int_{-\infty}^\infty \left(\frac{d}{du} e^{iux} \right) \left[ ix f_X(x )\right] \, dx \hfill (10)$

$\displaystyle \frac{d^2}{du^2} \Phi_X(u) = \int_{-\infty}^\infty e^{iux} \left[ i^2 x^2 f_X(x )\right] \, dx \hfill (11)$

which when evaluated at $u=0$ is a scaled version of the second moment of $X$ ,

$\displaystyle \left. \frac{d^2}{du^2} \Phi_X(u)\right|_{u=0} = i^2 \int_{-\infty}^\infty x^2 f_X(x) \, dx = i^2E[X^2]. \hfill (12)$

By induction, we can apprehend the pattern, and prove the general result, for the case of the $n$ th derivative,

$\displaystyle \left. \frac{d^n}{du^n} \Phi_X(u)\right|_{u=0} = i^n \int_{-\infty}^\infty x^n f_X(x) \, dx = i^nE[X^n]. \hfill (13)$

Since $1/i^n = (-i)^n$ , we have the familiar form of the moment-generating function,

$\displaystyle E[X^n] = (-i)^n \left. \frac{d^n}{du^n} \Phi_X(u)\right|_{u=0} = \int_{-\infty}^\infty x^n f_X(x) \, dx. \hfill (14)$

Another way of looking at this is to go back to (2), the definition of the characteristic function, and replace the complex exponential with its Taylor-series representation,

$\displaystyle e^x = \sum_{k=0}^\infty \frac{x^k}{k!} = 1 + \frac{x}{1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + \ldots \hfill (15)$

Replacing $x$ with $iux$ , we have the series representation

$\displaystyle e^{iux} = \sum_{k=0}^\infty \frac{(iux)^k}{k!} = 1 + \frac{iux}{1!} + \frac{(iux)^2}{2!} + \frac{(iux)^3}{3!} + \ldots \hfill (16)$

$\displaystyle = \sum_{k=0}^\infty i^k u^k \frac{x^k}{k!}. \hfill (17)$

Substituting (17) into (2) yields

$\displaystyle \Phi_X(u) = \sum_{k=0}^\infty \frac{i^k}{k!} u^k \left[ \int_{-\infty}^\infty x^k f_X(x) \, dx \right] \hfill (18)$

$\displaystyle = \sum_{k=0}^\infty \frac{i^k}{k!} u^k M_k, \hfill (19)$

which agrees with the previous formulation.

If you can measure the moments of some random variable, then you can estimate the probability density function for that variable by computing the characteristic function through its series representation, and then Fourier transforming the result.

Connection to the Cumulant-Generating Function

Although it is difficult to prove in general (The Literature [R200]), the cumulants of $X$ are the coefficients of the series expansion of the natural logarithm of the characteristic function,

$\displaystyle \ln \left( \Phi_X(u) \right) = \sum_{k=0}^\infty \frac{i^k}{k!} u^k C_k, \hfill (20)$

where $C_k$ is the $k$ th-order cumulant of $X$ .

Thinking back to our various examples of random variables, we recall that the probability density function for a Gaussian random variable $X$ with mean $\mu$ and variance $\sigma^2$ is an exponential raised to a quadratic-in- $u$ polynomial,

$\displaystyle f_X(u) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{-(u \mu)^2/(2\sigma_X^2)}. \hfill (21)$

Because exponentiation and the logarithm are inverse operations,

$\displaystyle \ln \left(e^{g(x)}\right) = g(x), \hfill (22)$

the logarithm of the Gaussian PDF is a quadratic polynomial in $u$ . Therefore the series representation of that logarithm has only a first-order term and a second-order term, and so all cumulants with order three or greater are zero for a Gaussian variable.

Example

The characteristic function can be complicated, since it is the Fourier transform of a probability density function, or a joint probability density function, and such functions themselves can be arbitrarily complicated. Here, though, we wish to illustrate the concepts and the relationships between the various involved functions, so let’s showcase a simple random variable: the binary zero-mean random variable $X$ that takes on values $\{\pm 1\}$ with some probabilities $P(X = -1) = p,$ $P(X=1) = 1-p$ .

The random variable $X$ is involved with our usual go-to example of the rectangular-pulse BPSK signal, and so is a pleasing and useful choice from the point of view of the CSP Blog. For the BPSK signal, we typically take $p= 0.5$ .

Viewing $X$ as a continuous–rather than discrete–random variable, the probability density function is impulsive,

$f_X(u) = p\delta(u+1) + (1-p)\delta(u-1) = \frac{1}{2} \left[ \delta(u+1) + \delta(u-1) \right] \hfill (23)$

The characteristic function is the Fourier transform of this density function, which is easy for us to evaluate,

$\displaystyle \Phi_X(u) = E\left[ e^{iuX} \right] \hfill (24)$

$\displaystyle = \int_{-\infty}^\infty e^{iux} f_X(x) \, dx \hfill (25)$

$\displaystyle = \int_{-\infty}^\infty \left[ \frac{1}{2} \left( \delta(x-1) + \delta(x+1) \right) \right] e^{iux} \, dx \hfill (26)$

$\displaystyle = \frac{1}{2} \int_{-\infty}^\infty \left( \delta(x-1)e^{iu} + \delta(x+1)e^{-iu} \right) \, dx \hfill (26)$

$\displaystyle = \frac{1}{2} \left[ e^{iu} + e^{-iu} \right] = \cos(u) \hfill (27)$

Result (27) shouldn’t surprise us much if we recall the form of the Fourier transform of $\sin$ and $\cos$ .

For our simple binary random variable $X$ , the characteristic function is a simple one: $\cos(u)$ . We can find the series expansion of $\cos(u)$ by following the general Taylor series formula, which involves finding all the derivatives of $\cos(u)$ , we can look it up, or perhaps some of us just remember it:

$\displaystyle \Phi_X(u) = \cos(u) = \sum_{k=0}^\infty \frac{(-1)^k}{(2k)!} u^{2k} \hfill (28)$

$= 1 - \frac{1}{2!}u^2 + \frac{1}{4!}u^4 - \frac{1}{6!}u^6 + \ldots \hfill (29)$

Note that we can express $(-1)^k$ in terms of $i = \sqrt{-1}$ by

$(-1)^k = (i^2)^k = i^{2k}. \hfill (30)$

Our series expansion for the characteristic function for $X$ is then

$\displaystyle \Phi_X(u) = \sum_{k=0}^\infty \frac{i^{2k}}{(2k)!} u^{2k} \hfill (31)$

which when compared to (19) reveals that the moment $M_k$ is equal to one for even $k$ and is zero otherwise. All the even moments of the symmetric binary random variable are equal to one. How can we check this? By directly computing the moments using expectation.

Let’s write the expression for the $n$ th-order moment for $X$ ,

$\displaystyle M_n = E[X^n] = \int_{-\infty}^\infty x^n \left[ \frac{1}{2} \left(\delta(x-1) + \delta(x+1)\right) \right] \, dx \hfill (32)$

$\displaystyle = \frac{1}{2} \int_{-\infty}^\infty (1)^n\delta(x-1) + (-1)^n\delta(x+1) \, dx \hfill (33)$

$\displaystyle = \frac{1}{2} \left( 1 + (-1)^n \right) \hfill (34)$

$\displaystyle = \left\{ \begin{array}{ll} 1, & n \mbox{\rm \ even} \\ 0, & n \mbox{\rm \ odd}, \end{array} \right. \hfill (35)$

which agrees with the series-expansion approach.

Turning to the cumulants of $X$ , we recall that these were provided in Table 1 of the post on PSK and digital QAM, up to an order of ten. The odd-order cumulants, like the moments, are zero, and the first five even-order cumulants are 1, -2, 16, -272, 7936.

Can we compute the logarithm of the characteristic function for $X$ as a series, thereby allowing us to confirm these cumulants? Let’s try. The logarithm of the characteristic function is

$\displaystyle \ln \Phi_X(u) = \ln\left(\cos(u)\right) \hfill (36)$

To find the series expansion, we can try to find the derivatives of this expression. The first derivative is

$\displaystyle \frac{d}{du} \ln \Phi_X(u) = \frac{1}{\cos(u)} \left(\frac{d}{du} \cos(u) \right) \hfill (37)$

$\displaystyle = \frac{-\sin(u)}{\cos(u)} = -\tan(u) \hfill(38)$

Recall from (20) that the $n$ th derivative of the logarithm of the characteristics function, evaluated at $u=0$ is a scaled version of the $n$ th cumulant $C_n$ ,

$\displaystyle \left. \frac{d^n}{du^n} \ln \Phi_X(u) \right|_{u=0} = i^n C_n \hfill (39)$

The first-order cumulant (mean) is equal to $i$ times the first derivative evaluated at $u=0$ , which is $i$ times $-\tan(0) = 0$ , or zero. Which checks out, since it is easy to directly compute the mean using the PDF–for the symmetric binary variable $X$ with $p=0.5$ , the mean is indeed zero.

The second derivative, evaluated at $u=0$ and scaled by $i^2 = -1$ , is given by the derivative of the first derivative (38) or

$\displaystyle \frac{d^2}{du^2} \ln \Phi_X(u) = \frac{d}{du} \left( -\tan(u)\right) = -\frac{d}{du} \tan(u) \hfill (40)$

Here we can substitute a known series expansion for $\tan(u)$ to facilitate the differentiation,

$\displaystyle \tan(u) = \sum_{n=1}^\infty \frac{B_{2n} (-4)^n (1-4^n)}{(2n)!} u^{2n-1} \hfill (41)$

where $B_n$ is a Bernoulli number. Now we need to take the first derivative of (41) to find the second derivative of the logarithm of the characteristic function, which will lead to the second moment when we scale and evaluate at $u=0$ .

The lowest-degree power of $u$ in (41) corresponds to $n=1$ , which is $u^1$ , which will be the only term that is nonzero after setting $u=0$ ,

$\displaystyle -\left. \frac{d}{du} \tan(u)\right|_{u=0} = -\frac{B_2 (-4)^1 (1-4^1)}{2!}(2(1)-1) = i^2 C_2 \hfill (42)$

Now table lookup tells us that $B_2 = 1/6$ so that our second-order cumulant is given by

$C_2 = \frac{(1/6)(-4)(-3)}{2} = 12/12 = 1, \hfill (43)$

which checks!

Now we need the third derivative of the logarithm of the characteristic function, which is the second derivative of (41). Since $u$ appears in (41) only for odd orders $2n-1$ , the second derivative of (41) evaluated at $u=0$ will be zero. This will be true for all even-order derivatives of (41), and therefore all odd-order derivatives of the logarithm of the characteristic function will be zero (which checks).

On to the fourth derivative of the logarithm of the characteristic function. The third derivative of (41) evaluated at $u=0$ will pick off the term associated with $u^{2(n)-1} = u^3$ , which corresponds to $n=2$ . We have

$\displaystyle -\left. \frac{d^3}{du^3} \tan(u)\right|_{u=0} = -\frac{B_4 (-4)^2 (1-4^2)}{4!}(2n-1)! = i^4 C_4 = C_4 \hfill (44)$

$\displaystyle = -\frac{(-1/30) (-4)^2 (1-4^2)}{4!}(2(2)-1)! \hfill (45)$

$\displaystyle = - \frac{(16)(-15)}{(24)(30)}(6) = -\frac{48}{24} = -2 = C_4 \hfill (46)$

which checks again.

The pattern should be clear now. $C_5$ is zero, and to get $C_6$ , we’ll need the fifth derivative of (41) evaluated at $u=0$ , which will pick off the term corresponding to $n=3$ ( $u^{2(3)-1} = u^5$ ). This leads to

$\displaystyle \left. \frac{d^6}{du^6} \ln \Phi_X(u)\right|_{u=0} = i^6 C_6 \hfill (47)$

$\displaystyle = -\frac{B_6(-4)^3 (1-4^3)(2(3)-1)!}{(2n)!} \hfill (48)$

$\displaystyle = -\frac{(1/42)(-64)(-63)(5!)}{6!} = -\frac{2^6 (7)(9)}{2(3)(2)(7)(3)} = -\frac{64}{4} = -16\hfill (49)$

and with $i^6 = -1$ , we obtain $C_6 = 16$ , as expected.

The eighth-order cumulant checks as well (-272), but I didn’t do the arithmetic for the tenth-order cumulant.

The Significance of the Characteristic Function in CSP

The characteristic function isn’t a function that I use much on the CSP Blog nor in my day-to-day signal-processing work. It isn’t quite as fundamental as the cumulative distribution function, the probability density function, expectation, moments, or cumulants. However, it plays a significant role in the theory connecting $n$ th-order moments to $n$ th-order cumulants.

In CSP, most of our focus is on first- and second-order statistics and probabilistic parameters, such as the time-varying mean and the time-varying second moment (or variance when the mean is zero). And so we dwell on the cyclic autocorrelation and the spectral correlation function.

However, when we do employ higher-order statistics in CSP, it is almost always a cyclic cumulant, which is a Fourier-series component of the general periodically time-variant cumulant for a cyclostationary signal or process. Since the logarithm of the characteristic function has a series representation that explicitly involves cumulants, the characteristic function is a key part of the theory–if not the practice–of cyclostationary signals.

Complex Variables and Processes

You may have noticed that I focused on real-valued random variables in this SPTK post, whereas in our SP and CSP work, we almost always focus on complex-valued random variables, signals, and processes. There are some complications in extending the usual characteristic function to complex random variables. I take this up in My Papers [6] and you can also read about it on Wikipedia. The conclusion is that we can use the Shiryaev-Leonov moment-to-cumulant relation (The Literature [R200]) to form the cumulants of a complex random variable from its moments. Of course we need to be very careful about how we apply conjugations, but we’ve covered that in some detail here on the CSP Blog already.

Previous SPTK Post: I and Q Next SPTK Post: The Matched Filter

Author: Chad Spooner

I'm a signal processing researcher specializing in cyclostationary signal processing (CSP) for communication signals. I hope to use this blog to help others with their cyclo-projects and to learn more about how CSP is being used and extended worldwide. View all posts by Chad Spooner

2 thoughts on “SPTK: The Characteristic Function”

Lauritz says:

September 11, 2025 at 4:23 am

Hi Dr. Spooner – I *love* your blog! It’s like a living, growing CSP encyclopedia. Thank you for all the time and effort you put into it and for keeping it freely available to all. One quick typo (which admittedly doesn’t change the end result) is in equation (16) where the expansion on the RHS is still just in terms of $x$ rather than $iux$ . Looks like probably just a copy-paste-forgot to update kind of thing. Thanks again.

Loading...

1. Chad Spooner says:
  
  September 11, 2025 at 6:58 am
  
  Welcome to the CSP Blog Lauritz! And thanks for the comment.
  
  I’ve fixed (16). Let me know if you find further problems.
  
  If you want to use Latex to typeset some symbols or equations in a comment, use the dollar-sign delimiters, but just after the left dollar sign, add the keyword latex. I’ve edited your comment, and now the two symbols you included should appear as Latex-formatted math.
  
  Finally, if you want to pitch in to help keep the Blog ad-free, please donate using the Donate button on the right side of any CSP Blog page. Strictly optional of course! But I do pay WordPress to keep ads off, and I pay for dataset and media storage on their servers.
  
  Loading...