The two non-parametric spectral-correlation estimators we’ve looked at so far–the frequency-smoothing and time-smoothing methods–require the choice of key estimator parameters. These are the total duration of the processed data block, , and the spectral resolution .

For the frequency-smoothing method (FSM), an FFT with length equal to the data-block length is required, and the spectral resolution is equal to the width of the smoothing function . For the time-smoothing method (TSM), multiple FFTs with lengths are required, and the frequency resolution is (in normalized frequency units).

The choice for the block length is partially guided by practical concerns, such as computational cost and whether the signal is persistent or transient in nature, and partially by the desire to obtain a reliable (low-variance) spectral correlation estimate. The choice for the frequency (spectral) resolution is typically guided by the desire for a reliable estimate.

The reliability of the estimate is inversely proportional to the time-bandwidth product . Note that this is the time-bandwidth product of the measurement, not of the signal(s) in the data. This result has been established in [R1] (see Chapter 15, Eq (87a)). In that work, the coefficient of variation of a point estimate of the spectral correlation function is shown to be inversely proportional to the time-bandwidth product

where is a constant and the spectral correlation estimate is for frequency and cycle frequency . Here the function is the spectral coherence for the data at spectral frequency and cycle frequency .

The (non-conjugate) coherence is defined by

where is the PSD. When , we have

and when is white Gaussian noise, , the noise spectral density value. Moreover, for ,

Suppose . Then and

So the coefficient of variation is also inversely proportional to the square of an SNR measure. As this SNR decreases, the coefficient of variation increases. To regain a smaller coefficient of variation, the resolution product must then be increased.

A simpler result for is well-known in the spectrum analysis community. For example, reference [R43] asserts that the coefficient of variation for a power spectrum estimate (spectral frequency , cycle frequency ) is inversely proportional to (see Section 5.4).

### Interpretation

These basic approximate results tell us that if we want to improve the reliability (reduce the variance) of a measurement, we need to increase the data-block length, increase the spectral resolution (make it coarser), or both. It also tells us that we cannot simultaneously have very fine spectral resolution (small ) and low estimate variability (small ) unless the block length is made very large.

### Numerical Examples

As an illustration, we consider the rectangular-pulse BPSK signal (of course). First we produce a sequence of FSM spectral correlation estimates for only those cycle frequencies exhibited by the signal. The block length is fixed at samples. The resolution product is varied by varying the width of the rectangular spectral smoothing window . The spectral correlation estimates are plotted as surfaces above the plane and collected in a movie:

When the resolution product is small, the spectral correlation features are difficult to make out due to their highly erratic appearance. As the product increases, the features begin to look like the ideal ones. However, as it continues to increase, the spectral resolution of the measurement becomes larger than the spectral widths of the features (peaks and valleys) in the spectral correlation function, causing the features to become smeared in appearance. At this extreme of the resolution product, the estimates are highly reliable (low variance) but are deterministically distorted (high bias).

The reliability of the spectral correlation estimate is especially important when the cycle frequencies of interest are not known in advance of estimation. In such a case, a cycle-frequency search must be performed. The FSM and TSM are not particularly efficient for this purpose (we will cover an efficient method, the strip spectral correlation analyzer, in a future post), but they are sufficient to illustrate the resolution product effect here.

The FSM was used to estimate the spectral correlation and spectral coherence functions for each unique cycle frequency in the principal domain using a block length of samples and various spectral resolution widths . The cycle-frequency resolution of a spectral correlation measurement is approximately the reciprocal of the block length . So the set of visited cycle frequencies has adjacent-cycle-frequency spacing of . That’s a lot of cycle frequencies. The resulting sequence of spectral correlation estimate magnitudes is shown in the following movie:

Perhaps even more important is the spectral coherence, which is a more useful detection statistic than is the spectral correlation. Here is the effect of increasing resolution product on the FSM-based coherence estimates:

My rules of thumb for choosing the block length and the spectral resolution are to use as large a as possible, considering computational cost and whether the signal(s) in the data persist, and a large when performing a blind search for cycle frequencies. When the cycle frequencies are known in advance, use large , but choose the spectral resolution so that the features in the spectral correlation (and PSD) are adequately resolved, but are not over-smoothed.

For the TSM and FSM, good default values for lie in the range of 1-5% of the sampling frequency for the input data to be analyzed. For the FSM, this means a smoothing window with width (number of frequency bins) equal to about to . For the TSM, this means a FFT block length of about to .