Category: Textbook Signals

PSK/QAM Cochannel Dataset for Modulation Recognition Researchers [CSPB.ML.2023]

The next step in dataset complexity at the CSP Blog: cochannel signals.

I’ve developed another dataset for use in assessing modulation-recognition algorithms (machine-learning-based or otherwise) that is more complex than the original sets I posted for the ML Challenge (CSPB.ML.2018 and CSPB.ML.2022). Half of the new dataset consists of one signal in noise and the other half consists of two signals in noise. In most cases the two signals overlap spectrally, which is a signal condition called cochannel interference.

We’ll call it CSPB.ML.2023.

Continue reading “PSK/QAM Cochannel Dataset for Modulation Recognition Researchers [CSPB.ML.2023]”

Neural Networks for Modulation Recognition: IQ-Input Networks Do Not Generalize, but Cyclic-Cumulant-Input Networks Generalize Very Well

Neural networks with CSP-feature inputs DO generalize in the modulation-recognition problem setting.

In some recently published papers (My Papers [50,51]), my ODU colleagues and I showed that convolutional neural networks and capsule networks do not generalize well when their inputs are complex-valued data samples, commonly referred to as simply IQ samples, or as raw IQ samples by machine learners.(Unclear why the adjective ‘raw’ is often used as it adds nothing to the meaning. If I just say Hey, pass me those IQ samples, would ya?, do you think maybe he means the processed ones? How about raw-I-mean–seriously-man–I-did-not-touch-those-numbers-OK? IQ samples? All-natural vegan unprocessed no-GMO organic IQ samples? Uncooked IQ samples?) Moreover, the capsule networks typically outperform the convolutional networks.

In a new paper (MILCOM 2022: My Papers [52]; arxiv.org version), my colleagues and I continue this line of research by including cyclic cumulants as the inputs to convolutional and capsule networks. We find that capsule networks outperform convolutional networks and that convolutional networks trained on cyclic cumulants outperform convolutional networks trained on IQ samples. We also find that both convolutional and capsule networks trained on cyclic cumulants generalize perfectly well between datasets that have different (disjoint) probability density functions governing their carrier frequency offset parameters.

That is, convolutional networks do better recognition with cyclic cumulants and generalize very well with cyclic cumulants.

So why don’t neural networks ever ‘learn’ cyclic cumulants with IQ data at the input?

The majority of the software and analysis work is performed by the first author, John Snoap, with an assist on capsule networks by James Latshaw. I created the datasets we used (available here on the CSP Blog [see below]) and helped with the blind parameter estimation. Professor Popescu guided us all and contributed substantially to the writing.

Continue reading “Neural Networks for Modulation Recognition: IQ-Input Networks Do Not Generalize, but Cyclic-Cumulant-Input Networks Generalize Very Well”

Epistemic Bubbles: Comments on “Modulation Recognition Using Signal Enhancement and Multi-Stage Attention Mechanism” by Lin, Zeng, and Gong.

Another brick in the wall, another drop in the bucket, another windmill on the horizon …

Let’s talk more about The Cult. No, I don’t mean She Sells Sanctuary, for which I do have considerable nostalgic fondness. I mean the Cult(ure) of Machine Learning in RF communications and signal processing. Or perhaps it is more of an epistemic bubble where there are The Things That Must Be Said and The Unmentionables in every paper and a style of research that is strictly adhered to but that, sadly, produces mostly error and promotes mostly hype. So we have shibboleths, taboos, and norms to deal with inside the bubble.

Time to get on my high horse. She’s a good horse named Ravager and she needs some exercise. So I’m going to strap on my claymore, mount Ravager, and go for a ride. Or am I merely tilting at windmills?

Let’s take a close look at another paper on machine learning for modulation recognition. It uses, uncritically, the DeepSig RML 2016 datasets. And the world and the world, the world drags me down…

Continue reading “Epistemic Bubbles: Comments on “Modulation Recognition Using Signal Enhancement and Multi-Stage Attention Mechanism” by Lin, Zeng, and Gong.”

Some Concrete Results on Generalization in Modulation Recognition using Machine Learning

Neural networks with I/Q data as input do not generalize in the modulation-recognition problem setting.

Update May 20, 2022: Here is the arxiv.org link.

Back in 2018 I posted a dataset consisting of 112,000 I/Q data files, 32,768 samples in length each, as a part of a challenge to machine learners who had been making strong claims of superiority over signal processing in the area of automatic modulation recognition. One part of the challenge was modulation recognition involving eight digital modulation types, and the other was estimating the carrier frequency offset. That dataset is described here, and I’d like to refer to it as CSPB.ML.2018.

Then in 2022 I posted a companion dataset to CSPB.ML.2018 called CSPB.ML.2022. This new dataset uses the same eight modulation types, similar ranges of SNR, pulse type, and symbol rate, but the random variable that governs the carrier frequency offset is different with respect to the random variable in CSPB.ML.2018. The purpose of the CSPB.ML.2022 dataset is to facilitate studies of the dataset-shift, or generalization, problem in machine learning.

Throughout the past couple of years I’ve been working with some graduate students and a professor at Old Dominion University on merging machine learning and signal processing for problems involving RF signal analysis, such as modulation recognition. We are starting to publish a sequence of papers that describe our efforts. I briefly describe the results of one such paper, My Papers [51], in this post.

Continue reading “Some Concrete Results on Generalization in Modulation Recognition using Machine Learning”

Wow, Elsevier, Just … Wow. Comments On “Cyclic Correntropy: Properties and the Application in Symbol Rate Estimation Under Alpha-Stable Distributed Noise,” by S. Luan et al.

Can we fix peer review in engineering by some form of payment to reviewers?

Let’s talk about another paper about cyclostationarity and correntropy. I’ve critically reviewed two previously, which you can find here and here. When you look at the correntropy as applied to a cyclostationary signal, you get something called cyclic correntropy, which is not particularly useful except if you don’t understand regular cyclostationarity and some aspects of garden-variety signal processing. Then it looks great.

But this isn’t a post that primarily takes the authors of a paper to task, although it does do that. I want to tell the tale to get us thinking about what ‘peer’ could mean, these days, in ‘peer-reviewed paper.’ How do we get the best peers to review our papers?

Let’s take a look at The Literature [R173].

Continue reading “Wow, Elsevier, Just … Wow. Comments On “Cyclic Correntropy: Properties and the Application in Symbol Rate Estimation Under Alpha-Stable Distributed Noise,” by S. Luan et al.”

Shifted Dataset for the Machine-Learning Challenge: How Well Does a Modulation-Recognition DNN Generalize? [Dataset CSPB.ML.2022]

Another RF-signal dataset to help push along our R&D on modulation recognition.

Update February 2023: A third dataset has been posted to the CSP Blog: CSPB.ML.2023. It features cochannel signals.

Update January 2023: I’m going to put Challenger results in the Comments. I’ve received a Challenger’s decisions and scored them in January 2023. See below.

In this post I provide a second dataset for the Machine-Learning Challenge I issued in 2018 (CSPB.ML.2018). This dataset is similar to the original dataset, but possesses a key difference in that the probability distribution of the carrier-frequency offset parameter, viewed as a random variable, is not the same, but is still realistic.

Blog Note: By WordPress’ count, this is the 100th post on the CSP Blog. Together with a handful of pages (like My Papers and The Literature), these hundred posts have resulted in about 250,000 page views. That’s an average of 2,500 page views per post. However, the variance of the per-post pageviews is quite large. The most popular is The Spectral Correlation Function (> 16,000) while the post More on Pure and Impure Sinewaves, from the same era, has only 316 views. A big Thanks to all my readers!!

Continue reading “Shifted Dataset for the Machine-Learning Challenge: How Well Does a Modulation-Recognition DNN Generalize? [Dataset CSPB.ML.2022]”

The Principal Domain for the Spectral Correlation Function

What are the ranges of spectral frequency and cycle frequency that we need to consider in a discrete-time/discrete-frequency setting for CSP?

Let’s talk about that diamond-shaped region in the (f, \alpha) plane we so often see associated with CSP. I’m talking about the principal domain for the discrete-time/discrete-frequency spectral correlation function. Where does it come from? Why do we care? When does it come up?

Continue reading “The Principal Domain for the Spectral Correlation Function”

SPTK (and CSP): Random Processes

The merging of conventional probability theory with signal theory leads to random processes, also known as stochastic processes. The ideas involved with random processes are central to cyclostationary signal processing.

Previous SPTK Post: Examples of Random Variables Next SPTK Post: The Sampling Theorem

In this Signal Processing ToolKit post, I provide an introduction to the concept and use of random processes (also called stochastic processes). This is my perspective on random processes, so although I’ll introduce and use the conventional concepts of stationarity and ergodicity, I’ll end up focusing on the differences between stationary and cyclostationary random processes. The goal is to illustrate those differences with informative graphics and videos; to build intuition in the reader about how the cyclostationarity property comes about, and about how the property relates to the more abstract mathematical object of a random process on one hand and to the concrete data-centric signal on the other.

So … this is the first SPTK post that is also a CSP post.

Continue reading “SPTK (and CSP): Random Processes”

The Signal-Processing Equivalent of Resume-Padding? Comments on “A Robust Modulation Classification Method Using Convolutional Neural Networks” by S. Zhou et al.

Does the use of ‘total SNR’ mislead when the fractional bandwidth is very small? What constitutes ‘weak-signal processing?’

Or maybe “Comments on” here should be “Questions on.”

In a recent paper in EURASIP Journal on Advances in Signal Processing (The Literature [R165]), the authors tackle the problem of machine-learning-based modulation recognition for highly oversampled rectangular-pulse digital signals. They don’t use the DeepSig datasets (one, two, three, four), but their dataset description and use of ‘signal-to-noise ratio’ leaves a lot to be desired. Let’s take a brief look. See if you agree with me that the touting of their results as evidence that they can reliably classify signals with ‘SNRs of -10 dB’ is unwarranted and misleading.

Continue reading “The Signal-Processing Equivalent of Resume-Padding? Comments on “A Robust Modulation Classification Method Using Convolutional Neural Networks” by S. Zhou et al.”

Comments on “Deep Neural Network Feature Designs for RF Data-Driven Wireless Device Classification,” by B. Hamdaoui et al

Another post-publication review of a paper that is weak on the ‘RF’ in RF machine learning.

Let’s take a look at a recently published paper (The Literature [R148]) on machine-learning-based modulation-recognition to get a data point on how some electrical engineers–these are more on the side of computer science I believe–use mathematics when they turn to radio-frequency problems. You can guess it isn’t pretty, and that I’m not here to exalt their acumen.

Continue reading “Comments on “Deep Neural Network Feature Designs for RF Data-Driven Wireless Device Classification,” by B. Hamdaoui et al”

Spectral Correlation and Cyclic Correlation Plots for Real-Valued Signals

Spectral correlation surfaces for real-valued and complex-valued versions of the same signal look quite different.

In the real world, the electromagnetic field is a multi-dimensional time-varying real-valued function (volts/meter or newtons/coulomb). But in mathematical physics and signal processing, we often use complex-valued representations of the field, or of quantities derived from it, to facilitate our mathematics or make the signal processing more compact and efficient.

So throughout the CSP Blog I’ve focused almost exclusively on complex-valued signals and data. However, there is a considerable older literature that uses real-valued signals, such as The Literature [R1, R151]. You can use either real-valued or complex-valued signal representations and data, as you prefer, but there are advantages and disadvantages to each choice. Moreover, an author might not be perfectly clear about which one is used, especially when presenting a spectral correlation surface (as opposed to a sequence of equations, where things are often more clear).

Continue reading “Spectral Correlation and Cyclic Correlation Plots for Real-Valued Signals”

SPTK: Convolution and the Convolution Theorem

Convolution is an essential element in everyone’s signal-processing toolkit. We’ll look at it in detail in this post.

Previous SPTK Post: Interconnection of Linear Systems      Next SPTK Post: Ideal Filters

This installment of the Signal Processing Toolkit series of CSP Blog posts deals with the ubiquitous signal-processing operation known as convolution. We originally came across it in the context of linear time-invariant systems. In this post, we focus on the mechanics of computing convolutions and discuss their utility in signal processing and CSP.

Continue reading “SPTK: Convolution and the Convolution Theorem”

Stationary Signal Models Versus Cyclostationary Signal Models

What happens when a cyclostationary time-series is treated as if it were stationary?

In this post let’s consider the difference between modeling a communication signal as stationary or as cyclostationary.

There are two contexts for this kind of issue. The first is when someone recognizes that a particular signal model is cyclostationary, and then takes some action to render it stationary (sometimes called ‘stationarizing the signal’). They then proceed with their analysis or algorithm development using the stationary signal model. The second context is when someone applies stationary-signal processing to a cyclostationary signal model, either without knowing that the signal is cyclostationary, or perhaps knowing but not caring.

At the center of this topic is the difference between the mathematical object known as a random process (or stochastic process) and the mathematical object that is a single infinite-time function (or signal or time-series).

A related paper is The Literature [R68], which discusses the pitfalls of applying tools meant for stationary signals to the samples of cyclostationary signals.

Continue reading “Stationary Signal Models Versus Cyclostationary Signal Models”

DeepSig’s 2018 Dataset: 2018.01.OSC.0001_1024x2M.h5.tar.gz

The third DeepSig dataset I’ve examined. It’s better!

Update February 2021. I added material relating to the DeepSig-claimed variation of the roll-off parameter in a square-root raised-cosine pulse-shaping function. It does not appear that the roll-off was actually varied as stated in Table I of [R137].

DeepSig’s datasets are popular in the machine-learning modulation-recognition community, and in that community there are many claims that the deep neural networks are vastly outperforming any expertly hand-crafted tired old conventional method you care to name (none are usually named though). So I’ve been looking under the hood at these datasets to see what the machine learners think of as high-quality inputs that lead to disruptive upending of the sclerotic mod-rec establishment. In previous posts, I’ve looked at two of the most popular DeepSig datasets from 2016 (here and here). In this post, we’ll look at one more and I will then try to get back to the CSP posts.

Let’s take a look at one more DeepSig dataset: 2018.01.OSC.0001_1024x2M.h5.tar.gz.

Continue reading “DeepSig’s 2018 Dataset: 2018.01.OSC.0001_1024x2M.h5.tar.gz”

All BPSK Signals

An analysis of DeepSig’s 2016.10A dataset, used in many published machine-learning papers, and detailed comments on quite a few of those papers.

Update March 2021

See my analyses of three other DeepSig datasets here, here, and here.

Update June 2020

I’ll be adding new papers to this post as I find them. At the end of the original post there is a sequence of date-labeled updates that briefly describe the relevant aspects of the newly found papers. Some machine-learning modulation-recognition papers deserve their own post, so check back at the CSP Blog from time-to-time for “Comments On …” posts.

Continue reading “All BPSK Signals”

SPTK: The Fourier Transform

An indispensable tool in CSP and all of signal processing!

Previous SPTK Post: The Fourier Series      Next SPTK Post: Linear Systems

This post in the Signal Processing Toolkit series deals with a key mathematical tool in CSP: The Fourier transform. Let’s try to see how the Fourier transform arises from a limiting version of the Fourier series.

Continue reading “SPTK: The Fourier Transform”

Symmetries of Higher-Order Temporal Probabilistic Parameters in CSP

What are the unique parts of the multidimensional cyclic moments and cyclic cumulants?

In this post, we continue our study of the symmetries of CSP parameters. The second-order parameters–spectral correlation and cyclic correlation–are covered in detail in the companion post, including the symmetries for ‘auto’ and ‘cross’ versions of those parameters.

Here we tackle the generalizations of cyclic correlation: cyclic temporal moments and cumulants. We’ll deal with the generalization of the spectral correlation function, the  cyclic polyspectra, in a subsequent post. It is reasonable to me to focus first on the higher-order temporal parameters, because I consider the temporal parameters to be much more useful in practice than the spectral parameters.

This topic is somewhat harder and more abstract than the second-order topic, but perhaps there are bigger payoffs in algorithm development for exploiting symmetries in higher-order parameters than in second-order parameters because the parameters are multidimensional. So it could be worthwhile to sally forth.

Continue reading “Symmetries of Higher-Order Temporal Probabilistic Parameters in CSP”

SPTK: The Fourier Series

A crucial tool for developing the temporal parameters of CSP.

Previous SPTK Post: Signal Representations            Next SPTK Post: The Fourier Transform

This installment of the Signal Processing Toolkit shows how the Fourier series arises from a consideration of representing arbitrary signals as vectors in a signal space. We also provide several examples of Fourier series calculations, interpret the Fourier series, and discuss its relevance to cyclostationary signal processing.

Continue reading “SPTK: The Fourier Series”

A Gallery of Cyclic Correlations

For your delectation.

There are some situations in which the spectral correlation function is not the preferred measure of (second-order) cyclostationarity. In these situations, the cyclic autocorrelation (non-conjugate and conjugate versions) may be much simpler to estimate and work with in terms of detector, classifier, and estimator structures. So in this post, I’m going to provide surface plots of the cyclic autocorrelation for each of the signals in the spectral correlation gallery post. The exceptions are those signals I called feature-rich in the spectral correlation gallery post, such as DSSS, LTE, and radar. Recall that such signals possess a large number of cycle frequencies, and plotting their three-dimensional spectral correlation surface is not helpful as it is difficult to interpret with the human eye. So for the cycle-frequency patterns of feature-rich signals, we’ll rely on the stem-style (cyclic-domain profile) plots that I used in the spectral correlation gallery post.

Continue reading “A Gallery of Cyclic Correlations”

Simple Synchronization Using CSP

Using CSP to find the exact values of symbol rate, carrier frequency offset, symbol-clock phase, and carrier phase for PSK/QAM signals.

In this post I discuss the use of cyclostationary signal processing applied to communication-signal synchronization problems. First, just what are synchronization problems? Synchronize and synchronization have multiple meanings, but the meaning of synchronize that is relevant here is something like:

syn·chro·nize: To cause to occur or operate with exact coincidence in time or rate

If we have an analog amplitude-modulated (AM) signal (such as voice AM used in the AM broadcast bands) at a receiver we want to remove the effects of the carrier sine wave, resulting in an output that is only the original voice or music message. If we have a digital signal such as binary phase-shift keying (BPSK), we want to remove the effects of the carrier but also sample the message signal at the correct instants to optimally recover the transmitted bit sequence. 

Continue reading “Simple Synchronization Using CSP”