In our articles about FM and AM, we used audio files as our "transmission medium". That's what we could do in 2010, but it left for question whether the concepts really work when true radio is the medium.
Well, we are in 2019 and SDR (software-defined radio dongles are everywhere. So we can finally put our code to test, at least for receiving. I don't possess a TX-capable SDR, but it is on my backlog.
Most SDRs in market are clones of RTL-SDR. Even though all of them work for reasonably strong signals, I recommend buying the real thing; the device is better and the antenna included in the kit is very nice.
Whatever the mode (AM, SSB, FM, digital), demodulation always starts by "mixing" – multiplying the radio signal by a locally generated carrier. In essence, the SDR dongle just does the mixing, and digitally samples the result. From that point on, software takes over.
RTL-SDR homodyne or direct conversion receiver, since it moves the radio signal to baseband in one go. Analog receiver, and high-quality SDRs, are heterodyne or double conversion: they mix once to move the signal to a fixed intermediate frequency, and then mix once more to get the baseband. This allows fo better filtering. Some radios even do triple conversion.
Some modes demand mixing by two versions (phased by 90º) of the carrier, in order to extract the phase information. This is known as quadrature or I/Q demodulation, which is implemented by RTL-SDR. The I/Q demodulator is a "universal receiver" and can be the front-end of any mode. (Likewise, an I/Q modulator is a "universal transmitter").
The SDR techniques are not restricted to SDR dongles. Many professional transceivers use them, generally at receiving side. For now, it is more often seen in low-cost devices.
When we engage the RTL-SDR dongle, we need to supply at least two parameters: carrier and sampling rate. For example, the command below asks for a 121MHz carrier and a sample rate of 1920k samples per second:
rtl_sdr -f 121M -s 1920k sample.rtl
The SDR filters the signal to avoid sampling aliasing, therefore the sampling rate also means the recorded bandwidth. In the example above, we are capturing a 1.92MHz band centered at 121Mhz (120.04MHz to 121.96MHz). Since the filtering is not perfect, the "clean" bandwidth is about 80% of the sampling rate (in the case, 120.20MHz to 120.80Mhz).
RTL-SDR sample precision is 8 bits. Each sample is a tuple of two values: I and Q. A sample rate of 1920k implies a data rate of 3.84MB/s. Individual values go from -127.5 to +127.5, offset by 127.5 so they fit in an integer byte. (Which means negative values are not expressed in two-complement.)
The Python and shell scripts mentioned from now on can be found in this GitHub repository.
In the article about FM modulation, the receiver front-end is I/Q demodulation followed by low-pass filtering. The SDR dongle already does these things for us, so our "Python radio" doesn't need a front-end. All we need to do, is to tune the RTL-SDR carrier on radio frequency, and use a sampling rate that limits bandwidth to a single station Excerpt from python_fm_mono:
rtl_sdr -f 89.5M -s 256k -n 2560000 teste.iq cat teste.iq | ./fm1.py > teste.raw sox -t raw -r 256000 -b 16 -c 1 -L -e signed-integer teste.raw \ -d rate 32000
RTL-SDR imposes a limitation: sampling rate must be lower than 300k or bigger than 900k. If we want to capture a bandwidth of e.g. 500kHz, we must capture a wider band and filter on software. Since the FM station bandwidth is about 200kHz, we didn't have to worry about this.
On the other hand, we could have captured a wider band (RTL-SDR can go beyond 2.4MHz) and listen/record several FM stations at the same time. The software side would get more complex, but it can be useful.
When the receiver front-end is I/Q, and the carrier is centered on FM station frequency, the signal frequency deviations can be detected as phase variations. The absolute phase does not matter; it is the rate of variation or phase rotation that interests us.
It is equally of no interest whether the signal is strong or weak; just the relationship between I and Q are important for FM. (The exact opposite happens in AM detection: the raw material for audio detection is the sum of I and Q.)
The Python script fm1.py implements a minimally viable FM receiver that ingests SDR samples and spits audio samples.
The scripts python_fm_mono records 10 seconds worth of SDR samples, and converts them into an audio file. The scripts python_fm_rt plays radio continuously, if the computer is fast enough to do it real-time.
The FM "audio" has a very large bandwidth (75-100kHz), but the mono audio is limited to 15kHz. The ultrasonic part can carry extra information, depending on region and station.
The script fm1s.py implements a FM stereo decoder (unfortunately, it doesn't do RDS data yet). The scripts python_fm_stereo and python_fm_rt_stereo start RTL-SDR and forward samples to the Python script. Both use the dongle the exact same way: the stereo magic happens after FM modulation is completed.
The implementation of stereo decoding in sofware was a piece of work...
As we tried to explain in the article bout AM, decoding AM-SC is difficult, because we need to generate an exact replica of the transmitter's carrier at the receiver, equal in frequency and phase. That's why stereo FM sends a 19kHz pilot tone. The stereo and RDS carriers are in-phase cosines whose frequencies are integer multiples of the pilot tone.
The usual method of generating such carriers is using a PLL (Phase-Locked Loop) with zero-crossing detection. The idea is, when the pilot tone changes polarity, being a cosine function, its phase is either 90º or 270º. In these moments, the local carrier should be at 180º.
As we our carrier is "rushing" or "dragging" in relation to the pilot tone, we ajust the frequency. The difficulty of PLL is to make these adjustments so the carrier gets in sync fast enough, while avoiding a runaway oscillation.
The first thing is to isolate the pilot tone with a notch filter. We also need filters to isolate the mono signal, the modulated stereo signal, and the demodulated result. The FIR filters employed in our "stereo set" can be found at filters.py. I had a NIH moment and wrote the filters myself, but the SciPy package (and possibly others) offer ready-to-use filters.
Before someone asks: it is not good enough to generate a 38kHz cosine wave to demodulate stereo. Neither you computer nor the FM station have 100.0000% precise clocks, so there will always be a drift, enough to spoil the stereo signal. Yes, I have tried that way; the sound keeps "coming" and "going away" as the carrier gets in and out of phase.
At least the pilot tone is always pretty close to 19kHz. If you are debugging a PLL and your carrier deviates too much from 19kHz, the highest probability is a bug in PLL, not a problem on the FM station.
Perhaps someone have noted our mono FM receiver has too much treble.
In the stereo version, the audio is low-pass filtered to 15kHz, and it is also "deemphasized". Deemphasis is a low-pass filter with very gentle ramp (10dB across the band).
This is necessary because FM stations "emphasize" the audio, that is, they boost treble using a 10dB ramp. The idea is to increase treble resistance to radio noise, which increases linearly with frequency. (Vinyl LPs employ the same technique.)
As happens with such legacy technologies, the emphasis/deemphasis definition is confusing and leaves room to interesting interpretations. In theory, emphasis is defined as the time constant of the hypothetical emphasis RC filter. For example, an emphasis of 75µs (the standard in the Americas) means a cutoff (-3dB) frequency of around 2100Hz.
An RC filter has a ramp of 6dB/octave, which means a ramp of 15dB between 2100 and 15000Hz. In pratice, a 10dB ramp is used, and even the time constant is different in every region (e.g. it is 50µs in Europe). It is possible that some FM radio manufacturer uses a different ramp to sound "nicer" and HiFi-like.
The first FM stations were mono. When stereo was introduced, it had to be backward compatible with older or simpler receiver sets. Being in the ultrasonic range, the stereo sub-band is either inaudible or filtered out by mono sets.
The 0-15Khz mono signal is the sum of left and right channels (L+R). This is what a mono listener expects to hear. The 23-53kHz sub-band carries the difference between channels (L-R). This technique is known as joint-stereo and it is also used in e.g. MP3 files.
The reeiver reconstitutes the L and R channels by adding and subtracting the mono and joint-stereo signals:
Since the FM audio band is 75-100kHz, we don't skimp on data rate, we use 256k from SDR to joint-stereo detection. Right at the end, we downsample the audio to 32k.
256k:32k is an integer 8:1 relationship, making it simple to downsample. We just decimate, that is, use 1 every 8 samples. The signal must be pre-filtered before decimation so it fits in the new bandwidth, but we already did that in deemphasis filter.
By the way, upsampling is equally simple. E.g. to go from 32k to 256k, just stuff seven zero samples along with every original sample, and filter the result to remove the artifacts above 16kHz. For rational, non-integer relationships (e.g. 32k to 48k is 3:2) one can upsample and then downsample. The post-upsampling and pre-downsampling filters can be combined into one, and use the polyphase filter optimization.
Debugging took a lot of listening and checking audio files on Audacity. The most complicated component is the PLL. Since it is not that easy to hear when stereo is bad when the mono signal is still good, the debug mode (-d) records mono and joint-stereo channels instead of left and right. A third channel contains the pilot tone.
Most of the time, a PLL bug causes the decoded stereo to "come and go", the volume keeps oscillating, which is immediately visible on Audacity.`
Moreover, in debug mode, no downsampling takes place and audio is recorded at 256k samples. Mono channel is not low-pass filtered; it is still 'raw' audio, so it is possible to find all sub-bands (mono, stereo, pilot tone, RDS) in the spectrum.
In general, "Python FM" code style privileges clarity over speed. The mono version is particularly minimalistic. (Using NumPy gives a big boost in speed without sacrificing clarity.)
Unfortunately the stereo version could not play in real-time, so we had to resort to Cython. The script can run in two modes: pure Python or using the Cython module, passing the -o parameter (you need to compile the module before use).
At the moment, the optimized version uses 50% of a single CPU, so yeah, there is a lot of room for improvement, but it is good enough for our purposes.
Stereo decoding should only be activated when the pilot tone is clearly "there". Notify when stereo is activated/deactivated.
DC (very low Hz) signal removal without sacrificing bass.
RDS data text decoding and printing.