A guide to the core knowledge required for DSP application development and feature engineering – by Dunstan Power and Guest Author John Edwards of Sigma Numerix

Digital Signal Processing (DSP) has been a key part of many real time applications for several decades but over the last few years the use of hybrid devices, which are sometimes called bridge devices, is becoming increasingly common. It is no longer necessary to use a dedicated DSP device for these applications so the choice of architecture to use has become much more complex.

A good example of where these new devices are being utilized is the enhancement of signals for later processing by algorithms such as Deep Neural Networks (DNNs). In applications from voice recognition to vibration analysis, Feature Extraction, or Feature Engineering as it is increasingly being called, is an application of DSP algorithms that needs to connect to a wider network for the cloud based back-end processing.

Feature Engineering is the art of pre-processing signals before passing them to the training engine or the classifier. Extracting the best features for an ML model to classify is an art and can make the difference between success and failure. Feature Engineering includes processes such as event counting, period or statistics; filtering; statistics analysis such as mean and variance; peak or zero crossing detection; frequency spectrum or histogram estimation.

Implementing real-time signal processing algorithms requires the design engineer to traverse two potential minefields; choice of algorithm and choice of hardware. The two are inextricably linked – the wrong choice of either will often end up with a sub-optimal decision regarding the other.
Historically, DSP algorithms have been run on specialized DSP devices that are optimized and targeted towards these applications. In more modern times, DSP functionality has become far more commonly integrated into general purpose microprocessors and microcontrollers so the term “DSP device” refers to any processor executing signal processing algorithms.

So, for integrating digital signal processing algorithms into your future application, let’s go through each of the top Digital Signal Processing fundamentals every electronics engineer should know – starting with:

1. Time Domain Algorithms

Signal processing algorithms can be split into two main categories for the majority of applications – Time Domain and Frequency Domain.

The most common time domain algorithms are filters, of which there are two basic types. The most basic filter uses a feedforward path for all of the data processing and these filters are often referred to as Finite Infinite Response (FIR) filters, also known as convolution filters. The more advanced type of filter is the Infinite Impulse Response (IIR) filter, which incorporates feedback.

This equation shows the FIR filter or convolution algorithm:

Which is implemented using the following flow diagram:

The FIR filter or convolution algorithm is the combination of 2 mathematical operators – the multiplication of the data values by the filter coefficients and then the accumulation of the products to get the final result. This Multiply-Accumulate (MAC) operation is performed over all the filter coefficients for every data sample so it requires a lot of operations for large filters. Despite their high operational complexity FIR filters are, by their nature, easy and safe to use because they are unconditionally stable.

IIR filters on the other hand include feedback elements, as this equation shows:

Here is the corresponding the flow diagram:

IIR filters are, therefore, an extension of the same Multiply-Accumulate architecture used in FIR filters. The inclusion of the feedback path makes these filters much more efficient than the FIR filter. However, it also leads to possible instability and hence they require much more testing.

FIR filters can have a linear phase response, when the impulse response (coefficients) is symmetrical, and this is a useful property for some applications but most applications do not require linear phase so the enhanced performance, lower memory requirements and higher efficiency of IIR filters means that they are the most commonly used. All modern digital filter design packages can support both FIR and IIR filter design.

2. Frequency Domain Algorithms

Real-world signals are sampled in the time domain, using an Analog To Digital Converter but the time domain is not necessarily the most efficient domain for processing the data. Most engineers are familiar with the frequency domain being used for spectral analysis; but we should also consider that our time domain data stream can be converted to the frequency domain for processing as well as analysis.

The Discrete Fourier Transform (DFT) – typically implemented using the Fast Fourier Transform (FFT) algorithm – is the most common method for performing the transform from the time domain to the frequency domain. The DFT algorithm is shown in the following equation:

While we are not going to dive into the details of this algorithm, in this article, it is useful to consider it from a high level. If you look at this algorithm you will see how similar it is to the convolution algorithm, that we looked at earlier, and this observation is true because the DFT is essentially just a bank of digital filters.

The FFT, being an optimization of the DFT data flow for minimizing the MIPS requirements, is typically used to convert from the time domain to the frequency domain for analysis – as can be seen here:

But the frequency domain can also be used for processing, using a property that convolution in the time domain is the equivalent to multiplication in the frequency domain. As we have seen previously, convolution is a Multiply-Accumulate operation so it is much more MIPS intensive than simply cross multiplying two arrays in the frequency domain.  This diagram illustrates Frequency domain filtering…

To complete your tour of Digital Signal Processing fundamentals, download our handy guide – 8 DSP Fundamentals Every Electronics Engineer Should Know:

Learn more about Guest Author – John Edwards

John Edwards, Digital Signal Processing and Embedded Systems Consultant, Sigma Numerix Ltd

Since obtaining his BEng (Hons) from the University of Bradford, in the 1980s, John has worked as a Signal Processing Applications Engineer for companies such as Loughborough Sound Images, Blue Wave Systems, Motorola, Picochip Designs and XMOS Semiconductors.

John now works as a Digital Signal Processing and Embedded Systems Consultant for Sigma Numerix Ltd. He has worked with DSPs in a wide range of applications including wireless (2G, 3G WCDMA and 4G LTE), Voice Over IP, voice band and broadband modems, control, medical instrumentation and noise and vibration analysis.

John is a member of the IET, IEEE, and is a regular contributor of papers at international DSP conferences. In addition, John has been a visiting lecturer at the University of Oxford since 1993 and teaches the annual Digital Signal Processing course, as part of the Summer Engineering Program, for industry.

How can ByteSnap help you today?

From start-ups to blue chips, ByteSnap’s embedded systems software developers & electronic design engineers are enabling companies to stay a step ahead by providing them with bespoke solutions. Maintain your competitive edge – contact us today and let your business be among them!