3 techniques for improved throughput in real-time Data Streaming & Processing using HSDIO cards

Real-time data streaming and processing from high-speed sensors/ADCs often prove to be challenging because of the limited onboard memory of the HSDIO hardware. One can overcome this limitation by optimization techniques and acquire data for a longer period of time (from as low as 100 ms without optimization to 200 seconds and beyond with optimization).

Example application

Consider a requirement to acquire XYZ position data from a gyrometer and to plot the position on a graph in real time using National Instruments PXIe-6556. Typically, the application is programmed to run at the maximum clock rate of the HSDIO, 200 MHz in this case, and the maximum SPI clock that the device/sensor supports. This choice of clock rate for the HSDIO would work well in general but is not optimized for maximizing the data capture time.

This article describes 3 ways in which to optimize the onboard memory usage to capture high sample rate data from ADCs or sensors and to process the data in real time – (1) Selection of HSDIO sample clock rate based on sensors data rate (2) Stopping the acquisition during data idle time (3) Real-time streaming of block of data using fetch method.

optimize the on-board memory usage to capture high sample rate data from ADCs or sensors and to process the data in real time

Selection of HSDIO Clock Rate

When the HSDIO acquires samples at 200 MHz (max rate), it reads a lot of redundant sample points than required for SPI communication at 20 MHz. Let us assume that the gyrometer supports 20 MHz SPI clock. Then, it is sufficient to run the HSDIO at 40 MHz to achieve 20 MHz of SPI communication with the gyrometer. The following waveform illustrates the SPI clock waveform and the HSDIO sample points when acquired at 200 MHz (max rate) and at 40 MHz (twice the SPI clock rate).

SPI clock waveform and the HSDIO sample points when acquired at 200 MHz (max rate) and at 40 MHz (twice the SPI clock rate)

Green dots: Acquiring samples at 200 MHz
Red dots: Acquiring samples at 40 MHz

When the optimum HSDIO sample rate for SPI communication is chosen, the number of redundant points captured is reduced, thereby increasing the capture time for the available onboard memory; in this case, from 40 ms to 200 ms.

Capture time comparison:

Memory Per Channel8 Mbits
HSDIO Sample Rate200 MHz40 MHz
Capture Time40 ms200 ms

 

Stopping the acquisition during data idle time:

If the gyrometer (sensor) sends XYZ data at 1,000 Hz, then the time between each sample from the sensor is 1 ms. If the data acquisition is stopped in this period, which is the idle time in between actual data, then the number of acquired data points will be reduced and data capture time can be extended.

Stopping the acquisition during data idle time

How can the system be configured to capture the points of interest in the communication (sensor data) and to not capture any data during the idle time?

This can be achieved by configuring the Pattern Match Start Trigger and Pattern Match Pause Trigger of the HSDIO driver as described in the steps below. The idea is to capture the points only on the rising/falling edge of the SCLK, depending on the SPI Mode.

  • When configuring the dynamic acquisition for HSDIO, assign all channels (SCLK, MOSI, MISO, CS) to the session. This enables us to configure pattern match trigger in any of the channels.
  • Connect the SCLK to the Channel list of the Start and Pause Trigger configuration VI.
  • Choose the start trigger and pause trigger pattern based on the SPI mode and connect it to the respective VIs as shown in the image below.
  • For the rising edge, data should be captured during the rising edge of the SCLK and paused when the SCLK becomes LOW.

configuring the Pattern Match Start Trigger and Pattern Match Pause Trigger of the HSDIO driver

With the above pattern match trigger configuration, only the points during the rising edge of the clock are captured as highlighted by the diamond shape in the waveform image below. Thus, we have eliminated data capture in the long idle time of the HSDIO acquisition session.
eliminating data capture in the long idle time of the HSDIO acquisition session

Capture Time Comparison:

Let us say that the gyrometer sends 8 bits of position data for each axis (X,Y,Z). The SPI command format consists of 8 bits of register address and 24 bits of XYZ data, thus a total of 32 bits for each transaction. With this optimized Pattern Match Trigger, we can read the data for 250 seconds.

Gyrometer Sample Rate1,000 Hz
Gyrometer Sample Time1 ms
HSDIO Memory Per Channel8 Mbits
No of bits per Gyro Sample32 bits
Capture Time8M/32 = 2,50,000 ms = 250 s

 

Now, despite capturing only the required data, the onboard memory of the HSDIO card limits us from capturing the data continuously (depending on the memory in the card) without clearing the buffer for a long time. This brings up the next question.

How to capture data continuously and display it in real time?
We can achieve this using a producer-consumer loop to separate the data capture from HSDIO card, from other activities like parsing the data and displaying it in a user interface. The producer loop will only fetch the data available in the onboard memory of the HSDIO card and pass it to the consumer loop through the Queue.

Read Block of data:

When you fetch data from HSDIO memory, reading all available samples in HSDIO sample may contain partial data. Consider reading N number of sensor data at a time from the HSDIO memory for every 50/100 ms. Calculate the number of HSDIO samples to read based on the loop rate and the number of HSDIO sample points per sensor data. For example;

No of sample per sensor data (per SPI command)32
No of sensor data for 100 ms100 (as per sample of 1000Hz)
No of HSDIO samples to read100*32 = 3200

 

This will make sure you have all the XYZ values of each sensor data sample at once for conversion. This also avoids extra work of the software like identifying, storing the extra samples in the application’s local memory and using it in the next iteration.

Please refer the code snippet below for rounding-off the number of sensor data points and to fetch the samples from the HSDIO buffer.

rounding-off the number of sensor data points and to fetch the samples from the HSDIO buffer

The consumer loop will obtain the data from producer loop and perform any post-processing like extracting the actual XYZ data from the sensor (gyrometer) in the SPI packet and converting them to the ‘engineering unit’ required (based on sensor), and update the display graph.

Soliton has used these optimization techniques to build applications for long duration sensor functional testing and ADC validation that enables the user to detect missing data or drift in values over a period of time. Contact us at [email protected] to know more about our PXI based solutions:

Upcoming WebinarEffortless Digital Interface Compliance Testing

Join our free, one-hour webinar on January 12, 2022, to explore Soliton's Protocol Validation Solutions.

Click below to register and join our upcoming webinar!