3 techniques for improved throughput in real-time Data Streaming & Processing using HSDIO cards
Real-time data streaming and processing from high-speed sensors/ADCs often prove to be challenging because of the limited onboard memory of the HSDIO hardware. One can overcome this limitation by optimization techniques and acquire data for a longer period of time (from as low as 100 ms without optimization to 200 seconds and beyond with optimization).
Consider a requirement to acquire XYZ position data from a gyrometer and to plot the position on a graph in real time using National Instruments PXIe-6556. Typically, the application is programmed to run at the maximum clock rate of the HSDIO, 200 MHz in this case, and the maximum SPI clock that the device/sensor supports. This choice of clock rate for the HSDIO would work well in general but is not optimized for maximizing the data capture time.
This article describes 3 ways in which to optimize the onboard memory usage to capture high sample rate data from ADCs or sensors and to process the data in real time – (1) Selection of HSDIO sample clock rate based on sensors data rate (2) Stopping the acquisition during data idle time (3) Real-time streaming of block of data using fetch method.
When the HSDIO acquires samples at 200 MHz (max rate), it reads a lot of redundant sample points than required for SPI communication at 20 MHz. Let us assume that the gyrometer supports 20 MHz SPI clock. Then, it is sufficient to run the HSDIO at 40 MHz to achieve 20 MHz of SPI communication with the gyrometer. The following waveform illustrates the SPI clock waveform and the HSDIO sample points when acquired at 200 MHz (max rate) and at 40 MHz (twice the SPI clock rate).
Green dots: Acquiring samples at 200 MHz
Red dots: Acquiring samples at 40 MHz
When the optimum HSDIO sample rate for SPI communication is chosen, the number of redundant points captured is reduced, thereby increasing the capture time for the available onboard memory; in this case, from 40 ms to 200 ms.
|Memory Per Channel||8 Mbits|
|HSDIO Sample Rate||200 MHz||40 MHz|
|Capture Time||40 ms||200 ms|
If the gyrometer (sensor) sends XYZ data at 1,000 Hz, then the time between each sample from the sensor is 1 ms. If the data acquisition is stopped in this period, which is the idle time in between actual data, then the number of acquired data points will be reduced and data capture time can be extended.
How can the system be configured to capture the points of interest in the communication (sensor data) and to not capture any data during the idle time?
This can be achieved by configuring the Pattern Match Start Trigger and Pattern Match Pause Trigger of the HSDIO driver as described in the steps below. The idea is to capture the points only on the rising/falling edge of the SCLK, depending on the SPI Mode.
With the above pattern match trigger configuration, only the points during the rising edge of the clock are captured as highlighted by the diamond shape in the waveform image below. Thus, we have eliminated data capture in the long idle time of the HSDIO acquisition session.
Let us say that the gyrometer sends 8 bits of position data for each axis (X,Y,Z). The SPI command format consists of 8 bits of register address and 24 bits of XYZ data, thus a total of 32 bits for each transaction. With this optimized Pattern Match Trigger, we can read the data for 250 seconds.
|Gyrometer Sample Rate||1,000 Hz|
|Gyrometer Sample Time||1 ms|
|HSDIO Memory Per Channel||8 Mbits|
|No of bits per Gyro Sample||32 bits|
|Capture Time||8M/32 = 2,50,000 ms = 250 s|
Now, despite capturing only the required data, the onboard memory of the HSDIO card limits us from capturing the data continuously (depending on the memory in the card) without clearing the buffer for a long time. This brings up the next question.
How to capture data continuously and display it in real time?
We can achieve this using a producer-consumer loop to separate the data capture from HSDIO card, from other activities like parsing the data and displaying it in a user interface. The producer loop will only fetch the data available in the onboard memory of the HSDIO card and pass it to the consumer loop through the Queue.
When you fetch data from HSDIO memory, reading all available samples in HSDIO sample may contain partial data. Consider reading N number of sensor data at a time from the HSDIO memory for every 50/100 ms. Calculate the number of HSDIO samples to read based on the loop rate and the number of HSDIO sample points per sensor data. For example;
|No of sample per sensor data (per SPI command)||32|
|No of sensor data for 100 ms||100 (as per sample of 1000Hz)|
|No of HSDIO samples to read||100*32 = 3200|
This will make sure you have all the XYZ values of each sensor data sample at once for conversion. This also avoids extra work of the software like identifying, storing the extra samples in the application’s local memory and using it in the next iteration.
Please refer the code snippet below for rounding-off the number of sensor data points and to fetch the samples from the HSDIO buffer.
The consumer loop will obtain the data from producer loop and perform any post-processing like extracting the actual XYZ data from the sensor (gyrometer) in the SPI packet and converting them to the ‘engineering unit’ required (based on sensor), and update the display graph.
Soliton has used these optimization techniques to build applications for long duration sensor functional testing and ADC validation that enables the user to detect missing data or drift in values over a period of time. Contact us at [email protected] to know more about our PXI based solutions:
Please fill in the following form and we shall get in touch with you soon to set up a demo of the requested service/product. Thank you![contact-form-7 404 "Not Found"]
Fill in this form and submit for your download to start automatically
Submit this form and the PRIM Brochure document will be sent to your email.
Join our free, one-hour webinar on January 12, 2022, to explore Soliton's Protocol Validation Solutions.
Click below to register and join our upcoming webinar!