SD card writing limitations

Application Dev
1

Hi there,

I’m trying to write messages to the Spotter SD card using the bm_serial.py library. I have my mote wired up to a Raspberry Pi RP2040 over UART and can correctly save messages to the SD. Now that it’s workings, I’d like to push the limits on message size and speed.

How fast can I write to the SD card using this approach and what are the message size limitations?

I’ve been experimenting with the message size and write speed but keep getting a “warning: user serial buffer full” message on the mote. This coincides with missing data on the SD - is the serial buffer between the Pi and Mote getting full?

Test conditions:
Baud rate = 115200
Message size = 75-200 bytes
Time between messages = 250-1000 milliseconds

When I look at the time stamp on the bm SD logs there seems to be a consistent 1 second delay between logs no matter how fast I try to write the data. Is there a hard coded delay in one of the apps that reduces the write speed to the SD using bm_serial.py?

This is for an IMU project to monitor the tilt angle of a node as it wobbles in the waves. I’d like to save the data to the SD card every 100-250 milliseconds (if possible) and I already aggregate the statistics after five minutes and send that over cellular. Faster = better for this project!

If that type of write speed is not possible then I’m considering saving the high frequency data to memory on the Pi for 5 min before doing a large data dump to the SD card using multiple bm_serial calls - the downside is I won’t get the UTC timestamp generated for each unique IMU read and will need to use the mote tick time instead.

Knowing serial buffer size and write speed limitations will be helpful for either approach.

Thanks for the help.

1 Like
2

What is the exact error message? I can’t find “warning: user serial buffer full” in the code anywhere. The most likely option that I see is “WARN payload uart user byte stream buffer full!” from the payload UART library. There’s also “buffer full, resetting” in the serial bridge app code.

There are a few buffers along the chain, but they all have length 2048.

If the message is “WARN payload uart user byte stream buffer full!” then that would mean that the serial_bridge app on the mote isn’t reading out of the PLUART byte stream buffer fast enough. The app code is very simple and is reading bytes in an infinite loop as soon as they are available.

The common app main code calls the user loop every 10 ms which would be a bottleneck for you. It means that the fastest the app could read a single 200-byte message would be 2 seconds.

I’d recommend experimenting with that delay. Try recompiling the serial_bridge app with that delay changed from 10ms to 1ms and see whether system performance stays acceptable. Your app will be allowing less time for the rest of the system to run, but since the serial_bridge app is so simple, I’m guessing it will be OK.

3

Yep, that’s the error alright. “WARN payload uart user byte stream buffer full!”

Looks like this is the impetus I needed to finally setup my development environment to work on BM code. I was skating by with just using bm_serial.

Gulp, now it’s the tricky stuff.

Thanks for the help, will report back if/when I can figure this one out.

1 Like