BM Sensor Integration - SDI-12 Multiparameter Sonde EXO3s

Introduction

This blog is an ongoing attempt to be able to communicate with SDI-12 sensors. This is not as popular as a protocol but is still a standard used widely in environment data acquisition.

The Bristlemouth DevKit has the terminal blocks available for SDI-12 communication. We have chosen the YSI EXO3s Multi-parameter Sonde for this effort since it has native SDI-12 communication.

This post will capture the steps, the challenges and any bugs I might come across while integrating the multi-parameter sonde EXO3s to the Bristlemouth DevKit.

What is SDI-12? It’s a single data line serial communication that is widely used in environment data acquisition. It has 3 lines - GND, Data and 12VDC. Here is more information on the nitty gritty details of what makes SDI-12 communication

What is my strategy?

My approach to this project is pretty simple. The steps I plan to follow are -

  • The sensor shows up and can be connected to on a terminal on the laptop.
  • Send commands and receive responses. Understand the data frames coming via the SDI-12 Data line.
  • Wire the SDI-12 lines to the corresponding terminal blocks on the BM DevKit and see if the sensor is sending the same data frames.
  • Write a firmware app for the BM Mote (brains of the Bristlemouth DevKit) to send commands, receive and parse responses from the sensor to a readable format.
  • Spotter can receive sensor data via Bristlemouth and we can log it the the SD card and/or see it on the dashboard

End Goal

Able to integrate and communicate with an SDI-12 protocol capable sensor (Multi-parameter Sonde) via Bristlemouth.

:wave: Follow along as I work on integrating a new sensor to our Bristlemouth Ecosystem! Also feel free to add comments or post any questions!

8 Likes

This post documents how to wire and connect to the YSI EXO3s Multi-parameter Sonde. A note about EXO3s - it is the same as EXO3 but without a battery pack inside the housing.

Materials

Sensor Pinout and cable

For this section, I want to wire the SDI-12 lines from the sensor to an SDI-12 to USB adapter to make sure we have the pinout and wring right, and the sensor shows up as a USB port.

Based on the EXO3’s manual page 30, the wiring for native SDI-12 communication is GND on black and bare wire, Data on orange wire, 12VDC on red wire.

Unfortunately, I don’t have the flying lead cable that is mentioned in the manual. We have a similar “subconnector” cable available but the cable colors are most likely not the same as the flying lead cable.

To use this cable, I need the pinout of the of the male connector on the sensor side. The YSI support team provided us with the pinout details -

After using a multimeter and checking which color of the subconn conductor is which pin, I figured out which color of the subconn cable wires is which SDI-12 net. Here below:

net name pin cable color (flying lead) cable color (subconn)
SDI12 Data 1 orange black
GND 2 & 4 black, bare white, green
12V 3 red red

Wiring

3 Likes

Basic SDI-12 commands and responses

After wiring as mentioned above, I used pyserial-miniterm to find the sensor and connect to it (any serial terminal should work). I also added echo to view the commands I type to send. I noticed I receive the response immediately after typing ! at the end of the command to send.

Here is what I saw on the terminal. The EXO3s Multi-parameter Sonde shows up as a USB port with description:
/dev/cu.usbserial-D30FEU73 'FT231X USB UART - FT231X USB UART'

uma@Umas-MacBook-Pro-2 ~ % pyserial-miniterm -e
— Available ports:
— 1: /dev/cu.BLTH ‘n/a’
— 2: /dev/cu.Bluetooth-Incoming-Port ‘n/a’
— 3: /dev/cu.usbserial-D30FEU73 ‘FT231X USB UART - FT231X USB UART’
— Enter port index or full name: 3
— Miniterm on /dev/cu.usbserial-D30FEU73 9600,8,N,1 —
— Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
␀?!0
0!0
0I!013YSIIWQSGEXOSND100
0M!00629
0
0D!0
0D0!0+0.098+22.783+5.50+6.87
0D1!0-7.13+83.10+7.16+5.95
0D2!0+1.22+12.41
0D3!0

Breaking it down -

I used this website to look up SDI-12 commands since it’s easy to read.

command response interpretation
?! 0 Address Query: 0 is the address
0! 0 ACK: 0 is the address
0I! 013YSIIWQSGEXOSND100 Identification: 0 sdi-12 sensor address; 13 sdi-12 version number; YSIIWQSG vendor identification; EXOSND sensor model; 100 sensor version
0M! 00629 Measure command: 0 sdi-12 sensor address; 062 time is seconds until data available; 9 number of value to expect after 62 seconds (0 will be rxd to indicate measurement is done)
0D0! 0+0.098+22.783+5.50+6.87 Send Data command from channel 0: 0 sdi-12 sensor address; 0.098 value 1; 22.783 value 2; 5.50 value 3; 6.87 value 4
0D1! 0-7.13+83.10+7.16+5.95 Send Data command from channel 1: 0 sdi-12 sensor address; -7.13 value 5; 83.10 value 6; 7.16 value 7; 5.95 value 8
0D2! 0+1.22+12.41 Send Data command from channel 2: 0 sdi-12 sensor address; 1.22 value 9; 12.41 value 10?

Next Steps

The next step is understanding this data. Some questions that came up after getting this data –

  • What do the values mean?
  • In what order are they being sent?
  • Can I configure them and choose which reading is sent first?
  • Does the position of the sensors inside the sonde matter?

After having a little back and forth with the YSI support team, I understand I would need to use their Kor Software and along with their DCP Signal Output Adapter to configure the data frames being sent from the sonde. My next post will be about my discoveries using the Kor software + DCP SOA with the EXO3s.

2 Likes

Sonde Data format

Sensors inside the EXO3s Multi-parameter Sonde I have are

1. pH
2. Wiped Cond/T
3. Turbidity
4. Optical DO
5. Central Wiper

Kor Software

To figure out the value sequence being output from the EXO3s Multi-parameter Sonde, it is crucial to view the sensor on the Kor Software.

The sonde will be discoverable via bluetooth on the Kor Software on power up (flashing blue light). The sonde shows up like this on the software.

On the Deployments tab, I clicked on Create Template from Sonde. This opens a dialogue box with the current configurations.


The DCP ADAPTER OUTPUT subsection has the SDI-12 relevant settings. The SDI-12 slave address is 0 and the sequence of the values can be set here on the right side.

I made a note of this sequence for the example firmware app I am developing for this sensor.

Sequence of data values from the EXO3s

1. Temperature (° Celcius) 
2. Sp Cond (µS/cm)
3. pH
4. pH (mV)
5. Dissolved Oxygen (% Sat)
6. Dissolved Oxygen (mg/L)
7. Turbidity (NTU)
8. Wiper position (V)
9. Depth (m)
10. Sonde Cable Power (V)
1 Like

Using Bristlemouth Development board

Wiring the sonde to the Development board

Creating an example app for SDI-12

I made a branch in the bm_protocol repository - sdi12_example.
The SDI-12 Specifications state that the protocol needs these settings -

  • Transmissions ASCII characters
  • Baud rate needs to be set to 1200
  • Data Frame configurations
    • 1 start bit
    • 7 data bits
    • 1 even parity bit
    • 1 stop bit
  • Serial line is negative logic
  • Before sending a command, it needs a break (data line held HIGH for 12 ms) and mark (data line held LOW for 9 ms)

The next post will have more details on the code changes I made to add these specs!

2 Likes

SDI-12 protocol in the firmware app

I have created the library files required for the EXO3s communication - exo3s_sensor.cpp and exo3s_sensor.h along with user_code.cpp and user_code.h files in my branch based off of the serial_payload_example.

Setting the Baud rate to 1200

Within the init() in the exo3s_sensor.cpp, this is what I added to set the baud rate to 1200. BAUD_RATE is defined to be 1200.


  PLUART::setBaud(BAUD_RATE);

Data Frame configurations

By default, the settings for the LPUART are - 1 start bit, 8 data bits, no parity bit, 1 stop bit.
But we need to change this. The only successful way to configure it in this way is to set this in the Lower Level files - usart.c
(Note: This is not the right way to do it since I am editing a bsp, board support package, file)

Within the MX_LPUART1_UART_Init(), these are the configs I changed to set the LPUART to 1 start bit, 7 data bits, 1 even parity bit, 1 stop bit


  LPUART_InitStruct.DataWidth = LL_LPUART_DATAWIDTH_8B; // including the parity bit
  LPUART_InitStruct.Parity = LL_LPUART_PARITY_EVEN;

Serial line is negative logic

The data line is negative logic and it has to be inverted. I added this in the same MX_LPUART1_UART_Init() function.


LL_LPUART_SetTXPinLevel(LPUART1, LL_LPUART_TXPIN_LEVEL_INVERTED);

Break and Wake command

Before sending a command, the protocol expects a break (data line held HIGH for 12 ms) and mark (data line held LOW for 9 ms). This is a pretty crucial step and took me some time to figure out. After some debugging and using the Saleae logic analyzer, this was the way to achieve that.
For a short amount of time, the TX pin needs to multiplexed to be a GPIO output after disabling UART. Setting TX pin to HIGH for 12 ms break, followed by setting it to LOW for 9 ms and then multiplexing the TX pin back to GPIO ALTERNATE.

void SondeEXO3sSensor::sdi_break_mark(void) {
  flush();
  uint32_t timeStart;
  PLUART::disable();
  //Set TX pin to output
  PLUART::configTxPinOutput();
  // HIGH at TX pin
  PLUART::setTxPinOutputLevel();
  // Break - hold HIGH for 12 ms
  timeStart = uptimeGetMs();
  while(uptimeGetMs() - timeStart < breakTimeMin);
  // LOW at TX pin
  PLUART::resetTxPinOutputLevel();
  // Mark - hold LOW for 9 ms
  timeStart = uptimeGetMs();
  while(uptimeGetMs() - timeStart < markTimeMin);
  // Set TX pin back to TX (alternate) mode
  PLUART::configTxPinAlternate();
  // Re-enable UART
  PLUART::enable();
}

This sdi_break_mark() is used in the sdi_transmit() funciton before sending each command.

void SondeEXO3sSensor::sdi_transmit(const char *ptr) {
  // TX enable
  PLUART::startTransaction();
  sdi_break_mark();
  PLUART::write((uint8_t *)ptr, strlen(ptr));
  // TX disable
  PLUART::endTransaction(100); 
}

Here, startTransactions() and endTransactions() is to enable and disable the OE line (output enable).

SDI-12 Commands and Responses

  • Inquiry Command - is sent to get the sensor address

    sdi_transmit("?!");
    
  • Identification Command - is sent to get the identification string from the sensor

    sdi_transmit("0I!");
    
  • Measure Command - is sent to ask the sensor to start measurements.

    sdi_transmit("0M!");
    

    Sending this command, we immediately receive the a response 00629\r\n. This means it would take 62 seconds to send 9+1 values. After 62 seconds have elapsed, the sensor send a 0 to convey the measurement is done and new data can be inquired.

  • Data Command - is sent to retrieve the new data.

    sdi_transmit("0D0!");
    

    The maximum values the EXO3s sensor can send in a single frame is 4. Therefore, the next few values can be inquired by using the following commands.

     sdi_transmit("0D1!");
     sdi_transmit("0D2!");
    
2 Likes

In this post, I am adding screenshots of how the SDI-12’s data line looks when a command is sent and the response it receives on the same data line. I’m using the Saleae Logic Analyzer with it’s Logic 2 software to do this.

Saleae Logic Analyzer filter settings

The Async Serial filter settings are - 1200 baud, 7 bits per transfer, 1 stop bit, 1 even parity bit, LSB first, Signal Inverted

Command to get the slave address - ?!

Response is 0\r\n. 0 is the sensor address.

Command to get the identification string - 0I!

Response is 013YSIIWQSGEXOSND100\r\n.
0 sdi-12 sensor address;
13 sdi-12 version number;
YSIIWQSG vendor identification;
EXOSND sensor model;
100 sensor version;

Measure Command - 0M!

Response is 00629\r\n.
0 sdi-12 sensor address;
062 time is seconds until data available;
9+1 number of value to expect after 62 seconds;

After 62 seconds, the sensor sends over a 0\r\n to indicate the measurement is done. And now we can inquire values using the data commands.

Data Commands - 0D0!, 0D1!, 0D2!

Response to 0D0! is 0+21.501+5.88+7.17-24.00\r\n.

Response to 0D1! is 0+102.62+9.06+13.56+1.22\r\n.

Response to 0D2! is 0+0.251+12.12\r\n.

3 Likes

For this section where I mentioned I had to edit a bsp file, I found a better way to do it where we don’t have to touch the bsp usart.c file.

Resolved this by adding 2 functions in payload_uart.cpp to set the even parity bit and invert the data. I needed to disable LPUART, configure the parameter and enable LPUART —

void setEvenParity(void){
  LL_LPUART_Disable(static_cast<USART_TypeDef *>(uart_handle.device));
  LL_LPUART_SetParity(static_cast<USART_TypeDef *>(uart_handle.device),
                      LL_LPUART_PARITY_EVEN);
  LL_LPUART_Enable(static_cast<USART_TypeDef *>(uart_handle.device));
}

void enableDataInversion(void){
  LL_LPUART_Disable(static_cast<USART_TypeDef *>(uart_handle.device));
  LL_LPUART_SetTXPinLevel(static_cast<USART_TypeDef *>(uart_handle.device),
                       LL_LPUART_TXPIN_LEVEL_INVERTED);
  LL_LPUART_Enable(static_cast<USART_TypeDef *>(uart_handle.device));
}
1 Like

Defining struct based on the sensors in the EXO3s MP Sonde

The EXO3s Multiparameter Sonde I am workign with is installed with pH, Wiped Condensation/Temperature, Turbidity, Optical Dissolved Oxygen and Central Wiper sensors inside. And the Kor Software helps define the sequence of the values on the SDI-12 data frames (as mentioned in the post above)

Therefore, the struct in the sdi12_example app is defined accordingly –

  • In exo3_sensor.h file,
    struct __attribute__((packed)) EXO3sample {
      float temp_sensor;    // Celcius
      float sp_cond;        // uS/cm
      float pH;
      float pH_mV;          // mV
      float dis_oxy;        // % Sat
      float dis_oxy_mg;     // mg/L
      float turbidity;      // NTU
      float wiper_pos;      // volt
      float depth;          // meters
      float power;          // volt
    };
    
  • In exo3_sensor.cpp file, within sdi_cdd() case 2, the 3 data frames’ responses are populated to the appropriate struct variables —
    • 0D0!

           _latest_sample.temp_sensor = (float) d0_0.data.double_val;
           _latest_sample.sp_cond = (float) d0_1.data.double_val;
           _latest_sample.pH = (float) d0_2.data.double_val;
           _latest_sample.pH_mV = (float) d0_3.data.double_val;
      
    • 0D1!

           _latest_sample.dis_oxy = (float) d1_0.data.double_val;
           _latest_sample.dis_oxy_mg = (float) d1_1.data.double_val;
           _latest_sample.turbidity = (float) d1_2.data.double_val;
           _latest_sample.wiper_pos = (float) d1_3.data.double_val;
      
    • 0D2!

           _latest_sample.depth = (float) d2_0.data.double_val;
           _latest_sample.power = (float) d2_1.data.double_val;
      
2 Likes

Connecting the Dev Board to the Spotter

Spotter + Dev board Setup

I disconnected to 24V power supply to the Dev Board stack and instead wired it to the spotter using the 2 conductor cable.
To set up the Spotter, I needed to adjust some configurations to ensure it could continuously power the Dev board stack for development purposes.

Screened into the Spotter via pyserial-miniterm and viewed the configs by typing bm cfg status 0 s (0 is the node ID) –

bm cfg status 0 s
Succesfull status request send
2024-11-04T19:17:56.179Z [BRIDGE_CFG] [INFO] Response msg -- Node Id: 0000000000000000, Partition: system, Commit Status: 0
2024-11-04T19:17:56.179Z [BRIDGE_CFG] [INFO] Num Keys: 15
2024-11-04T19:17:56.179Z [BRIDGE_CFG] [INFO] Key 0: bridgePowerControllerEnabled
2024-11-04T19:17:56.179Z [BRIDGE_CFG] [INFO] Key 1: sampleIntervalMs
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 2: sampleDurationMs
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 3: dfu_confirm
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 4: subsampleIntervalMs
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 5: subsampleDurationMs
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 6: subsampleEnabled
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 7: alignmentInterval5Min
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 8: samplesPerReport
2024-11-04T19:17:56.183Z [BRIDGE_CFG] [INFO] Key 9: transmitAggregations
2024-11-04T19:17:56.187Z [BRIDGE_CFG] [INFO] Key 10: currentReadingPeriodMs
2024-11-04T19:17:56.187Z [BRIDGE_CFG] [INFO] Key 11: softReadingPeriodMs
2024-11-04T19:17:56.187Z [BRIDGE_CFG] [INFO] Key 12: rbrCodaReadingPeriodMs
2024-11-04T19:17:56.187Z [BRIDGE_CFG] [INFO] Key 13: ticksSamplingEnabled
2024-11-04T19:17:56.187Z [BRIDGE_CFG] [INFO] Key 14: turbidityReadingPeriodMs
2024-11-04T19:17:56.195Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 0
2024-11-04T19:17:56.203Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 3600000
2024-11-04T19:17:56.210Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 310000
2024-11-04T19:17:56.214Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 0
2024-11-04T19:17:56.222Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 60000
2024-11-04T19:17:56.230Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 30000
2024-11-04T19:17:56.238Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 0
2024-11-04T19:17:56.242Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 1
2024-11-04T19:17:56.250Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 1
2024-11-04T19:17:56.257Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 1
2024-11-04T19:17:56.261Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 60000
2024-11-04T19:17:56.269Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 500
2024-11-04T19:17:56.277Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 500
2024-11-04T19:17:56.281Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 0
2024-11-04T19:17:56.289Z [BRIDGE_CFG] [INFO] Node ID: 0000000000000000 Partition: system Value: 1000

We need to set bridgePowerControllerEnabled to be 0 to constantly power the dev board stack -
bm cfg set 0 s u bridgePowerControllerEnabled 0

Retrieving raw data

I used the curl command to query raw data from the API

curl "https://api.sofarocean.com/api/raw-messages?spotterId=SPOT-30824C&startDate=DATET00:00:00Z" -H 'token: YOUR-API-TOKEN' | jq .

I installed the jq package to display the raw data in a more readable format. Fill out the YOUR-API-TOKEN and DATE before running this command.

Raw data from the BM payload will have a header de and an example will look like this —

{
        "momsn": null,
        "transmitTime": "2024-11-04T18:40:51.000Z",
        "approxLon": "-122.39",
        "approxLat": "37.79",
        "cep": null,
        "message": "de672915349f7ca0123536d80006d40000caf3e2ef0ef2e488fabb7c18ba49a54185ebd1407b14e640b81ec7c1c375cd421f851341713d064148e19a3f3d0a573e85eb4141ee7ca5417b14ce40f628e4405c8facc1a470cd421f85134152b8024148e19a3f6210583e85eb4141fca9a541ae47c9400ad7e3406666a8c1c375cd42295c13419a990d41f6289c3fac1c5a3e85eb4141",
        "createdAt": "2024-11-04T18:41:13.396Z",
        "messageHeader": 222,
        "counter": null,
        "partNo": null,
        "processed": true,
        "commSource": "cellular",
        "processingSource": "embedded"
      }

Here the raw data is de672915349f7ca0123536d80006d40000caf3e2ef0ef2e488fabb7c18ba49a54185ebd1407b14e640b81ec7c1c375cd421f851341713d064148e19a3f3d0a573e85eb4141ee7ca5417b14ce40f628e4405c8facc1a470cd421f85134152b8024148e19a3f6210583e85eb4141fca9a541ae47c9400ad7e3406666a8c1c375cd42295c13419a990d41f6289c3fac1c5a3e85eb4141

Decoding the raw data

I made a simple script to interpret and decode the raw data. Here is the script - exo3_sdi12_decoder.py.
I have an example raw data stored in a variable called payload in the script. The raw data is decoded and displayed –

(bristlemouth) uma@Umas-MacBook-Pro-2 sdi12_example % python3 exo3_sdi12_decoder.py
- Detection data:
	temp_sensor: 22.9689998626709
	sp_cond: 5.769999980926514
	pH: 7.090000152587891
	pH_mV: -19.649999618530273
	dis_oxy: 103.73999786376953
	dis_oxy_mg: 8.899999618530273
	turbidity: 13.399999618530273
	wiper_pos: 1.2100000381469727
	depth: 0.30399999022483826
	power: 12.119999885559082
---------------------------------

- Detection data:
	temp_sensor: 22.97800064086914
	sp_cond: 5.829999923706055
	pH: 7.090000152587891
	pH_mV: -19.670000076293945
	dis_oxy: 103.73999786376953
	dis_oxy_mg: 8.899999618530273
	turbidity: 13.220000267028809
	wiper_pos: 1.2000000476837158
	depth: 0.30399999022483826
	power: 12.119999885559082
---------------------------------

- Detection data:
	temp_sensor: 22.98699951171875
	sp_cond: 5.650000095367432
	pH: 7.090000152587891
	pH_mV: -19.579999923706055
	dis_oxy: 103.7300033569336
	dis_oxy_mg: 8.899999618530273
	turbidity: 13.619999885559082
	wiper_pos: 1.2100000381469727
	depth: 0.3050000071525574
	power: 12.119999885559082
---------------------------------
5 Likes