Hi all, Charles Cross from Mission Robotics here.
I wanted to share with everyone an exciting new project within the Bristlemouth ecosystem that I am currently prototyping. As part of the effort to help drive the development and expansion of the core Bristlemouth protocols, I have embarked down the path of creating a native Linux C++ implementation of Bristlemouth.
Background
As a bit of background, at Mission Robotics, we focus primarily on marine vehicle software solutions, and Linux-focused single-board computers (like the Nvidia Jetson series of processors) are always at the core of our designs, whether they are serving as central processors, or supporting advanced peripheral functions like camera processing.
Part of our vision for the Bristlemouth standard is that specialized compute hardware and intelligence can be pushed into individual devices/peripherals, rather than requiring peripherals to be connected to a beefy, central processor which may not even have the ideal amount of I/O to connect everything you might want (especially as your vehicle design evolves). Given Bristlemouth’s ability to provide both power and networking, we see it supporting the natural evolution towards a more modular, scalable, and distributed future, with regards to vehicle architectures and marine sensing in general.
An important piece of this equation, of course, is providing support for Bristlemouth within the context of more powerful application processors, in contrast to microcontroller targets like the STM32 MCU used within the Bristlemouth DevKit Motes.
bm_serial
For the adventurous (or studious) among you, you may be aware that some level of non-mote support is currently provided by the bm_serial library. For context, bm_serial provides an abstracted, simple Bristlemouth interface for an external user device (such as an Arduino, RaspberryPi, etc) within the Bristlemouth network. The user device can perform functions like publishing or subscribing to topics over a serial port connected to a Bristlemouth Mote, and the Mote will perform all of the core networking functions (like routing, discovery, etc) on its behalf. From the perspective of all other Nodes in the Bristlemouth network, it is the Mote that expresses interest in topics or publishes the data the user device is sending, and the fact that the user device exists is mostly hidden from them.
There are some advantages and disadvantages to this approach:
Pros:
- The interface is very simple. With a small code footprint, it is very easy to interact with the Bristlemouth network from devices with extremely small amounts of RAM/CPU, like Arduinos.
- The amount of responsibilities the user device has to perform are minimal. The Mote can do all of the heavy lifting, and less code needs to be written for the user device.
- Porting to new platforms/OSes/architectures is easier - all you need is support for serial comms.
- A misbehaving or non-functional user device is generally isolated from the rest of the network, and will not jeopardize the overall performance of the system.
Cons:
- Serial comms typically run at speeds between 115 KBaud and 1 MBaud. While it is possible to get into the 5 MBaud region, it can be difficult, depending on the devices involved and the performance of available drivers. Because of this, the 10 Mbps throughput of the Bristlemouth network likely cannot be realized by the user device, and a bottleneck is created.
- Due to the slower physical transport speeds, and the fact that a mote has to relay and translate messages for the user device, additional latency and overhead is introduced in any comms the user device is involved in.
- The user device does not get the opportunity to interact on the network using IPv6 communication, and is restricted to the simplified interface that
bm_serialprovides.
To address these downsides, and unlock all of the capabilities of the BM protocol within more powerful processors, the plan is to introduce native support for application processors, starting with a Linux implementation.
What does this mean?
In general, a native Bristlemouth Node must be capable of a wide range of functions.
- Support for physically communicating via 10BASE-T1L over the Bristlemouth twisted-pair conductors (through a 10BASE-T1L PHY)
- Support for IPv6
- Support for multiple Ports (physical network interfaces - the DevKit mote has two, to allow daisy-chain topologies)
- Routing of incoming messages, both to local applications and other Bristlemouth Nodes on adjacent network segments
- All BCMP protocol features (discovery, resource info exchange, firmware/software updates, configuration, etc)
- Complete support for middleware functions (pub/sub, request/reply)
- And more…
In the current code-base (bm_protocol), these are implemented specifically for an STM32 component running FreeRTOS, using the LWIP IP stack, and an ADIN2111 2-port 10BASE-T1L MACPHY chip. Unfortunately, much of that work does not directly carry over to supporting Bristlemouth application development on ARM SBCs like the RaspberryPi and Jetson devices (especially if you want to develop Bristlemouth applications that can interact locally with non-BM applications like ROS2 nodes or existing codebases running on Linux).
Fortunately, the PHY hardware side of things is starting to be covered, thanks to a wider availability of 10BASE-T1L PHY components, most of which support either SPI or RMII host interfaces. The Linux kernel itself now has upstream support for the ADIN 10BASE-T1L components, making them available as normal network interfaces. Building on top of this, we were able to get to a working first prototype.
Towards a prototype
One of the products available within the Analog Devices ADIN product family is the EVAL-ADIN1100 10BASE-T1L to 10BASE-T media converter board. This board has an RJ-45 ethernet port that you can connect to with your computer, and a two-wire output that can be connected directly to the Bristlemouth network. It has a built-in microcontroller loaded with firmware that takes in ethernet frames on the 10BASE-T port and directly converts and forwards them to the ADIN1100 10BASE-T1L PHY, and vice-versa.
Below is a picture of my test setup, wherein I connect the EVAL-ADIN1100 board to one of the ports on a Bristlemouth Mote. From there, I connect my laptop to the EVAL-ADIN1100 via a USB-C-to-ethernet adapter (with a second USB micro cable to power the EVAL-ADIN1100 board).
The EVAL-ADIN1100 board, being for evaluation and development purposes, is not exactly optimized for size or cost. I’ve placed a RaspberryPi 4 beside it to give a sense of scale for what we’re working with. Eventually, you would expect to replace the EVAL-ADIN1100 electronics with another Mote-sized board. In the case of a RaspberryPi, you could even imagine this being a shield/hat.
On the software front, I have a few goals that I am trying to achieve:
- A solution that works from user-space using standard network interfaces within Linux
- A solution that does not require any modifications to the kernel, or the compilation/installation of additional kernel modules or modification to the Device Tree
- The ability to ensure that no traffic from non-Bristlemouth applications finds its way onto the Bristlemouth network (to prevent pointless traffic on the network from consuming bandwidth).
- The ability to have complete control over the IPv6 packet construction, such that we can implement the required modifications that the Bristlemouth protocol takes advantage of to implement neighbor discovery, topology introspection, and resource-based packet routing.
- Eventually (but not immediately) build towards a Bristlemouth library that implements the core BM protocol logic in a platform-agnostic manner, with platform-specific implementations of low-level functions like tasks, sockets, memory allocation, etc separated from the business logic. The Zenoh-Pico middleware library is a good example of this approach.
With the above goals in mind, my approach involves:
- Using the Linux Raw Sockets API, which allows you to craft complete Ethernet II frames and interact with a specific network interface (i.e. ‘eth0’ or some such). In this manner, you create a socket, much like any Posix socket application, and bind it to a specific network interface, then use send()/recv() and friends to interact with the network device.
- Placing the interface in promiscuous mode, as I found that my ethernet network devices were apt to drop packets for some of the multicast MAC addresses used within the BM Protocol (like 33:33:00:02:00:01). By putting them in promiscuous mode, I was able to ensure that the packets weren’t dropped by the interface and could make it to the network stack and then userland.
- Using Linux network namespaces to isolate the network device from other applications in the system. This stops certain types of traffic from leaking onto the interface, like broadcast and multicast packets from various services and discovery protocols.
- Applying a few additional routing table rules to prevent built-in IPv6 traffic generated by the kernel itself from leaking onto the network (like ICMPv6 neighbor discovery, which we replace with BCMP neighbor discovery).
So far, I have built out the most basic building blocks that demonstrate the feasibility of the first four goals, starting with the implementation of the most fundamental component of the Bristlemouth protocol - support for sending/receiving BCMP packets (a specialized IPv6 protocol value, 0xBC, that sits at the same level as ICMPv6), and specifically the ability to send and receive BCMP Heartbeat messages, which facilitate neighbor discovery between two Bristlemouth devices. Below is a (terrible) screenshot of the first establishment of bi-directional communication between my laptop and a BM mote running firmware 0.6.0 with the hello-world application.
On the left is my simple test application which sends heartbeats once a second (and processes incoming heartbeats), and on the right is a serial connection to my BM mote and the result of running the bm neighbors command which prints the list of currently discovered neighbors. On the left, we see that the Linux application has received a BCMP heartbeat from the mote, and inn the last line of output on the right, we see that the Mote has discovered our Linux application on it’s Port 1 (our Linux app’s 64-bit Node ID is 8aa5f9b372a6080c in this example).
Next Steps and Future Ideas
From here, the next steps involve building out the rest of the core BCMP functions, followed by implementation of the Bristlemouth Middleware protocol that allows pub/sub and req/rep communication centered around topics/resources. By going through this process, we also hope to uncover all of the aspects of the FreeRTOS-based implementation that need to be restructured to build towards the platform-agnostic library solution, such that we land on the right code structure and user-experience, and ultimately get to a single code-base/library that does not need to be redundantly maintained to ensure compatibility across different platforms.
Another goal that I have, once the core features are in place, is the ability to transparently bridge all of the discovered topic endpoints into other pub/sub middleware ecosystems, such as Zenoh, ROS2, or DDS, as these are commonly used within robotics ecosystems for both internal and external communications. Bristlemouth has been designed from the beginning to be concept-compatible with these protocols and frameworks to enable the merger of both traditional networks and existing code-bases with the new capabilities provided by the Bristlemouth standard.
Feedback, questions, and interest
If any of this work is of interest to you or your organization, please let us know! We are actively interested in feedback about what this library could evolve to look like. If there are certain constraints or aspects around how it takes shape (like choosing a specific language to implement the library in), or if there are concerns around the current proposed approach, please feel free to share so we can take everything into consideration.
That said, I will continue to share updates here as progress is made!
