Support external Edge Agent drivers #316
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We don't have a lot of experience using CMD all the way to a device in Factory+. In general supporting device CMDs is only possible when an address can be mapped straight to a single metric, with no 'path' component.
OPCUA and MTConnect support automatic polling within the protocol. S7 supports group queries. It is a good idea to allow drivers to be efficient where they can be. Also support an overall `active` topic for a driver, indicating whether the relevant Edge Agent connection is online. This also serves as a prompt for the driver to republish its status when the Edge Agent starts up.
Define a protocol for communication between the Edge Agent and drivers. This runs over MQTT, with the EA serving as broker. The topic structure is defined by the protocol; this is not a generic MQTT driver. The protocol allows the EA to poll the driver, and also allows the driver to produce data asynchronously.
This makes TypeScript a bit more bearable.
We are using the Aedes broker library. Create a broker and perform auth on clients. The broker URL to listen on and the location of the passwords for the drivers are taken from the environment rather than from the config as these will need to be synchronised with the driver deployment. This means they are Helm values rather than edge agent config items.
The messages from a driver are too complicated to try to emit as positional arguments; emit an object instead. This means we can pull out and validate the data topic name too.
These tasks are too small to go in Jira, and some will probably outlive this project.
Device represents a Sparkplug Device, and clearly lives on the Edge Agent side of the divide. It should not have any connection-specific functionality. Where there was specific code, this was concerned with arranging to subscribe to addresses on the southbound connection. Handle this in the Connection class instead.
This sits alongside the other connection types, for now, and speaks my new driver protocol. This doesn't handle CMDs or errors correctly yet, but otherwise works. I'm not entirely sure how to handle drivers that want to poll themselves. Currently the EA keeps sending poll messages regardless, and drivers like OPCUA can just not subscribe, and use the group poll instruction instead. I have disabled the watchdog timeout; it fires much too quickly when you're pretending to be a device manually. I'm not sure a fixed timeout with no ping is the right thing to do anyway; there are valid reasons why a connection might not speak for 10s.
This sits alongside the other connection types (for now) and communicates with an external driver using the new protocol. Configuration, polling and data packets are implemented. Command packets and error handling is not. The Device subclasses have been removed. Where they had functionality this has been moved into the DeviceConnections. The Devices now purely handle the Sparkplug side of things, leaving the DeviceConnection to handle the southbound details.
This accepts addresses specifying fixed functions. We need to supply a `version` in the address config.
This doesn't handle errors entirely properly yet, but it works.
We need to emit "open" and "close" on the Driver connection, but only if this is a change of status.
This class will only handle drivers which can't do their own polling, and ignore the address groups. Drivers like OPCUA will need a different implementation. This driver is also careful to always poll serially. Some drivers might be able to handle parallel polls, and we should allow this. (Modbus can't.)
We want to maintain a continuous connection to the modbus device. We also want to alert the edge agent promptly if we lose our connection. Handle reconnection via connection events rather than waiting for a poll. Delay 5s before attempting to reconnect, otherwise we may enter a fast loop.
These drivers do not need the GSSAPI code, so should not need our customised base images.
* Implement a driver for testing purposes. * Implement a modbus-tcp driver. This could be adapted to support modbus-serial fairly straightforwardly. * Factor out a JS library for writing edge drivers.
We are going to keep Devices limited to a single Connection.
This requests krbkeys to create and update a random value in a local Secret. Edge Agents will use this to communicate with on-cluster drivers.
Give the krbkeys operator the ability to create secrets which only exist on an edge cluster, not connected to a Kerberos principal. These can be used to authenticate edge drivers to the edge agent. Request these secrets using a new LocalSecret CRD.
* Give the EA access to a Secret for the driver passwords. This will need to be populated by krbkeys, I think. * Create a Service for the EA broker.
Given that all drivers are deployed identically, and that in general we want to run them on the same host as the edge agent, deploy them as part of the edge agent pod and from the same Helm chart. This does not preclude external drivers, but would need additional configuration to set up external IPs and to listen to the wildcard address.
Include driver images in the `image` values item. Pull them in by name instead of with a full image name. Support defaults for registry and repository. Pass a VERBOSE environment variable to the drivers.
For now we have `externalTrafficPolicy: Local` which will enforce that external drivers have to connect to the host the edge-agent is running on. K8s does allow us to be more flexible than that but I'm not sure if we want to.
Deploy Edge Agent drivers from within the edge-agent chart. There is no need for a separate chart as drivers all deploy identically. At the moment the drivers deploy as containers on the edge agent pod. This keeps things simple and a bit more secure and means on-cluster drivers can contact the edge agent on `localhost` without needing to go through a Service. If `externalIPs` is included in the values then the driver interface will be available externally to support off-cluster drivers. Authentication for such drivers will need to be handled through SealedSecrets and may need a little more work.
AlexGodbehere
previously approved these changes
Aug 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is currently no Manager support for drivers. This will need more work.