Skip to main content

Edge Deployments

navio has partnered up with TTTech Industrial to offer users the ability to deploy their machine learning models directly to edge devices. Integrated with the Nerve Management System, users can now directly execute their models on their connected Nerve devices. Connect to data streams produced by machines or sensors via the Nerve’s data gateway and use these as input to get predictions in real-time.

This enables users to reduce latency and improve performance by deploying the model directly to where the data is being produced.

navio is able to deploy the machine learning models to any kind of Azure IoT Edge device through Azure IoT Hub. For details checkout and its components checkout the Azure IoT deployment guide.

Prerequisites​

In order to make use of edge deployments, a user requires access to a Nerve management system as navio interacts with its API to manage the edge devices. In addition to this, a user requires at least one Nerve edge device to deploy the model to.

note

Get in contact with us if you would like to inquire about Nerve devices.

A docker registry is also required for serving the model containers to the Nerve management system.

Connecting a Nerve Management System​

  1. Go to workspace -> workspace settings -> Edge Management Systems

Here you can see your available and connected management systems that are used to manage the hardware devices on which you want to deploy a model.

  1. Click connect

The following dialog appears:

edge-management-system-add

  1. Put in your management system credentials and click Test Connection

If everything works fine, you'll get a green checkmark to let you know navio succesfullty connected.

  1. Click save

Now your management system will appear in the list of connected management systems.

Now you need to connect to your docker registry.

Connect to a docker registry​

A docker registry is needed for storing model containers and it is where the management system will pull the navio models from.

  1. Go to workspace -> workspace settings -> Docker registry

  2. Put in your docker registry credentials and click save

Now you're ready to start using edge deployments!

Managing Edge Devices​

Once the management system is set up, you can see the connected edge devices for a workspace by clicking on All edge devices in the left navigation bar.

Note that the status of the edge devices could be outdated as it's periodically polled. To fetch the latest state, press the sync button at the top of the page.

Deploying to an Edge Device​

  1. Go to the Deployments view in a use case of your choice.
  2. Click on New deployment
  3. Name your deployment, choose your model and select type *On edge" and click next
  4. Select a connected management system and navio will fetch the available edge devices.
    note

    Edge devices that are offline or unreachable will not show up in the list

  5. Select the devices you want to deploy to and click next.

edge-deployment

  1. Specify the MQTT properties

Currently, all models that are deployed via an edge deployment, communicate via the messaging protocol MQTT rather than REST over HTTP.

For this to work, some properties need to be defined:

MQTT Broker - the URL of the broker that the deployed container will communicate with. This is often deployed internally on a device.

Publish Topic - the topic to which the model will publish its predictions

Subscribe Topic - the topic the model will subscribe to, to fetch its input

  1. Click save

Your model will now be deployed on the selected edge devices!

note

Deploying a model to an edge device can take a couple of minutes to complete, please wait for the action to be completed.