GPU Support

tip

See the GPU Models guide for more information.

Sometimes leveraging a CPU to make predictions is not performant enough. This is often the case when dealing with a large volume of high-resolution images. To solve this problem, navio supports GPU-accelerated inference.

info

Only NVIDIA GPUs are currently supported.

If navio has been set up with GPU support enabled, an extra icon is added to the models in the list on Model Management view.

A model can use multiple GPUs but only one model can make use of a GPU at a time. Deactive the model to free up the GPU for another model.