> For the complete documentation index, see [llms.txt](https://nx.docs.scailable.net/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://nx.docs.scailable.net/ai-manager-v6.1.4/ai-accelerators-support/nvidia-support.md).

# NVIDIA Support

| Support Level | AI Accelerator | CPU Architecture | Operating System       | API/driver version              |
| ------------- | -------------- | ---------------- | ---------------------- | ------------------------------- |
| NVIDIA dGPU   | Full support   | x86\_64          | Ubuntu 20+ and Windows | CUDA 11, CUDA 12, CUDA 13       |
| NVIDIA dGPU   | Full support   | aarch64          | Ubuntu 24.04           | CUDA 13                         |
| NVIDIA Jetson | Full support   | aarch64          | Ubuntu 20+             | JetPack 5, JetPack 6, JetPack 7 |

AI models inference is available on a wide-range of NVIDIA GPUs whether it is consumer-grade like GeForce RTX 3060, data center-grade like A100 and DGX Spark, or embedded devices (Jetson platform).

To enable inference on the GPUs using the AI Manager, install compatible NVIDIA drivers and CUDA toolkit SDK versions based on the [compute capability](https://developer.nvidia.com/cuda-gpus) of the GPU.

{% hint style="warning" %}
AI Manager supports only GPUs compatible with **CUDA Toolkit 11** or higher.
{% endhint %}

{% hint style="info" %}
To determine the minimum CUDA version required, check the [CUDA GPU support table](https://en.wikipedia.org/wiki/CUDA#GPUs_supported) and the [cuDNN support matrix](https://docs.nvidia.com/deeplearning/cudnn/backend/v9.14.0/reference/support-matrix.html). Install the latest version compatible with the GPU.

To determine which GPU driver version to install based on the desired CUDA version (or vice versa), check the [CUDA toolkit release notes](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id6).
{% endhint %}

## Installation Procedure

{% hint style="info" %}
**Install GPU drivers and CUDA toolkit** ***versions*** **that are compatible with each other.**
{% endhint %}

## x86\_64 Windows

1. Go to <https://www.nvidia.com/en-us/drivers/>, and select your GPU model and OS. Then, download the driver installer, and run it.
2. Go to <https://developer.nvidia.com/cuda-toolkit-archive> and choose the CUDA version compatible with your installed GPU drivers.\
   (To find the recommended CUDA toolkit, run `nvidia-smi` on PowerShell. The version appears in the top right corner of the output.)\
   After a successful installation, the CUDA toolkit should be available at: `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\`

## x86\_64 Ubuntu

1. Install the GPU drivers by running this command: `sudo ubuntu-drivers autoinstall` , then reboot the machine.
2. Go to <https://developer.nvidia.com/cuda-toolkit-archive> and install the CUDA version compatible with your installed GPU drivers.\
   (To find the recommended CUDA toolkit, run `nvidia-smi` in a terminal window. The version appears in the top right corner of the output.)

## Verifying the Runtime Is Available

After installing the drivers and CUDA toolkit and restarting the Mediaserver, the NVIDIA runtime appears in the plugin's runtime selection list. Open any camera's settings, go to the **Integrations** tab, click **AI Manager**, and confirm that the NVIDIA runtime is listed.

<figure><img src="/files/3l9eveFst4JGVgdidJF8" alt="AI Manager plugin runtime selection panel listing available runtimes including NVIDIA CUDA"><figcaption><p>The NVIDIA runtime appears in the list once drivers and CUDA are installed and the server has been restarted</p></figcaption></figure>

## Troubleshooting

On systems using NVIDIA's JetPack SDK, especially recent installations, the `networkoptix-metavms` user might not be automatically added to the `render` group, which is required for the AI Manager plugin to use NVIDIA GPU acceleration. Add the user manually with the following steps:

## Step 1: Check If the `render` Group Exists

First, verify whether the `render` group exists on your system:

```bash
getent group render
```

* **Expected Output**

  If the `render` group exists, the output resembles:

  ```
  render:x:104:username
  ```

  This indicates that the group exists and lists the users currently in the group.
* **No Output**

  If there's **no output**, the `render` group does not exist on your system. In this case, there's no need to continue with the next steps.

## Step 2: Add `networkoptix-metavms` to the `render` Group

* Run the following command to add the user to the `render` group:

  ```bash
  sudo usermod -aG render networkoptix-metavms
  ```

## Step 3: Verify the Group Membership

Confirm that the `networkoptix-metavms` user has been added to the `render` group:

```bash
groups networkoptix-metavms
```

* **Expected Output**

  The command lists all groups the user belongs to, including `render`.

## Step 4: Restart the Network Optix Service

For the changes to take effect, restart the Network Optix Mediaserver service:

```bash
sudo systemctl restart networkoptix-mediaserver.service
```

* This command restarts the service, allowing it to recognize the updated group permissions.

{% hint style="info" %}
If the AI Manager does not work even when the conditions above are met, see the [general troubleshooting section](/ai-manager-v6.1.4/support-and-troubleshooting/how-to-get-support.md).
{% endhint %}