This documentation is for the upcoming version of the AI Manager
Read the documentation for the current version
Ctrlk
AI ManagerNx Meta
For the complete documentation index, see llms.txt. This page is also available as Markdown.

AI Certification Test

Run the AI Certification test to verify a device is compatible with the AI Manager.

Quickstart

The AI Certification test runs on any Ubuntu installation that is also compatible with Network Optix Server.

Ensure your device has at least a few gigabytes of free space, a working internet connection and a working Python 3 and Pip installation.

Python requirements
# on some systems you may need to install pip
# for example with the following command on ubuntu
sudo apt update
sudo apt install python3-pip

If you are testing a GPU or NPU, also make sure that the required drivers are installed, for example the Jetpack drivers for NVIDIA accelerators.

Execute the following commands:

Download tests
## Create and enter folder for test
mkdir nxai_test
cd nxai_test

## Download testing suite
wget https://artifactory.nxvms.dev/artifactory/nxai_open/NXAITest/nxai_test.tgz

## Unpack testing suite
tar -xvf nxai_test.tgz

The python3 Utilities/install_acceleration_library.py command automatically detects the available hardware acceleration on the device. If the script finds more than one option, it pauses execution and prompts for a choice. To run the command without pausing, add the accelerator name, such as in the "Nx CPU" example.

The download_models.py command downloads approximately 3.5GB of model data; download time varies with connection speed.

The test runs for a couple of hours and stresses the device. Do not power off the device.

Introduction

The AI Certification Test verifies that a device is compatible and stable enough to run the AI Manager for extended periods.

The test runs multiple common model architectures, with different sizes and multiples of those models, to check compatibility and expected performance on the device. The test also runs long-duration operations to check for memory issues or performance degradation caused by overheating or other factors.

Finally, the test gathers all results in a folder that can be used to generate a report, either on the device or elsewhere. The report contains enough information to determine whether the device is compatible.

Installing AI Manager

A script installs the AI Manager runtime locally within the test environment, so testing does not interfere with any existing installations on the device.

Installing Nx Acceleration Library

Acceleration libraries act as layers between the AI Manager and the acceleration hardware. Any device can run the tests on CPU, without acceleration hardware. A device with AI acceleration hardware, such as an Nvidia CUDA device or a Hailo AI chip, uses it to accelerate the AI pipeline.

Running this script automatically detects available hardware. If more than one option is found, the script presents a list to choose from.

Download Models

The Certification Test checks whether a variety of models run on the device and libraries. After installing the acceleration library, download the correct models for the device.

This step could use a lot of data as many large files need to be downloaded. This might also take a long time depending on network conditions.

Running Test

The AI Certification test includes a collection of tests that check different aspects of the device to confirm the AI Toolkit runs on it. To run all tests together, run:

This takes several hours to complete.

Gathering Hardware Information

This command detects hardware information about the device and packages it with the test results, giving context for better analysis of the device's performance.

This command prints the ID of the device. Use this ID to identify the device when the test results are uploaded. Run this command again at any time to retrieve the ID.

Uploading Results

An endpoint accepts uploads and displays the test results for the device. After the test completes, run the following command to upload the results to the cloud:

Custom Model Benchmark

The Custom Model Benchmark can only be used to benchmark models which have been uploaded to the AI Manager Cloud.

The test allows you to benchmark your own models on different devices. The Certification Test includes the functionality to add your own models to its benchmark test. If you already have that downloaded, you can skip downloading and extracting the benchmark test. If you are only interested in benchmarking a model, you can instead download the smaller benchmark package:

Install an AI Manager with runtime by following the steps at Installing AI Manager

Next, add as many models as you want to benchmark by running:

And entering the ID of the model you want to test. The ID of your model can be found by navigating to your model in the model cloud. For example:

After you have successfully added all the models you want to benchmark, you can download the model files from the cloud by running:

Once all your models have been downloaded, start the benchmark by running:

The test benchmarks all added models and gives an overview of how the model/device performs.

Troubleshooting

If the device is working well but the test is not passing, contact support. Include log files so support staff can identify what is going wrong on the device.

Make sure you're in the root folder of the test suite and run the following command to gather all the log files:

This creates a file called test_logs.tgz. Send this file with the support request.

PreviousAPI Documentation

Last updated