MemryX Support
Summary
MX3
Experimental
x86_64
Ubuntu 22.04, Ubuntu 24.04
2.1.0
Introduction
About MemryX
[Action needed: Please insert the MemryX company logo and a brief overview of the company and its AI chip(s) here. This section should clearly introduce the MemryX MX3 accelerator.]
Workflow
The Nx AI Manager simplifies the process of securely deploying your AI models on MemryX MX3 devices with the following steps:
Install all necessary software dependencies.
Install the Nx Meta Server on your machine and choose the MemryX runtime option.
Upload your model to the Nx AI Cloud for compilation.
Compile and install the required post-processor.
Set up your inference pipeline.
Hardware and Software Requirements
Hardware Requirements:
[Action needed: Specify the required hardware, such as minimum RAM, required CPU cores, and necessary disk space.]
Software Requirements:
Ubuntu 22.04 LTS.
[Action needed: List any other required software components or libraries.]
Installation steps
[Action needed: Provide clear instructions on how to locate the SDK, including installation steps for the driver and runtime.]
Verify installation
[Action needed: Include a simple method (e.g., running a model on a test image/input) to confirm the MX3 is correctly installed and ready for use.]
Getting Started
This guide walks you through deploying the Yolov8n model on an MemryX MX3 M.2 module using the Nx AI Manager.
Before you begin, ensure you are familiar with the Nx AI Manager and the process of running models on a CPU. If you need a refresher, please see the configuration section.
For this guide, we will use the following resources:
ONNX Model: The model is available at model.zip
Post-processor: The post-processor source is available at post_processor.zip
Upload model to Nx AI Cloud
Upload your ONNX model to the Nx AI Cloud following the steps outlined in the upload section.
Crucially, enable the MemryX conversion option within the model upload form and set the correct normalization values (as demonstrated in the image below).
The cloud platform will automatically convert the standard ONNX model into an MX3-compatible binary in the background. The model is ready for deployment once its status updates to ok, as shown below:
Select the MemryX runtime
The Nx AI Manager provides native support for multiple AI accelerators. To route inference to the chip, select the MemryX runtime in the Nx AI Manager settings page:
Install post-processor
Models running on the MX3 require an external post-processor to translate raw network output tensors (e.g., probability matrices) back into meaningful Nx metadata, such as object bounding boxes and classifications.
To deploy the provided YOLOv8m example, complete the following steps to compile and install the external C/C++ post-processor directly on the Nx Meta Server machine:
Extract the post-processor source code archive.
Execute the provided compilation script compile_install.sh. This builds the post-processor and automatically copies its config.json definition into the Nx AI Manager's post-processors directory:
bash ./compile_install.sh
To apply the change, restart the Nx VMS server. Instructions are available in this section.
Configure pipeline
Within the Nx Meta Client, configure the pipeline as shown in the image below:
Enable inference on camera
The final step is to activate inference for your camera by toggling the Device Active switch on the Nx AI Manager settings page (as shown below).
You should immediately begin seeing inference results in the Nx Meta Client after enabling the "Objects" tab.
To enable model inference on multiple cameras, simply repeat steps 4 and 5 for each camera.
Supported Models
The full list of supported models can be found in MemryX model zoo.
[Action needed: Please provide the link to your model zoo, or list the model architecture that are supported by MX3.]
Monitoring
Monitor pipeline throughput
To see the number of frames the Nx AI Manager is processing for a camera, enable the model FPS event. You can find detailed instructions on this page: Show AI inference framerate on video.
Monitor chip metrics
[Action needed: Please provide the command or method for viewing the chip's usage percentage, energy consumption, or temperature.]
Troubleshooting
Missing bounding-boxes
If you are not seeing bounding boxes visualized in the Nx Client, check the following potential factors:
Model Accuracy: The model might not be accurate enough for the specific environment where it is deployed.
Corrupted Configuration: If the pipeline configuration has become corrupted, try re-installing the post-processor and reconfiguring the pipeline.
Throughput Saturation: If you assign the pipeline to too many high-framerate streams, the pipeline throughput may become significantly lower than the stream framerate, causing the Nx AI Manager to drop most frames. To troubleshoot this, disable the AI Manager integration on all but one camera and gradually increase the number of cameras until you identify the machine's saturation limit.
Pipeline Incompatibility: If a camera on the server is assigned a model that is not compatible with the MemryX chip, the AI Manager integration will stop working for all cameras across the server. Verify that all cameras have the correct pipeline assigned. If any camera has an incompatible model, disable the AI Manager integration for that camera.
Technical Support
For technical questions or assistance, please reach out to us through the following channels:
MemryX Developer Support: [Action needed: Enter the correct support email address and/or link to the developer support zone here.]
Nx Customer Support Portal: Network Optix