MemryX Support
Deploy models to the MemryX MX3 accelerator with the AI Manager.
Summary
MX3
Experimental
x86_64
Ubuntu 22.04, Ubuntu 24.04
2.2.0
Introduction: About MemryX
MemryX, Inc. builds Edge AI acceleration hardware designed for efficient, scalable deployment. MemryX’s AI accelerator chips, known as MXAs, offer server-like performance and accuracy at low power and low cost and are well-positioned for Edge AI applications, including autonomous driving, robotics, machine vision, and more.
The MemryX MX3 M.2 AI Accelerator module has an M-key 2280 form factor and includes 4 MXA chips. This production module is now supported by the AI Manager.
Workflow
The AI Manager simplifies the process of securely deploying your AI models on MemryX MX3 devices with the following steps:
Install all necessary software dependencies.
Install the Nx Meta Mediaserver on your machine and choose the MemryX runtime option.
Upload your model to the AI Manager Cloud for compilation.
Compile and install the required post-processor.
Set up your inference pipeline.
Hardware and Software Requirements
Hardware Requirements:
Find detailed hardware requirements on the MemryX Developer Hub: see resource for installation requirements and recommendations.
In general, any modern x86_64 system with AVX2 and a suitable M.2 slot should work well with the MX3.
The MemryX driver and runtime libraries, which support multiple CPU and distribution combinations, are provided via the apt package repository.
Software Requirements:
Ubuntu 22.04 LTS
Ubuntu 24.04 LTS
Installation Steps
Follow the runtime installation steps from the MemryX Developer Hub: Installation Instructions
The instructions guide you through installing the MemryX runtime and drivers, both required to use the device with the AI Manager.
After installing the drivers and runtime, a system reboot is required. After rebooting, return to Step 7 of the Developer Hub installation steps to install the MemryX utilities and plugins, which are required for use with the AI Manager.
Verify Installation
To verify your MemryX installation, power on your system and run the following command:
The command should return the following to indicate that a single MemryX module is properly installed and ready to use:
To verify the installation further, run a workload on the accelerator module using the Hello, MXA! tutorial on the MemryX Developer Hub.
Getting Started
This guide walks you through deploying the YOLOv8m model on a MemryX MX3 M.2 module using the AI Manager.
Before starting, become familiar with the AI Manager and the process of running models on a CPU. See the configuration section for a refresher.
This guide uses the following resources:
ONNX Model: The model is available at model.zip
Post-processor: The post-processor source is available at post_processor.zip
Upload Model to AI Manager Cloud
Upload your ONNX model to the AI Manager Cloud following the steps outlined in the upload section.
Enable the MemryX conversion option in the model upload form and set the correct normalization values, as shown below.
The cloud platform will automatically convert the standard ONNX model into an MX3-compatible binary in the background. The model is ready for deployment once its status updates to ok, as shown below:
Select the MemryX Runtime
The AI Manager provides native support for multiple AI accelerators. To route inference to the chip, select the MemryX runtime in the AI Manager settings page:
Install Post-Processor
Models running on the MX3 require an external post-processor to translate raw network output tensors (e.g., probability matrices) back into meaningful Nx metadata, such as object bounding boxes and classifications.
To deploy the provided YOLOv8m example, complete the following steps to compile and install the external C/C++ post-processor directly on the Nx Meta Mediaserver machine:
Extract the post-processor source code archive.
Execute the provided compilation script compile_install.sh. This builds the post-processor and automatically copies its config.json definition into the AI Manager's post-processors directory:
To apply the change, restart the Nx VMS server. Instructions are available in this section.
Configure Pipeline
Within the Nx Meta Client, configure the pipeline as shown in the image below:
Enable Inference on Camera
The final step is to activate inference for your camera by toggling the Device Active switch on the AI Manager settings page (as shown below).
Inference results appear in the desktop client once the "Objects" tab is enabled.
To enable model inference on multiple cameras, repeat steps 4 and 5 for each camera.
Supported Models
An extensive list of supported models can be found in the MemryX Model Explorer.
MemryX does not provide a curated model zoo. Instead, BF16 activations let a wide variety of models run out-of-the-box without manual quantization or modification.
The MemryX Model Explorer is a great resource to characterize the types of supported model architectures, but users are certainly not limited to this selection of models.
Monitoring: Monitor Pipeline Throughput
To see the number of frames the AI Manager is processing for a camera, enable the model FPS event. You can find detailed instructions on this page: Show AI inference framerate on video.
Monitoring: Monitor Chip Metrics
Chip Temperatures
Two methods are available for viewing chip temperatures. The C++ MxAccl API provides two functions for temperature monitoring.
Use the following function to monitor the temperature of each chip on the specified device (docs):
To see the maximum chip temperature across all chips on the device, use (docs):
To monitor temperatures from the terminal, run the following command to check all chips on the device:
For continuous monitoring from the terminal, try the following command to view updated temperatures in 1-second intervals:
Temperature measurements are in degrees Celsius.
Chip Utilization
The C++ MxAccl API uses a metric referred to as ‘pressure’, which serves to approximate the throughput utilization of a given model on the device. Read more about the pressure metric in the pressure metric documentation.
Use the following C++ function to retrieve the MXA’s current pressure status (docs):
To monitor utilization continuously from the terminal, use the following command with 1-second intervals:
The percentage in this file is not measured the same way as the pressure metric described above; it approximates the current core utilization.
Troubleshooting: Missing Bounding-Boxes
If you are not seeing bounding boxes visualized in the desktop client, check the following potential factors:
Model Accuracy: The model might not be accurate enough for the specific environment where it is deployed.
Corrupted Configuration: If the pipeline configuration has become corrupted, try re-installing the post-processor and reconfiguring the pipeline.
Throughput Saturation: If you assign the pipeline to too many high-framerate streams, the pipeline throughput may become significantly lower than the stream framerate. This causes the AI Manager to drop most frames. To troubleshoot this, disable the AI Manager integration on all but one camera and gradually increase the number of cameras until you identify the machine's saturation limit.
Pipeline Incompatibility: If a camera on the server is assigned a model that is not compatible with the MemryX chip, the AI Manager integration will stop working for all cameras across the server. Verify that all cameras have the correct pipeline assigned. If any camera has an incompatible model, disable the AI Manager integration for that camera.
Technical Support
For technical questions, use the following support channels:
MemryX Developer Support: MemryX Developer Community Forum
Nx Customer Support Portal: https://support.networkoptix.com/
Nx Customer Support Portal: https://support.networkoptix.com/
Contact your reseller for platform support.
Last updated