``` sudo apt update sudo apt upgrade -y sudo apt-get install -y libgomp1 gdebi wget ``` * Download the [x64 Ubuntu Nx Meta server](https://meta.nxvms.com/download/linux) installer (metavms-server-6.x.x.x-linux\_x64.deb). \ OR \ Download the [ARM64 Ubuntu Nx Meta](https://meta.nxvms.com/download/releases/arm) server installer (metavms-server-x.x.x.x-linux\_arm64.deb). * You can use wget or curl in an SSH session to download the installer files directly to the server:
```sh # For example the the 6.0.5.41290 version for x64 architectures wget https://updates.networkoptix.com/metavms/41290/linux/metavms-server-6.0.5.41290-linux_x64.deb # Or the 6.0.5.41290 version for ARM64 architectures wget https://updates.networkoptix.com/metavms/41290/arm/metavms-server-6.0.5.41290-linux_arm64.deb ``` * Go to the directory where you downloaded the Nx Meta server package and install it: ``` sudo apt install -y ./metavms-server*.deb ``` * Next, set up the server by accessing the NX Meta Server web interface at port `:7001` for instance, at `https://localhost:7001/` or use the IP address for localhost `https://127.0.0.1:7001/` or use your external IP address with the same port. \ \ The web interface will have two buttons. Click the button "Setup New System", enter a System Name, click next. Choose a password. Click next again.\ \  * You can now connect your system to the Nx Cloud - or choose to do so later when you have added your system to the Nx Meta Client. If you do it now, just click "connect to cloud" in the Web Admin interface, and follow the prompts:
System Administration Cloud Tab
Camera Settings Navigation
Nx AI Manager Plugin Tab
The Nx Plugin Settings Tab with the Manage Device button
The Nx Client Popup where models can be managed for a device.
Visible Bounding Boxes in Nx Meta Client
Example Settings for Objects Counted Rule
Example of Objects Counted text overlay
A device is one model pipeline with a single model
A model pipeline form with changed settings that are not saved yet
A device detail page without any pipelines
The model catalogue with the assignment message and the option to select a model for the current pipeline
Message that the model will be added to the server
The device detail page with a single pipeline containing one model
Model pipeline with settings and buttons to manage models in the pipeline
A model pipeline with chaining options
A model pipeline with changes that need to be saved
The site details page with the list of available devices
The site details page with checkboxes for the available devices
The site details page with selected devices
The model catalogue with the assignment message and the option to select a model for the current pipeline
The model catalogue with a highlighted button to select a model for the current pipeline
The model catalogue with a dialog explaing the replacement
Message about the deployment to multiple devices
The device detail page with a single pipeline containing one model
Loitering Detection Settings
Left Behind Object Detection settings
Line Crossing Detection settings
Line Crossing Detection event configuration
Objects Counted event configuration
Input Masks setting
Class Visualisation setting
A high-level overview of the inference pipeline
External Postprocessor data flow
Tensor Postprocessor data flow
External Preprocessor data flow
## Steps We will describe the loop in 6 steps. 1. Add model to the Nx AI Cloud 2. Update postprocessor 3. Configure model and postprocessor 4. Run the model 5. Check the data 6. Update the model in the cloud 7. Update model in the system ### Step 1: Add model to Nx AI Cloud We will add the model of the training platform to the Nx AI Cloud. #### For Edge Impulse 1. Log in the Nx AI Cloud and click ‘Add a model’. 2. Select ‘Edge Impulse’, enter the API key and project ID. 3. Click ‘Link model’. Now the model will be downloaded from Edge Impulse and will be prepared for use.
### Step 2: Update postprocessor We will take an example postprocessor from the sclbl-integration-sdk and modify it with a specific condition to filter only specific images to send. #### For Edge Impulse 1. Open the sclbl-integration-sdk into your favourite editor. 2. Open the following Python script: postprocessor-python-edgeimpulse-example/postprocessor-python-edgeimpulse-example.py 3. Find the location where the API is set and replace it with the API-key from your project. Look for the line: `edgeimpulse.API_KEY` 4. Find the line `upload_sample = False` and modify the code below that to suit your condition. 5. Compile and install the postprocessor using the instructions in the README.md of the project. ### Step 3: Configure model and postprocessor We will select the model and postprocessor. 1. In the Nx client go to the device, right click and select ‘Camera settings’. 2. Go to the tab ‘Plugins’ and select the ‘Nx AI Manager’. 3. Click ‘Manage server’ and click the icon with left-right arrows called ‘Replace model’ or ‘Assign a new model’ if a model was not already assigned. 4. Choose the model from step 1 and click ‘Ok’. 5. Under the heading ‘Assigned Models’, select ‘Post Processor’ and select your post processor from step 2. If you didn’t rename it from the README it is called ‘Example-Postprocessor’. 6. Under the heading ‘Runtime Status’, click ‘Start’ and wait for the status to be ‘Runtime is running’. 7. Click ‘Ok’ at the bottom.
### Step 4: Run the model To run the model the device must be visible in the layout. 1. Double click your device to make it visible. 2. On the top right click the left icon ‘Object search’ if the model contains bounding boxes to see them.
### Step 5: Check the data Now we are going to check the incoming data, label it and then retrain. #### Edge Impulse See also the section on [importing models from Edge Impulse](/nx-ai-manager-v4.x/for-data-scientists/importing-models/from-edge-impulse). 1. Go to the project page. 2. Select ‘Data acquisition’. 3. Training and Test should contain data coming from the server. 4. Label the data. 5. Select ‘Retrain model’ and then ‘Train model’. ### Step 6: Update the model in the cloud We are going to update the model in the Nx AI Cloud. 1. Two ways to update the model 1. Go back to the ‘Manage server’ in the Nx Client / AI plugin and click the name of the model. 2. Log into the AI Cloud and go to the model. 2. Click the button ‘Update from \
### Step 7: Update model in the system (WorkInProgress) We are going to update the models in the system. 1. (option not available yet) In the Nx Client / AI plugin click the button ‘Update models’. 2. Manually delete the model cache. Follow these instructions: 1. On the cmdline: 1. `cd /opt/networkoptix-metavms/mediaserver/bin/plugins/nxai_plugin/nxai_manager/cache` 2. `rm *` 2. In the Nx Client / AI plugin stop and start the plugin # 7.5 Enable ini settings Advanced controls can be enabled through an ini file. By default the ini file does not exist. If you create this file and it is empty, the Network Optix Mediaserver will populate it with all available settings, their defaults, and some documentation. Create an empty file by running: ```bash sudo mkdir -p /home/networkoptix-metavms/.config/nx_ini sudo touch /home/networkoptix-metavms/.config/nx_ini/nxai_plugin.ini sudo chmod 666 /home/networkoptix-metavms/.config/nx_ini/nxai_plugin.ini ``` Then restart the mediaserver. Once the mediaserver is restarted the ini file should be filled with defaults. # Introduction In this section of our documentation, we describe the Nx AI Cloud platform. The Nx AI Cloud platform effectively allows for the following: * The Nx AI Cloud platform allows you to manage all the AI models that are available to you. Thus, you can see which AI models you can access, for which devices, and you can manage the version(s) of your user-generated models. * For [data scientists](/nx-ai-manager-v4.x/for-data-scientists/introduction), the Nx AI Cloud platform allows for the conversion of your own models. * The Nx AI Cloud platform allows you to (mass) deploy models to target edge devices. Effectively, you can "swap" the models that run on an edge device (configured using the [NX AI manager](/nx-ai-manager-v4.x/nx-ai-manager/get-started-with-the-nx-ai-manager-plugin)) remotely and change the device configuration. * The latter can be done at a large scale: you can flexibly group devices and deploy models to groups of devices. The following diagram shows some of the functions of the Nx AI Cloud platform and the relationships between the edge device(s) and model training platforms. {% @mermaid/diagram content="flowchart LR I1\[TFlite]-->A I2\[PyTorch]-->A I4\[...]-->A I3\[EdgeImpulse]-->A subgraph Platform A\[Model Conversion]-->B\[Model Library]-->C\[Model management] end C<-->E subgraph Edge D\[Pre-processing]-->E\[Model]-->F\[Post-processing] end H\[Input]-->D F-->G\[Output]" %} In the next sections, we detail each functionality. However, first, we describe how you can register for the platform. # Registration and log-in First, make sure you have created a [Network Optix account](https://nx.docs.scailable.net/nx-ai-manager/get-started-with-the-nx-ai-manager-plugin/1.-install-network-optix). Next, go to
Login form
overview of sites with one on-line system
listing of one site that is off-line
overview of a single on-line system
overview of a single off-line system
Overview of a single device in the system
models listing
multiple server selection activated
Shell script to gather logs and system information
```bash #!/bin/bash # This script is used to gather information about the HW and SW of the system, in addition to information about the Nx AI Manager. # The information is stored in a directory located in ~/nxai_troubleshooting and then compressed into a file named ~/nxai_troubleshooting.tgz. # To get support, please attach the compressed file to your support request. # NOTE: No sensitive information is collected, only the basic system information and the Nx AI Manager information. # Enable debug mode # set -x # Create directory where the information will be stored current_dir=$(pwd) info_dir=~/nxai_troubleshooting rm -rf $info_dir >/dev/null 2>&1 rm -rf $info_dir.tgz >/dev/null 2>&1 mkdir -p $info_dir # Redirect all output and error streams to a log file log_file="$info_dir/nxai_troubleshooting.log" exec > >(tee -a "$log_file") 2>&1 ############################### Basic System Information lsb_release -a >$info_dir/lsb_release.txt uname -a >$info_dir/uname.txt lscpu >$info_dir/lscpu.txt lspci >$info_dir/lspci.txt df -h >$info_dir/df.txt ldd --version >$info_dir/ldd_version.txt timedatectl >$info_dir/timedatectl.tx ############################### Check if the mediaserver is installed plugins_dir="" if [ -d /opt/networkoptix-metavms/mediaserver/bin/plugins/ ]; then plugins_dir="/opt/networkoptix-metavms/mediaserver/bin/plugins/" elif [ -d /opt/networkoptix/mediaserver/bin/plugins/ ]; then plugins_dir="/opt/networkoptix/mediaserver/bin/plugins/" echo "This is not a Meta installation" else echo "Mediaserver is not installed." fi # get Mediaserver installed version cat $plugins_dir/../../build_info.txt >$info_dir/mediaserver_info.txt ############################### Check if AI Plugin is installed if [ -f $plugins_dir/nxai_plugin/libnxai_plugin.so ]; then echo "AI Plugin is installed." else echo "AI Plugin is not installed." fi libnxai_plugin_dir=$plugins_dir/nxai_plugin/ # Check file permissions ls -lR $libnxai_plugin_dir/ >"$info_dir/nxai_plugin_permissions.txt" # Check if tree is installed if command -v tree >/dev/null 2>&1; then echo "Using tree to list files in the AI Plugin directory" tree -h --du "$libnxai_plugin_dir/" >"$info_dir/nxai_plugin_tree.txt" else echo "Using du to list file sizes in the AI Plugin directory" du -ah "$libnxai_plugin_dir/" >"$info_dir/nxai_plugin_du.txt" fi # Gather all log files in the AI Plugin directory find $libnxai_plugin_dir -name "*.log" -exec cp {} $info_dir/ \; find $libnxai_plugin_dir -name "*.log.*" -exec cp {} $info_dir/ \; ############################### Check if AI Manager is installed if [ -d $libnxai_plugin_dir/nxai_manager/bin ]; then echo "AI Manager is installed." else echo "AI Manager is not installed." fi nxai_manager_dir=$libnxai_plugin_dir/nxai_manager/ bin_dir=$nxai_manager_dir/bin/ ############################### Check AI Manager configuration if [ -f $bin_dir/installed_runtime.txt ]; then echo "Runtime might be installed." else echo "Runtime not installed." fi # Get the installed runtime information cp $bin_dir/installed_runtime.txt $info_dir/installed_runtime.txt # Get all settings files find $libnxai_plugin_dir -name "*.json" -exec cp {} $info_dir/ \; ############################### Check if AI Manager is running # get running processes ps aux | grep nxai >>$info_dir/nxai_manager_ps_aux.txt ############################### Check connectivity to the Nx AI Cloud # Check if curl or wget is available echo "Downloading test files from the Nx AI Cloud to measure download speed..." if command -v curl >/dev/null 2>&1; then echo "Using curl" # check if Nx AI Cloud is reachable curl -s https://api.sclbl.nxvms.com/dev/ >$info_dir/nxai_cloud_connectivity.txt # Download a file from the Nx AI Cloud to measure the download speed curl -s -m 20 "https://cdn.sclbl.nxvms.com/benchmark.bin?size=10" -o /dev/null -w "%{speed_download}" | awk '{print "Model download speed: " $1/1048576 " MB/sec"}' \ >$info_dir/nxai_cloud_download_speed_of_model.txt curl -s -m 20 "https://artifactory.nxvms.dev/artifactory/nxai_open/files/23MB.bin" -o /dev/null -w "%{speed_download}" | awk '{print "Runtime download speed: " $1/1048576 " MB/sec"}' \ >$info_dir/nxai_cloud_download_speed_of_runtime.txt elif command -v wget >/dev/null 2>&1; then echo "Using wget" wget -q -O "$info_dir/nxai_cloud_connectivity.txt" https://api.sclbl.nxvms.com/dev/ wget --timeout=20 "https://cdn.sclbl.nxvms.com/benchmark.bin?size=10" -O /dev/null >$info_dir/nxai_cloud_download_speed_of_model.txt 2>&1 wget --timeout=20 "https://artifactory.nxvms.dev/artifactory/nxai_open/files/23MB.bin" -O /dev/null >$info_dir/nxai_cloud_download_speed_of_runtime.txt 2>&1 else echo "ERROR: Neither curl nor wget is installed." fi # Get the latency to the Nx AI Cloud ping -c 10 api.sclbl.nxvms.com >$info_dir/nxai_cloud_ping.txt # Get information about DEEPX if available # Checking if dxrt-cli is installed if command -v dxrt-cli >/dev/null 2>&1; then echo "dxrt-cli is installed." dxrt-cli -s >$info_dir/dxrt_cli_version.txt 2>&1 else echo "dxrt-cli is not installed." fi # Get information about Nvidia if available # Checking if nvidia-smi is installed if command -v nvidia-smi >/dev/null 2>&1; then echo "nvidia-smi is installed." nvidia-smi >$info_dir/nvidia_smi.txt 2>&1 else echo "nvidia-smi is not installed." fi ############################### tar compress the information cd $info_dir/.. tar -cvf $info_dir.tgz "$(basename $info_dir)" >/dev/null || echo "ERROR: Failed to compress the information." rm -rf $info_dir >/dev/null 2>&1 echo "System information gathering complete." echo "The collected information is stored in $info_dir.tgz" echo "Please attach this archive to your support request." cd "$current_dir" ```
The popup window showing that the plugin is disabled for this device
The popup window showing that the plugin is enabled for this device, but the device is not active.
The popup window showing that the plugin is enabled and the device is active.
Add the following line to the `/etc/systemd/system/networkoptix-metavms-mediaserver.service` file to set the necessary environment variable:
```plaintext Environment="HAILO_MONITOR=1" ``` \ The updated configuration file should look like this:
```plaintext [Unit] Description=Network Optix Media Server After=network.target local-fs.target remote-fs.target Requires=networkoptix-metavms-root-tool.service [Service] PermissionsStartOnly=true ExecStartPre=/opt/networkoptix-metavms/mediaserver/lib/scripts/systemd_mediaserver_pre_start.sh ExecStart=/opt/networkoptix-metavms/mediaserver/lib/scripts/systemd_mediaserver_start.sh User=networkoptix-metavms Group=networkoptix-metavms Restart=always TimeoutStopSec=120 KillMode=process TasksMax=8191 LimitCORE=infinity Environment="HAILO_MONITOR=1" [Install] WantedBy=multi-user.target ``` 2. **Restart the NX Media Server:**
After updating the configuration file, restart the Network Optix Media Server for the changes to take effect. You can do this by running one of the following commands:
```bash sudo systemctl restart networkoptix-metavms-mediaserver.service ``` \ or
```bash sudo service networkoptix-metavms-mediaserver restart ``` 3. **Run hailortcli**\ \ `HAILO_MONITOR=1 hailortcli monitor`
Where to upload a NetsPresso model in the Nx AI Cloud.
Interface for editing the model normalization values.
# From Hugging Face ### About Hugging Face  > Hugging Face, Inc. is a French-American company headquartered in New York City, specializing in the development of computer tools for building applications using machine learning. Renowned for its transformers library tailored for natural language processing (NLP) applications, the company also offers a platform enabling users to share machine learning models and datasets while showcasing their work. Despite Hugging Face focusing mostly on NLP models based on the transformers architecture, they are also providing support for image classficatin models in their [timm](https://huggingface.co/docs/timm/en/index) library, and some object object detection models in the [transformers](https://huggingface.co/docs/transformers/en/index) library. ### Deploying Hugging Face models As stated previously, our interest is focused only on vision models not NLP models, and specifically image classification and object detection models. To deploy an image classification model, check out the [Common Models](/nx-ai-manager-v4.x/for-data-scientists/importing-models/common-models) guides. # From Ultralytics ### About Ultralytics  > Ultralytics is a software company specializing in computer vision and deep learning solutions. They are known for developing and maintaining popular open-source libraries such as YOLOV8, which is a state-of-the-art object detection model. YOLO (You Only Look Once) is a family of real-time object detection algorithms, and YOLOv8 is the latest iteration developed by Ultralytics. ### Deploying Ultralytics models Ultralytics provides two main models Yolov5 and Yolov8. You can find a guide that shows how to deploy these two categories of models [here](/nx-ai-manager-v4.x/for-data-scientists/importing-models/common-models). # From PyTorch ## About PyTorch  [PyTorch](https://pytorch.org) is a [machine learning](https://en.wikipedia.org/wiki/Machine_learning) [framework](https://en.wikipedia.org/wiki/Software_framework) based on the [Torch](https://en.wikipedia.org/wiki/Torch_\(machine_learning\)) library, used for applications such as [computer vision](https://en.wikipedia.org/wiki/Computer_vision) and [natural language processing](https://en.wikipedia.org/wiki/Natural_language_processing), originally developed by [Meta AI ](https://en.wikipedia.org/wiki/Meta_AI)and now part of the [Linux Foundation](https://en.wikipedia.org/wiki/Linux_Foundation) umbrella. It is [free and open-source software](https://en.wikipedia.org/wiki/Free_and_open-source_software) released under the [modified BSD license](https://en.wikipedia.org/wiki/Modified_BSD_license). Although the [Python](https://en.wikipedia.org/wiki/Python_\(programming_language\)) interface is more polished and the primary focus of development, PyTorch also has a [C++](https://en.wikipedia.org/wiki/C%2B%2B) interface. ## Model deployment from PyTorch Model deployment from PyTorch is simple to achieve by exporting your PyTorch model to ONNX and subsequently using (if neccesary) the `sclblonnx` package to [clean and check the resulting graph](/nx-ai-manager-v4.x/for-data-scientists/onnx-requirements#automatic-checking-using-the-sclblonnx-check-function) for an upload to the Nx AI cloud. * You can find details on PyTorch to ONNX exports [here](https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html). You can find an insightful tutorial [here](https://deci.ai/blog/how-to-convert-a-pytorch-model-to-onnx/). * You can find an example using PyTorch and sclblonnx [here](https://github.com/scailable/sclblonnx/blob/master/examples/example_03.py). After obtaining a [clean ONNX graph that adheres to our requirements](/nx-ai-manager-v4.x/for-data-scientists/onnx-requirements), you can [upload it to the Nx AI cloud](/nx-ai-manager-v4.x/nx-ai-cloud/upload-your-model) for deployment. # From TensorFlow / TFLite ## About TensorFlow  **TensorFlow** is a [free and open-source](https://en.wikipedia.org/wiki/Free_and_open-source_software) [software library](https://en.wikipedia.org/wiki/Library_\(computing\)) for [machine learning](https://en.wikipedia.org/wiki/Machine_learning) and [artificial intelligence](https://en.wikipedia.org/wiki/Artificial_intelligence). It can be used across a range of tasks but has a particular focus on [training](https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks#Training) and [inference](https://en.wikipedia.org/wiki/Statistical_inference) of [deep neural networks](https://en.wikipedia.org/wiki/Deep_neural_networks).[\[4\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-4)[\[5\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-YoutubeClip-5) TensorFlow was developed by the [Google Brain](https://en.wikipedia.org/wiki/Google_Brain) team for internal [Google](https://en.wikipedia.org/wiki/Google) use in research and production.[\[6\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-6)[\[7\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-7)[\[8\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-8) The initial version was released under the [Apache License 2.0](https://en.wikipedia.org/wiki/Apache_License_2.0) in 2015.[\[1\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-Credits-1)[\[9\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-Metz-Nov9-9)Google released the updated version of TensorFlow, named TensorFlow 2.0, in September 2019.[\[10\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-:12-10) TensorFlow can be used in a wide variety of programming languages, including [Python](https://en.wikipedia.org/wiki/Python_\(programming_language\)), [JavaScript](https://en.wikipedia.org/wiki/JavaScript), [C++](https://en.wikipedia.org/wiki/C%2B%2B), and [Java](https://en.wikipedia.org/wiki/Java_\(programming_language\)).[\[11\]](https://en.wikipedia.org/wiki/TensorFlow#cite_note-:13-11) This flexibility lends itself to a range of applications in many different sectors. ## Model deployment from TFLite You can upload a [TFLite](https://www.tensorflow.org/lite) model directly, and the cloud will take care of exporting the model to ONNX. ## Model deployment from TensorFlow Exporting from TensorFlow to ONNX is also possible using [the tf2onnx tools](https://github.com/onnx/tensorflow-onnx), as illustrated in these examples: [here](https://github.com/scailable/sclblonnx/blob/master/examples/example_04.py) and [here](https://onnxruntime.ai/docs/tutorials/tf-get-started.html#converting-a-model). Your TensorFlow model can be exported to ONNX and subsequently using cleaned and checked by the the `sclblonnx` package for an upload to the Nx AI cloud. After obtaining a [clean ONNX graph that adheres to our requirements](/nx-ai-manager-v4.x/for-data-scientists/onnx-requirements), you can upload it to the [Nx AI cloud for deployment](/nx-ai-manager-v4.x/nx-ai-cloud/upload-your-model). # From Scikit-learn ## About Scikit-learn  [**scikit-learn**](https://scikit-learn.org/stable/) is a Python module for machine learning built on top of SciPy and is distributed under the 3-Clause BSD license. The project was started in 2007 by David Cournapeau as a Google Summer of Code project, and since then many volunteers have contributed. Scikit-learn contains a collection of classic ML models that can be used to solve various practical tasks: classification, regression, data points clustering, etc. using for instance SVM, logistic regression, decision tree, isolation forest, etc. ## Model deployment from Scikit-learn Model deployment from scikit-learn is simple to achieve by exporting your sklearn model to ONNX using the [skl2onnx](https://onnx.ai/sklearn-onnx/) package. After obtaining a [clean ONNX graph that adheres to our requirements](/nx-ai-manager-v4.x/for-data-scientists/onnx-requirements), you can upload it to the [Nx AI cloud for deployment](/nx-ai-manager-v4.x/nx-ai-cloud/upload-your-model). # Common Models The Nx toolkit currently offers support for image classification and object detection models. To facilitate easy deployment, we provide several tutorials containing instructions on how to deploy common models such as MobileNet, ResNet, EfficientNet, ViT, LeViT for image classification, and YoloS, Yolov4, Yolov7, Yolov8 for object detection. Key steps involved in this process include: 1. Incorporating necessary post-processing steps such as masking and a configurable Non-Maximum Suppression (NMS) for object detection models and Softmax for image classification models directly into the ONNX graph. 2. Modifying the model's input and output shapes and names as per the requirements of the AI Manager. 3. Making sure the exported ONNX has an Operator Set Version less than 18. The primary objective of these tutorials is to equip AI developers with off-the-shelf scripts, enabling them to deploy these models within minutes. This is achieved by exporting the models to ONNX format, ensuring compatibility with the AI Manager as described [previously](/nx-ai-manager-v4.x/for-data-scientists/onnx-requirements). {% hint style="info" %} You can find the tutorials on this [Github repository](https://github.com/scailable/nxai-model-to-onnx). {% endhint %} # Nx AI Certification Test ## Certification Test ### Quickstart The Nx AI Certification test should be able to run on any Ubuntu installation which is also compatible with Network Optix Server. Ensure your device has at least a few gigabytes of free space, a working internet connection and a working Python 3 and Pip installation. {% code title="Python requirements" %} ```bash # on some systems you may need to install pip # for example with the following command on ubuntu sudo apt update sudo apt install python3-pip ``` {% endcode %} {% hint style="warning" %} If you are testing a GPU or NPU, also make sure that the required drivers are installed, for example the Jetpack drivers for NVIDIA accelerators. {% endhint %} Execute the following commands: {% code title="Download tests" %} ```bash ## Create and enter folder for test mkdir nxai_test cd nxai_test ## Download testing suite wget https://artifactory.nxvms.dev/artifactory/nxai_open/NXAITest/nxai_test.tgz ## Unpack testing suite tar -xvf nxai_test.tgz ``` {% endcode %} {% code title="Ubuntu quirk" %} ```bash # on newer ubuntu systems you need to activate a python venv sudo apt install python3-venv python3 -m venv ./ # create venv in current dir source ./bin/activate # activate python venv in current dir ``` {% endcode %} {% code title="Install acceleration library" %} ```bash ## Install all required Python packages pip3 install -r requirements.txt ## Install NX AI Manager ./Utilities/install_nxai_manager.sh ## Install acceleration runtime python3 Utilities/install_acceleration_library.py #python3 Utilities/install_acceleration_library.py "Nx CPU" ``` {% endcode %} {% hint style="info" %} Running the `python3 Utilities/install_acceleration_library.py` command will automatically detect the available hardware acceleration that is available on your device. If the script finds more than one option, it will pause execution and ask you to choose. If you want to execute these commands without pausing, add an the accelerator name to the command, such as the "Nx CPU" example. {% endhint %} {% code title="Download models" %} ```bash ## Download required models python3 Utilities/download_models.py ``` {% endcode %} {% hint style="warning" %} The `download_models.py` command will download approximately 3.5GB of model data, depending on your connection this may take a while. {% endhint %} {% code title="Run tests" %} ```bash ## Start test python3 all_suites.py ``` {% endcode %} {% code title="Gather info and print ID" %} ``` ## Gather info python3 Utilities/gather_hwinfo.py ``` {% endcode %} ``` ## Upload results to cloud python3 Utilities/upload_results.py ``` {% hint style="warning" %} The test will run for a couple of hours and will stress the device. Do not power off your device. {% endhint %} ### Introduction The Nx AI Certification Test is meant to test any device to ensure it is compatible and stable enough to run the Nx AI Manager for extended periods of time. The test will attempt to run multiple common model architectures, with different sizes, and even multiples of those models to test compatibility and what can be expected to run on the device. The test will also run long operation tests to check if there are any memory issues or degradation in performance due to overheating or other reasons. Finally, the test will gather all results in a folder which can be used to generate a report, either on the device or somewhere else. The report should contain enough information to determine if the device is indeed compatible. ### Installing Nx AI Manager A useful script is provided to install the Nx AI Manager runtime locally within the test environment. This will ensure that this testing does not interfere with any existing installations on the device. ```bash ./Utilities/install_nxai_manager.sh ``` ### Installing Nx Acceleration Library Acceleration libraries act as layers between the Nx AI Manager and your acceleration hardware. Any device should be able to run the tests on CPU, without acceleration hardware. However if your device has AI acceleration hardware, such as an Nvidia Cuda device, or a Hailo AI chip, these can be used to accelerate the AI pipeline. Running this script will automatically detect available hardware. If more than one is found, the script will present a list of options which you can choose from. ``` python3 Utilities/install_acceleration_library.py ``` ### Download Models The Certification Test will test a variety of models to see if they can run on your device and libraries. After installing the acceleration library the correct models for your device can be downloaded. {% hint style="warning" %} This step could use a lot of data as many large files need to be downloaded. This might also take a long time depending on network conditions. {% endhint %} ``` ## Download required models python3 Utilities/download_models.py ``` ### Running test The Nx AI Certification test includes a collection of tests to test different aspects of your device to ensure that the Nx AI Toolkit can run on your device. To run all of these tests in one large test, run: ```bash python3 all_suites.py ``` This could take many hours to complete. ### Gathering Hardware Information This command will detect some hardware information about your device and package it with the test results. This will gives context to the test results and allows for better analysis about the device's performance. ``` ## Gather info python3 Utilities/gather_hwinfo.py ``` This command will clearly print the ID of your device. This ID can then later be used to identify your device when the test results are uploaded. This command can be run again at any time if you need the ID again. ### Uploading Results An endpoint was created where you can upload and view the test results of your device. After the test has completed, run the following to automatically upload the script to the cloud: ```bash python3 Utilities/upload_results.py ``` ## Custom Model Benchmark {% hint style="info" %} The Custom Model Benchmark can only be used to benchmark models which have been uploaded to the Nx AI Cloud. {% endhint %} The test allows you to benchmark your own models on different devices. The Certification Test includes the functionality to add your own models to its benchmark test. If you already have that downloaded, you can skip downloading and extracting the benchmark test. If you are only interested in benchmarking a model, you can instead download the smaller benchmark package: {% code title="Download benchmark" %} ```sh ## Create and enter folder for test mkdir nxai_benchmark cd nxai_benchmark ## Download testing suite wget https://artifactory.nxvms.dev/artifactory/nxai_open/NXAITest/nxai_benchmark.tgz ## Unpack testing suite tar -xvf nxai_benchmark.tgz ``` {% endcode %} Install an Nx AI Manager with runtime by following the steps at [#installing-nx-ai-manager](#installing-nx-ai-manager "mention") Next, add as many models as you want to benchmark by running: ```bash python3 Utilities/add_benchmark_model.py
My Nx User
my.networkoptix.com
- Contact
- Company
- Account
- Other
- BUSINESS
- ACCOUNT
- MARKETING
- PURPOSE
- ADMINISTRATIVE
Nx Cloud User
nxvms.com
- Contact
- Company
- Account
- System
- Other
- ACCOUNT
- PURPOSE
- ADMINISTRATIVE
Nx customer, partner, or reseller
e.g. daily operations
- Contact
- Company
- Billing
- Other
- BUSINESS
- PURPOSE
- MARKETING
Interested party
e.g. Events, exhibitions, and offline
- Contact
- Company
- Other
- BUSINESS
- PURPOSE
- MARKETING
Support
e.g. support portal, supported devices list, developer feedback forms…
- Contact
- Company
- Other
- PURPOSE
| “Software” or “Content” | means any data, software, On-device Software consisting of,amongst others but not limited to the AI Manager and the runtime software, system or computer-implemented method and machine learned models, object codes, algorithms, images, (set of arranged) instructions, or innovations, or any written summaries, photographs, or drawings describing any data, data, software, algorithms, method, models, artificial intelligence (“AI”) or innovations made available by Scailable for the purpose of using the Platform, including all patches, revisions, updates, upgrades or replacements of such software, and, unless subject to a separate license agreement, all separate or additional scripts, modules or software to be used with or in conjunction with such software, all related Documentation, sample code and model libraries (in any form) and all copies thereof |
| “AI Manager” | means the portion of the Software specified in the Documentation of the applicable License Agreement. |
| “On-device Software” | means the AI Manager, run-time software and all other Software except the Software available on the Platform as may be specified in a License Agreement, in other document made part of the Agreement or on our website. |
| “Platform” | means the software platform, its content and related services hosted by Scailable and made available to Licensee under this Agreement. |
| “Documentation” | means any and all materials and documentation related to the Platform or Software or any other Services, including, without limitation, manuals, guides, training materials, policies, help files, programmer's notes and remarks, and any other written, graphic or printed materials, whether in hard copy or digital format, provided or made available by us, whether through posting on our Platform, a web site, or otherwise. |
Camera Settings Navigation
The popup window showing that the plugin is disabled for this device
The popup window showing that the plugin is enabled for this device, but the device is not active.
The popup window showing that the plugin is enabled and the device is active.
Loitering Detection Settings
Left Behind Object Detection settings
Line Crossing Detection settings
Line Crossing Detection event configuration
Objects Counted event configuration
Input Masks setting
Class Visualization setting
Camera Settings Navigation
Nx AI Manager Integration Tab
The Plugin Settings Tab with the Manage Device button
The Nx Client Popup where models can be managed for a device.
Visible Bounding Boxes in Nx Meta Client
Example Settings for Objects Counted Rule
Example of Objects Counted text overlay
A device detail page without any pipelines
The model catalogue with the assignment message
and the option to select a model for the current pipeline
Message that the model will be added to the server
The device detail page with a single pipeline containing one model
Model pipeline with settings and buttons to manage models in the pipeline
A model pipeline with chaining options
A model pipeline with changes that need to be saved
A device is one model pipeline with a single model
A model pipeline form with changed settings that are not saved yet
{% endcolumn %}
{% endcolumns %}
This also means that the image can not be another shape than a rectangle, and you may need to change the model resizing method to letterbox for the models to work correctly.
{% hint style="warning" %}
Note that using masks and region of interest at the same time is not supported.
{% endhint %}
### Custom external processing
If this behavior is not the desired behavior, is possible to create your own preprocessor to create a mask or postprocessor to exclude results from an area that you designate.
For more information on custom preprocessors and postprocessors you can checkout the examples in our GitHub repository
{% embed url="The site details page with the list of available devices
The site details page with checkboxes for the available devices
The site details page with selected devices
External Preprocessor data flow
A high-level overview of the inference pipeline
External Postprocessor data flow
Tensor Postprocessor data flow
sudo chmod -R 777 /opt/networkoptix/mediaserver/var/nx_ai_manager
sudo chmod -R 777 /opt/networkoptix/mediaserver/var/nx_ai_manager/
sudo chmod -R 777 /opt/networkoptix-metavms/mediaserver/var/nx_ai_manager
## Steps We will describe the loop in 6 steps. 1. Add model to the Nx AI Cloud 2. Update postprocessor 3. Configure model and postprocessor 4. Run the model 5. Check the data 6. Update the model in the cloud 7. Update model in the system ### Step 1: Add model to Nx AI Cloud We will add the model of the training platform to the Nx AI Cloud. #### For Edge Impulse 1. Log in the Nx AI Cloud and click ‘Add a model’. 2. Select ‘Edge Impulse’, enter the API key and project ID. 3. Click ‘Link model’. Now the model will be downloaded from Edge Impulse and will be prepared for use.
### Step 2: Update postprocessor We will take an example postprocessor from the sclbl-integration-sdk and modify it with a specific condition to filter only specific images to send. #### For Edge Impulse 1. Open the sclbl-integration-sdk into your favourite editor. 2. Open the following Python script: postprocessor-python-edgeimpulse-example/postprocessor-python-edgeimpulse-example.py 3. Find the location where the API is set and replace it with the API-key from your project. Look for the line: `edgeimpulse.API_KEY` 4. Find the line `upload_sample = False` and modify the code below that to suit your condition. 5. Compile and install the postprocessor using the instructions in the README.md of the project. ### Step 3: Configure model and postprocessor We will select the model and postprocessor. 1. In the Nx client go to the device, right click and select ‘Camera settings’. 2. Go to the tab ‘Plugins’ and select the ‘Nx AI Manager’. 3. Click ‘Manage server’ and click the icon with left-right arrows called ‘Replace model’ or ‘Assign a new model’ if a model was not already assigned. 4. Choose the model from step 1 and click ‘Ok’. 5. Under the heading ‘Assigned Models’, select ‘Post Processor’ and select your post processor from step 2. If you didn’t rename it from the README it is called ‘Example-Postprocessor’. 6. Under the heading ‘Runtime Status’, click ‘Start’ and wait for the status to be ‘Runtime is running’. 7. Click ‘Ok’ at the bottom.
### Step 4: Run the model To run the model the device must be visible in the layout. 1. Double click your device to make it visible. 2. On the top right click the left icon ‘Object search’ if the model contains bounding boxes to see them.
### Step 5: Check the data Now we are going to check the incoming data, label it and then retrain. #### Edge Impulse See also the section on [importing models from Edge Impulse](/nx-ai-manager-v6.1.1/ai-models-support/importing-models/from-edge-impulse). 1. Go to the project page. 2. Select ‘Data acquisition’. 3. Training and Test should contain data coming from the server. 4. Label the data. 5. Select ‘Retrain model’ and then ‘Train model’. ### Step 6: Update the model in the cloud We are going to update the model in the Nx AI Cloud. 1. Two ways to update the model 1. Go back to the ‘Manage server’ in the Nx Client / AI plugin and click the name of the model. 2. Log into the AI Cloud and go to the model. 2. Click the button ‘Update from \
### Step 7: Update model in the system (WorkInProgress) We are going to update the models in the system. 1. (option not available yet) In the Nx Client / AI plugin click the button ‘Update models’. 2. Manually delete the model cache. Follow these instructions: On the commandline: ``` cd /opt/networkoptix-metavms/mediaserver/var/nx_ai_manager/cache rm * ``` In the Nx Client / AI plugin stop and start the plugin # Introduction In this section of our documentation, we describe the Nx AI Cloud platform. The Nx AI Cloud platform effectively allows for the following: * The Nx AI Cloud platform allows you to manage all the AI models that are available to you. Thus, you can see which AI models you can access, for which devices, and you can manage the version(s) of your user-generated models. * For [data scientists](/nx-ai-manager-v6.1.1/ai-models-support/introduction), the Nx AI Cloud platform allows for the conversion of your own models. * The Nx AI Cloud platform allows you to (mass) deploy models to target edge devices. Effectively, you can "swap" the model pipelines that run on an edge device (configured using the [NX AI manager](/nx-ai-manager-v6.1.1/nx-ai-manager/getting-started)) remotely and change the device configuration. * The latter can be done at a large scale: you can flexibly group devices and deploy model pipelines to groups of devices. The following diagram shows some of the functions of the Nx AI Cloud platform and the relationships between the edge device(s) and model training platforms. {% @mermaid/diagram content="flowchart LR I1\[TFlite]-->A I2\[PyTorch]-->A I4\[...]-->A I3\[EdgeImpulse]-->A subgraph Platform A\[Model Conversion]-->B\[Model Library]-->C\[Model management] end C<-->E subgraph Edge D\[Pre-processing]-->E\[Model]-->F\[Post-processing] end H\[Input]-->D F-->G\[Output]" %} In the next sections, we detail each functionality. ### Advanced Only for advanced users: direct links to the Nx AI Cloud interface. In 6.1.1, the Nx Cloud UI is officially supported only on Nx Meta.\ \ Nx Witness:\
overview of sites with one on-line system
listing of one site that is off-line
overview of a single on-line system
overview of a single off-line system
Overview of a single device in the system
models listing
multiple server selection activated
The popup window showing that the plugin is disabled for this device
The popup window showing that the plugin is enabled for this device, but the device is not active.
The popup window showing that the plugin is enabled and the device is active.
Where to upload a NetsPresso model in the Nx AI Cloud.
Interface for editing the model normalization values.
--- [Next Page](/llms-full.txt/1)