Intel® Edge Software Device Qualification (Intel® ESDQ) for Robotics SDK

Overview

Intel® Edge Software Device Qualification (Intel® ESDQ) for Robotics SDK provides customers with the capability to run an Intel® provided test suite at the target system, with the goal of enabling partners to determine their platform’s compatibility with the Robotics SDK.

The target of this self certification suite is the Robotics SDK compute systems. These platforms are the brain of the Robot Kit. They are responsible to get input from sensors, analyze them, and give instructions to the motors and wheels to move the Robotics SDK.

How It Works

The Robotics SDK Test Modules interact with the Intel® ESDQ Command Line Interface (CLI) through a common test module interface (TMI) layer which is part of the Intel® ESDQ binary. Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which you can manually choose to email to: edge.software.device.qualification@intel.com. More detailed information is available at Intel® Edge Software Device Qualification (Intel® ESDQ) Overview.

Note

Each test and its pass/fail criteria is described below. To jump to the installation process, go to Download and Install Intel® ESDQ for Robotics SDK.

Intel® ESDQ for Robotics SDK contains the following test modules.

Intel® RealSense™ Camera

This module verifies the capabilities of the Intel® RealSense™ technology on the target platform. For more information, go to the Intel® RealSense™ website.

The tests within this module verify that the following features are installed properly on the target platform and that Robotics SDK and Intel® RealSense™ camera are functioning properly.

The tests are considered PASS if:

  • The Intel® RealSense™ SDK 2.0 libraries are installed on the target system.

  • A simple C++ file can be compiled using the g++ compiler and the -lrealsense2 compilation flag.

  • Intel® RealSense™ Topics are listed and published.

  • The number of FPS (Frames Per Second) are as expected.

Intel® Intel® VTune™ Profiler Profiler

This module runs the Intel® Intel® VTune™ Profiler Profiler on the target system. For more information, go to the Intel® VTune™ Profiler website.

The test is considered PASS if:

  • Intel® VTune™ Profiler Profiler runs without errors.

  • Intel® VTune™ Profiler Profiler collects Platform information.

rviz2 and FastMapping

This module runs the FastMapping application (the version of octomap optimized for Intel®) on the target system and uses rviz2 to verify that it works as expected. For more information, go to the rviz wiki.

The test is considered PASS if:

  • FastMapping is able to create a map out of a pre-recorded ROS 2 bag.

Intel® oneAPI Base Toolkit

This module verifies some basic capabilities of Intel® oneAPI Base Toolkit on the target platform. For more information, go to the Intel® oneAPI Base Toolkit website.

The tests within this module verify that the DPC++ compiler features are functioning properly on the target platform.

This test is considered PASS if:

  • A simple C++ file can be compiled using the DPC++ compiled and it runs as expected.

OpenVINO™ Toolkit

This module verifies two core features of the OpenVINO™ Toolkit:

  • OpenVINO™ model optimizer

  • Object detection using TensorFlow*

The test is considered PASS if:

  • The OpenVINO™ model optimizer is capable to transform a TensorFlow model to an Intermediate Representation (IR) of the network, which can be inferred with the Inference Engine.

OpenVINO™ Query for inferencing devices

This module executes the Hello Query Device C++ sample application of the OpenVINO™ toolkit. This application identifies all available devices that can be used for inferencing.

The test is considered PASS if:

  • The OpenVINO™ Hello Query Device sample application can identify the inferencing devices CPU and GPU.

  • On Intel® Core™ Ultra Processors, in addition the NPU must be be identified as an inferencing device.

GStreamer* Video

This module verifies if a GStreamer* Video Pipeline using GStreamer* Plugins runs on the target system.

The test is considered PASS if:

  • The Video Pipeline was opened on the host without errors.

GStreamer* Audio

This module verifies if a GStreamer* Audio Pipeline using GStreamer* Plugins runs on the target system.

The test is considered PASS if:

  • The Audio Pipeline was opened on the host without errors.

GStreamer* Autovideosink Plugin - Display

This module verifies if a stream from a camera compatible with libv4l2 can be opened and displayed using GStreamer*.

The test is considered PASS if:

  • No Error messages are displayed while running the gst-launch command.

This test may Fail, or it may be skipped if the target system does not have a Web Camera connected.

ADBSCAN

This module verifies if the ADBSCAN algorithm works on the target system.

The test is considered PASS if:

  • The ADBSCAN algorithm works on the target system.

Collaborative Visual SLAM

This module verifies if the collaborative visual SLAM algorithm works on the target system.

The test is considered PASS if:

  • The collaborative visual SLAM algorithm works on the target system.

Kudan Visual SLAM

This module verifies if the Kudan Visual SLAM algorithm works on the target system.

The test is considered PASS if:

  • The Kudan Visual SLAM algorithm works as expected when the ORB feature extraction is performed by the CPU.

  • The Kudan Visual SLAM algorithm works as expected when the GPU ORB Extractor is used.

Get Started

This tutorial takes you through the installation and execution of the Intel® ESDQ CLI tool. Configure your target system to satisfy the necessary Prerequisites before you proceed with the Download and Install Intel® ESDQ for Robotics SDK. Execute your self-certification process by selecting from the three available certification types:

Refer to How It Works for more detailed information about the test modules.

Prerequisites

Satisfy the Intel® ESDQ prerequisites by:

  • Installing OpenVINO™ Development Tools and specifying tensorflow as the extras parameter of the described “Step 4. Install the Package” instructions:

    pip install openvino-dev[tensorflow]
    
    Copy to clipboard
  • Installing the intel-basekit Deb package by following the Intel® oneAPI Base Toolkit Installation Guide for Linux* OS instructions.

  • Installing GStreamer* by following the “Install GStreamer on Ubuntu or Debian” instructions.

  • Installing the pre-built Intel® RealSense™ SDK 2.0 packages librealsense2-utils, librealsense2-dev and librealsense2-dbg by following the “Installing the packages” instructions.

  • Configuring your Intel® VTune™ Profiler installation as described in the “Additional System setup for CPU and GPU profiling” section of Intel® VTune™ Profiler for CPU and GPU profiling.

  • Configuring your target device as described in Kudan Visual SLAM.

  • Installing the OpenVINO™ Runtime by executing these steps:

    1. Add the OpenVINO™ APT package sources as described in section “OpenVINO™ Installation Steps” on page Install OpenVINO™ Packages.

    2. Make sure that your file /etc/apt/preferences.d/intel-openvino pins the OpenVINO™ version of all components to 2024.2.0* or above. Consider that earlier OpenVINO™ versions do not support the NPU of Intel® Core™ Ultra Processors.

    3. Install the OpenVINO™ Runtime by using:

      sudo apt-get install openvino
      
      Copy to clipboard

    Additional information can be found in the OpenVINO™ documentation.

  • Installing the Intel® NPU Driver as described on page Install the Intel® NPU Driver on Intel® Core™ Ultra Processors. Don’t execute this step if your system does not have an Intel® Core™ Ultra Processor.

Note

Make sure that Git is installed on your target system.

Download and Install Intel® ESDQ for Robotics SDK

Complete the following two installation steps in order to properly configure your test setup:

  1. Download and Install Intel® ESDQ CLI

  2. Download and Install the Test Modules

Download and Install Intel® ESDQ CLI

Download the Intel® ESDQ CLI to your device from here: edge-software-device-qualification-11.0.0.zip

Set the ESDQ_INSTALLATION variable to point to the desired installation location. For example, if you want to install the the Intel® ESDQ CLI under the ~/esdq directory, just set the this variable as follows:

export ESDQ_INSTALLATION=~/esdq
mkdir $ESDQ_INSTALLATION
Copy to clipboard

Directly from the download directory, unzip the downloaded file into the installation location.

unzip edge-software-device-qualification-11.0.0.zip -d $ESDQ_INSTALLATION
Copy to clipboard

Set the convenient ROBOTICS_SDK variable that is going to be used in the next installation steps.

export ROBOTICS_SDK=$ESDQ_INSTALLATION/edge-software-device-qualification-11.0.0/
Copy to clipboard

Install the Intel® ESDQ CLI executing the following commands:

cd $ROBOTICS_SDK
./setup.sh -i
export PATH=$PATH:$HOME/.local/bin
Copy to clipboard

Check the successful installation of the Intel® ESDQ CLI verifying that the execution of the following command prints Version: 11.0.0 on the terminal:

esdq --version
Copy to clipboard

Download and Install the Test Modules

To download and install the Intel® Robotics SDK test modules on your target device follow the steps below:

  1. Install the ros-humble-amr-esdq Deb package from Intel® Robotics SDK APT repository.

    sudo apt update
    sudo apt install ros-humble-amr-esdq
    
    Copy to clipboard
  2. The tests are conducted from the directory pointed by the previously set ROBOTICS_SDK variable. Copy the installed test suite into the directory.

    cp -r /opt/ros/humble/share/amr-esdq/AMR_Test_Module/ $ROBOTICS_SDK/modules/
    
    Copy to clipboard
  3. Verify the appropriate permissions for the test modules directory by executing the following command:

    cd $ROBOTICS_SDK
    chmod -R +xw  modules/AMR_Test_Module
    
    Copy to clipboard
  4. Check that the Intel® Robotics SDK test module is correctly installed by verifying that the output of the following command lists the Robotics_SDK module.

    esdq module list
    
    Copy to clipboard
  5. Download the necessary assets required by the test suite.

    esdq --verbose module run Robotics_SDK --arg download
    
    Copy to clipboard

Run the Self-Certification Application for Compute Systems

  1. Use the groups command to verify whether the current user belongs to the render, video, and dialout groups. If the user does not belong to these groups, add the group membership by means of:

    sudo usermod -a -G render,video,dialout $USER
    
    Copy to clipboard

    Log out and log in again.

  2. If you have just installed the ros-humble-amr-esdq Deb package as described in the Download and Install Intel® ESDQ for Robotics SDK section, reboot your system.

    Otherwise, there is a possibility that the tests that depend on the GPU ORB Extractor encounter issues accessing the GPU. Further information can be found in the L0 error 78000001 in a native environment section of the Kudan Visual SLAM documentation. If you plan to integrate the Intel® Robotics SDK into a containerized environment, also consider the hints in the Running Kudan Visual SLAM in a containerized environment section.

  3. Make sure that the environment variable ROBOTICS_SDK is initialized as shown in Download and Install Intel® ESDQ for Robotics SDK and change the working directory:

    echo $ROBOTICS_SDK
    cd $ROBOTICS_SDK
    
    Copy to clipboard
  4. If your system uses a Linux* Kernel 6.7.5 or later, read the section GPU device is not detected with Linux* Kernel 6.7.5 or later. If your system is impacted by this issue, export the following debug variables as a workaround:

    export NEOReadDebugKeys=1
    export OverrideGpuAddressSpace=48
    
    Copy to clipboard
  5. Run the Intel® ESDQ test, and generate the report:

    export ROS_DOMAIN_ID=19
    esdq --verbose module run Robotics_SDK
    
    Copy to clipboard
  6. Visualize the report by opening the reports/report.html file in your browser.

    Expected output (These results are for illustration purposes only.)

    ../../_images/esdq_execution_summary.png ../../_images/esdq_test_module.png ../../_images/esdq_test_results.png

    Note

    All the tests are expected to pass. The Intel® VTune™ Profiler test failure and the Intel® RealSense™ test skip above are shown for demonstration purposes only. For example, the execution of the Intel® RealSense™ test is skipped if no Intel® RealSense™ camera is connected to the target system.

    If individual test cases do not pass, you can check the detailed log files in folder $ROBOTICS_SDK/modules/AMR_Test_Module/output/.

Run the Self-Certification Application for RGB Cameras

This self-certification test expects the camera stream to be on the /camera/color/image_raw topic. This topic must be visible in rviz2 using the camera_color_frame fixed frame. If your camera ROS 2 node does not stream to that topic by default, use ROS 2 remapping to publish to that topic.

Note

The following steps use the Intel® RealSense™ ROS 2 node as an example. You must change the node to your actual camera ROS 2 node.

You can check your current configuration by:

  1. Running the RGB camera node in a ROS 2 environment after setting the ROS_DOMAIN_ID.

    source /opt/ros/humble/setup.bash
    # set a unique id here that is used in all terminals
    export ROS_DOMAIN_ID=19
    ros2 launch realsense2_camera rs_launch.py camera_namespace:=/ &
    
    Copy to clipboard
  2. Verifying the presence of the topic in the topic list.

    ros2 topic list
    
    Copy to clipboard
  3. Once your configuration is set, you can proceed to run the Intel® ESDQ test and generate the report.

    cd $ROBOTICS_SDK
    export ROS_DOMAIN_ID=19
    esdq --verbose module run Robotics_SDK --arg sensors_rgb
    
    Copy to clipboard

Run the Self-Certification Application for Depth Cameras

This self-certification test expects the camera stream to be on the /camera/depth/color/points and on the /camera/depth/image_rect_raw topics. These topics must be visible in rviz2 using the camera_link fixed frame. If your camera ROS 2 node does not stream to that topic by default, use ROS 2 remapping to publish to that topic.

Note

The following steps use the Intel® RealSense™ ROS 2 node as an example. You must change the node to your actual camera ROS 2 node.

You can check your current configuration by:

  1. Running the depth camera node in a ROS 2 environment after setting the ROS_DOMAIN_ID.

    source /opt/ros/humble/setup.bash
    # set a unique id here that is used in all terminals
    export ROS_DOMAIN_ID=19
    ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true camera_namespace:=/ &
    
    Copy to clipboard
  2. Verifying the presence of the topic in the topic list.

    ros2 topic list
    
    Copy to clipboard
  3. Once your configuration is set, you can proceed to run the Intel® ESDQ test and generate the report.

    cd $ROBOTICS_SDK
    export ROS_DOMAIN_ID=19
    esdq --verbose module run Robotics_SDK --arg sensors_depth
    
    Copy to clipboard

Send Results to Intel®

Once the automated and manual tests are executed successfully, you can submit your test results and get your devices listed on the Intel® Edge Software Recommended Hardware site.

Send the zip file that is created after running Intel® ESDQ tests to: edge.software.device.qualification@intel.com.

For example, after one of our local runs the following files were generated in the $ROBOTICS_SDK/reports/ directory: report.html and report.zip.

Troubleshooting

For issues, go to: Troubleshooting for Robotics SDK Tutorials .

Support Forum

If you’re unable to resolve your issues, contact the Support Forum.