Wandering App Based on a Collab SLAM Using AAEON Robot Kits#
Hardware Prerequisite
You have one of these AAEON* robot kits:
This tutorial uses the UP Xtreme i11 Robotic Kit.
If you need help assembling your robot, see AAEON* Resources.
You can use one of these teleop methods to validate that the robot kit hardware setup was done correctly.
-
Start at step 2 (insert the USB dongle in the robot); and, for step 5, run the yml file exactly as shown in the example (ignore the instruction to replace it with your own generic yml file).
-
For step 2, instead of customizing your file, use the exact command in the example.
Note
The full-sdk docker image is only present in the Robot Complete Kit, not in the Robot Base Kit or Up Xtreme i11 Robotic Kit.
Check Your Installation#
Check if your installation includes the following Docker* images:
docker images |egrep "amr-aaeon-amr-interface|amr-ros-base|amr-imu-tools|amr-robot-localization|amr-nav2|amr-wandering|amr-realsense|amr-collab-slam|amr-collab-slam-gpu|" #if you have them installed, the result is: amr-aaeon-amr-interface amr-ros-base amr-imu-tools amr-robot-localization amr-realsense amr-collab-slam amr-collab-slam-gpu amr-nav2 amr-wandering
Note
If these images are not installed, continuing with these steps triggers a build that takes longer than an hour (sometimes, a lot longer depending on the system resources and internet connection).
If these images are not installed, Intel® recommends installing the Robot Complete Kit with the Get Started Guide for Robots.
Check that EI for AMR environment is set:
echo $AMR_TUTORIALS # should output the path to EI for AMR tutorials /home/user/edge_insights_for_amr/Edge_Insights_for_Autonomous_Mobile_Robots_2023.1/AMR_containers/01_docker_sdk_env/docker_compose/05_tutorials
If nothing is output, refer to Get Started Guide for Robots Step 5 for information on how to configure the environment.
Calibrate Your Robot’s Inertial Measurement Unit (IMU) Sensor#
The IMU sensor is used to determine the robot’s orientation. Moving the robot interferes with calibration, so do not move the robot while performing these steps.
Run aaeon-imu-calibration.sh script. The script pauses to allow you to make sure that the robot is standing still. Press Enter to continue after checking the robot state:
$ $CONTAINER_BASE_PATH/01_docker_sdk_env/artifacts/01_amr/aaeon/calib $ ./aaeon-imu-calibration.sh Calibrating robot. Make sure the robot is not moving and it is on a plane surface. Press Enter to start calibration: [+] Running 2/0 ✔ Container 05_tutorials-aaeon-amr-interface-1 Created 0.0s ✔ Container 05_tutorials-aaeon-imu-calibration-1 Created 0.0s Attaching to 05_tutorials-aaeon-amr-interface-1, 05_tutorials-aaeon-imu-calibration-1 05_tutorials-aaeon-amr-interface-1 | 05_tutorials-aaeon-imu-calibration-1 | 05_tutorials-aaeon-imu-calibration-1 | ros_distro = foxy 05_tutorials-aaeon-imu-calibration-1 | USER = eiforamr 05_tutorials-aaeon-imu-calibration-1 | User's HOME = /home/eiforamr 05_tutorials-aaeon-imu-calibration-1 | ROS_HOME = /home/eiforamr/.ros 05_tutorials-aaeon-imu-calibration-1 | ROS_LOG_DIR = /home/eiforamr/.ros/ros_log 05_tutorials-aaeon-imu-calibration-1 | ROS_WORKSPACE = /home/eiforamr/ros2_ws 05_tutorials-aaeon-imu-calibration-1 | 05_tutorials-aaeon-imu-calibration-1 | Sleep 5 seconds to allow aaeon_amr_interface to start 05_tutorials-aaeon-amr-interface-1 | ros_distro = foxy 05_tutorials-aaeon-amr-interface-1 | USER = eiforamr 05_tutorials-aaeon-amr-interface-1 | User's HOME = /home/eiforamr 05_tutorials-aaeon-amr-interface-1 | ROS_HOME = /home/eiforamr/.ros 05_tutorials-aaeon-amr-interface-1 | ROS_LOG_DIR = /home/eiforamr/.ros/ros_log 05_tutorials-aaeon-amr-interface-1 | ROS_WORKSPACE = /home/eiforamr/ros2_ws 05_tutorials-aaeon-amr-interface-1 | 05_tutorials-aaeon-amr-interface-1 | Running aaeon-amr-interface for 18 seconds 05_tutorials-aaeon-amr-interface-1 | [INFO] [1684508864.652431716] [IoContext::IoContext]: Thread(s) Created: 2 05_tutorials-aaeon-amr-interface-1 | [INFO] [1684508864.653106183] [AMR_node]: Serial opened on /dev/serial/by-id/usb-1a86_USB_Serial-if00-port0 at 115200 05_tutorials-aaeon-amr-interface-1 | [INFO] [1684508865.668642650] [AMR_node]: Hardware is now online 05_tutorials-aaeon-imu-calibration-1 | Collecting data for 8 seconds... 05_tutorials-aaeon-imu-calibration-1 | Computed values 05_tutorials-aaeon-imu-calibration-1 | 05_tutorials-aaeon-imu-calibration-1 | # gyro: 05_tutorials-aaeon-imu-calibration-1 | gyro_x=-0.00426730 05_tutorials-aaeon-imu-calibration-1 | gyro_y=-0.00434478 05_tutorials-aaeon-imu-calibration-1 | gyro_z=0.00335319 05_tutorials-aaeon-imu-calibration-1 | # accelerometer: 05_tutorials-aaeon-imu-calibration-1 | acc_x=0.31487880 05_tutorials-aaeon-imu-calibration-1 | acc_y=-0.09670373 05_tutorials-aaeon-imu-calibration-1 | acc_z=0.00000000 05_tutorials-aaeon-imu-calibration-1 | Updated /home/tester/eiforamr-2023.1/01_docker_sdk_env/artifacts/01_amr/amr_generic/param/aaeon_node_params.yaml 05_tutorials-aaeon-imu-calibration-1 | Calibration done. 05_tutorials-aaeon-imu-calibration-1 exited with code 0 05_tutorials-aaeon-amr-interface-1 | Exiting. 05_tutorials-aaeon-amr-interface-1 exited with code 0 [+] Running 2/0 ✔ Container 05_tutorials-aaeon-imu-calibration-1 Removed 0.0s ✔ Container 05_tutorials-aaeon-amr-interface-1 Removed 0.0s Calibration finished.
After calibration completes, the robot is ready to use.
Map an Area with the Wandering Application and UP Xtreme i11 Robotic Kit#
The goal of the Wandering application is to map an area and avoid hitting objects.
Place the robot in an area with multiple objects in it.
Start mapping the area:
docker compose -f $AMR_TUTORIALS/aaeon_mapping_realsense_collab_slam_nav2_ukf.tutorial.yml up
Expected result: The robot starts wandering around the room and mapping the entire area.
On a different terminal, prepare the environment to visualize the mapping and the robot using rviz2.
Note
If available, use a different development machine because rviz2 consumes a lot of resources that may interfere with the robot.
# If using a different machine set the same ROS_DOMAIN_ID as on the robot export ROS_DOMAIN_ID=27 docker compose -f $AMR_TUTORIALS/rviz_robot_wandering.yml up
To see the map in 3D, can check the MarkerArray:
Note
Displaying in 3D consumes a lot of the system resources. Intel® recommends opening rviz2 on a development system. The development system needs to be in the same network and have the same ROS_DOMAIN_ID set.
To stop the robot from mapping the area, do the following:
Type
Ctrl-cin the terminal executing the Docker* containerUse
docker compose downcommand to remove the containers:
docker compose -f $AMR_TUTORIALS/aaeon_wandering__aaeon_realsense_collab_slam_fm_nav2_ukf.tutorial.yml down --remove-orphans
If the robot moves in an unpredictable way and hits objects easily, there may be some hardware configuration issues. See the Troubleshooting section for suggestions.
Start the UP Xtreme i11 Robotic Kit in Localization Mode#
Prerequisites:
Collaborative visual SLAM is based on visual keypoints, so use a room with multiple obstacles in it.
If the room has bland walls, consider adding pictures to it so that visual SLAM has enough keypoints to localize itself.
Localization mode only works if a map was already generated. To generate a map go to Map an Area with the Wandering Application and UP Xtreme i11 Robotic Kit.
The pre-generated file contains enough information to allow the robot to navigate without having to map the entire area.
With this pre-generated map, the robot only has to localize itself relative to the pre-generated map.
Run the command:
docker compose -f $AMR_TUTORIALS/aaeon_localization_realsense_collab_slam_nav2_ukf.tutorial.yml up
Expected result: The robot starts moving in the already mapped area and reports if it is able to localize itself or not.
If the robot is able to localize itself, the amr-collab-slam node increases the tracking success number, and the “relocal fail number” stays constant:
amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.720] [info] tracking success number: 102, fail number: 0, relocal number: 1, relocal fail number: 51 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.766] [info] Started Localization! current frame id is 154 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.779] [info] valid tracked server landmarks num: 590 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.779] [info] Successfully tracked server map! amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.785] [info] tracking success number: 103, fail number: 0, relocal number: 1, relocal fail number: 51 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.835] [info] Started Localization! current frame id is 155 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.848] [info] valid tracked server landmarks num: 571 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 06:59:58.848] [info] Successfully tracked server map!
If the robot is not able to localize itself, the amr-collab-slam node keeps the tracking success number constant and increases the “relocal fail number”:
amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 07:06:49.475] [info] tracking success number: 5740, fail number: 6, relocal number: 6, relocal fail number: 119 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 07:06:49.584] [info] tracking success number: 5740, fail number: 6, relocal number: 6, relocal fail number: 120 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 07:06:49.667] [info] tracking success number: 5740, fail number: 6, relocal number: 6, relocal fail number: 121 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 07:06:49.806] [info] tracking success number: 5740, fail number: 6, relocal number: 6, relocal fail number: 122 amr-collab-slam | [univloc_tracker_ros-2] [2022-09-05 07:06:49.859] [info] tracking success number: 5740, fail number: 6, relocal number: 6, relocal fail number: 123
To visualize the robot localizing itself and updating its pose, run in a different terminal:
docker compose -f $AMR_TUTORIALS/rviz_robot_localization.yml up
Note
If the robot is not able to localize itself, the robot does not start navigating the room, and rviz2 reports map not found. To avoid this, move the robot to the room where the map was created and face it towards a keypoint. It also helps if the room you mapped has a lot of keypoints
Perform Object Detection While Mapping an Area with the UP Xtreme i11 Robotic Kit#
Place the robot in an area with multiple objects in it.
Start mapping the area and listing the detected objects:
docker compose -f $AMR_TUTORIALS/aaeon_wandering_local_inference.yml up | grep Label
Expected result: The robot starts wandering around the room and listing what objects it sees:
AAEON* Resources#
Development Kit: up-board/up-community
Hardware Assembly: up-board/up-community
Power Management: up-board/up-community
Troubleshooting#
If the server fails to load the map:
amr-collab-slam | [univloc_server-1] [2022-09-19 06:45:08.520] [critical] cannot load the file at /tmp/aaeon/maps/map.msg
amr-collab-slam | [univloc_server-1] terminate called after throwing an instance of 'std::runtime_error'
amr-collab-slam | [univloc_server-1] what(): cannot load the file at /tmp/aaeon/maps/map.msg
amr-collab-slam | [ERROR] [univloc_server-1]: process has died [pid 72, exit code -6, cmd '/home/eiforamr/workspace/CollabSLAM/prebuilt_collab_slam_core/univloc_server/lib/univloc_server/univloc_server --ros-args -r __node:=univloc_server --params-file /tmp/launch_params_iiuu5pfp'].
Change your folder permissions so that the Docker* user is able to write the map on your system:
cd $CONTAINER_BASE_PATH
sudo find . -type d -exec chmod 775 {} +
sudo chown $USER:$USER * -R
If the tracker (univloc_tracker_ros) fails to start, giving the following
error, see Collaborative Visual SLAM on Intel® Atom® Processor-Based Systems.
amr-collab-slam | [ERROR] [univloc_tracker_ros-2]: process has died [pid 140, exit code -4, cmd '/home/eiforamr/workspace/CollabSLAM/prebuilt_collab_slam_core/univloc_tracker/lib/univloc_tracker/univloc_tracker_ros --ros-args -r __node:=univloc_tracker_0 -r __ns:=/ --params-file /tmp/launch_params_zfr70odz -r /tf:=tf -r /tf_static:=tf_static -r /univloc_tracker_0/map:=map'].
If the robot does not start moving, the firmware might be stuck. To make it work again:
cd $CONTAINER_BASE_PATH
./run_interactive_docker.sh amr-aaeon-amr-interface:2023.1 eiforamr -c aaeon_robot
ros2 run ros2_amr_interface amr_interface_node --ros-args -p try_reconnect:=true -p publishTF:=true --remap /amr/cmd_vel:=/cmd_vel -p port_name:=/dev/ttyUSB0
ctrl-c
ros2 run ros2_amr_interface amr_interface_node --ros-args -p try_reconnect:=true -p publishTF:=true --remap /amr/cmd_vel:=/cmd_vel -p port_name:=/dev/ttyUSB0
# Look for the text: [INFO] [1655311131.144572706] [AMR_node]: Hardware is now online
# If you don't get this repeat the commands from the docker image and check if the motor controller is not attached to /dev/ttyUSB0.
# If it is not attached to /dev/ttyUSB0, find out which one it is and adapt the commands accordingly.
# When you get the [INFO] [1655311131.144572706] [AMR_node]: Hardware is now online, exit the docker image:
exit
If the robot is not behaving as instructed when using the teleop_twist_keyboard, try the following steps.
Check the direction of the wheels. The way they are facing is very important, as shown in the following picture.
Each wheel says R (Right) or L (Left). Intel had to use the following wheel setup:
R (wheel) <<<>>> L (wheel)
L (wheel) <<<>>> R (wheel)
Check the connection between the wheels (left in the following picture) and the motor controller.
It is very important to have the hardware setup correctly configured. If it is not correct, it is evident when testing with the teleop_twist_keyboard.
The wheels not turning suggests a possible problem with the motor control board. The specified input voltage in the board’s datasheet is 12V. However, testing revealed that inputs of 5V, 8V, and 10V inputs do work properly.
If the IMU gives errors and you did not install the librealsense udev rules when you configured the host, install the librealsense udev rules now:
git clone https://github.com/IntelRealSense/librealsense
# Copy the 99-realsense-libusb.rules files to the rules.d folder
cd librealsense
sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger
For general robot issues, go to: Troubleshooting for Robot Tutorials.