Jetson Orin NX 开发指南（4）: 安装 CUDA 和 Realsense

一、硬件说明

可以在官网查看各个类型的相机说明

Intel® RealSense™ Computer Vision - Depth and Tracking cameras

目前实验室常用的是 D400 系列的相机，如 D435i相机，D455相机等，本文主要使用 D435i 相机

D435i 相机是一个RGB-D相机，也就是我们常说的深度相机，目前深度相机的实现原理主要分为三种：分别是结构光，Tof，双目成像。

Realsense 使用的是结构光的方案。正面的四个摄像头，从左向右以次是左红外相机，红外点阵投射仪，右红外相机，和RGB相机。

二、安装 cuda

由于 Jetson 的 CPU 性能较差，因此安装 Realsense 前建议先安装 cuda，从而在启动 Realsense 时可以通过 GPU 进行加速，因此这里先安装 cuda，由于我们在烧录 Jetson Orin NX 开发板时安装了相关组件，其中就包括 cuda 组件，因此可以直接通过 apt-get 安装 cuda，终端输入

sudo apt-get update
sudo apt-get install cuda-toolkit-11-4

安装完成后查看是否已经 export 安装路径

如果没有这两行则加入这两行

export PATH=/usr/local/cuda-11.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH

添加之后 source 一下，使其生效

source ~/.bashrc

最后检查 cuda 是否安装成功，终端输入

nvcc -V

这表明 cuda 安装成功！

三、安装 realsense 驱动

Jetson 系列有专门针对 realsense 安装方式，当然传统的 realsense 安装方式也是可行的

3.1 传统安装 librealsense

传统的安装方式可以参考下文

3.2 Jetson 方法安装 librealsense

但是这里我们采用专门针对 Jetson 的方式进行安装 librealsense，参考

首先从 github 上下载安装包，终端输入

git clone https://github.com/jetsonhacks/installRealSenseSDK.git

下载完成后，进入安装包并开始安装，终端输入

cd ./installRealSenseSDK
./installLibrealsense.sh

至此 Librealsense 就安装成功了，最后测试 realsense 驱动是否能够正常运行。

首先将 realsense 相机连接上 Jetson Orin NX 开发板（注意这里必须使用 USB 3.0 的数据线连接），然后终端输入

realsense-viewer

正常显示图片

这表明 Realsense 驱动安装成功！

四、安装 realsense-ros

作用：将 realsense 数据转换为 ros 数据并发布到 ros 话题中，相当于realsense 和 ros 的桥梁。

参考 realsense-ros 的 github 官方教程：

GitHub - IntelRealSense/realsense-ros at ros1-legacy

官方提供了直接方式和源码方式两种方法安装 realsense-ros

4.1 直接方式安装

终端输入：

sudo apt-get install ros-$ROS_DISTRO-realsense2-camera

其中 $ROS_DISTRO 为自己的 ros 版本， Ubuntu 18.04 对应 melodic，Ubuntu 20.04 对应 noetic，这种方法的安装路径为

/opt/ros/noetic/share/realsense2_camera/

4.2 源码方式安装

创建工作目录

mkdir -p ~/catkin_pkg/src/
cd ~/catkin_pkg/src/

下载源码

git clone https://github.com/IntelRealSense/realsense-ros.git
cd realsense-ros/
git checkout `git tag | sort -V | grep -P "^2.\d+\.\d+" | tail -1`
cd ..

请确保安装了 ros 软件包 ddynamic_reconfigure，如果没有安装，可以通过 apt 安装

sudo apt-get install ros-$ROS_DISTRO-ddynamic-reconfigure

或者将其源码下载到当前工作目录

git clone https://github.com/pal-robotics/ddynamic_reconfigure.git

注意，这里如果通过 apt 安装了 ddynamic_reconfigure 就不要再下载源码到工作空间一起编译了，可能会冲突报错。

接下来编译工作空间

catkin_init_workspace
cd ..
catkin_make clean
catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release
catkin_make install

最后将其加到 ~/.basrc 文件中

echo "source ~/catkin_pkg/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc

这两种方法选择一种安装就可以。

测试 realsense-ros 是否安装成功，通过 ros 启动相机，终端输入

roslaunch realsense2_camera rs_camera.launch

如果发现报错
[camera/realsense2_camera_manager-2] process has died [pid 3282, exit code 127, cmd /opt/ros/noetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/onx001/.ros/log/064220d6-6818-11ee-82cd-347de4babe3e/camera-realsense2_camera_manager-2.log].

参考

No GPU usage on jetson xavier NX · Issue #2326 · IntelRealSense/realsense-ros · GitHub

librealsense2 camera.so: undefined symbol:ZN2cV3MatC1EV解决笔记记录-CSDN博客

查看能否收到相机数据。

解决方案

修改 CMakeLists.txt 文件

添加 OpenCV 功能包，由于安装 ROS 时自带 OpenCV，因此不需要另外安装 OpenCV，直接修改 CMakeLists.txt 文件即可

上面红字为添加的内容，也可以复制下面写的

find_package(catkin REQUIRED COMPONENTS
    message_generation
    nav_msgs
    roscpp
    sensor_msgs
    std_msgs
    std_srvs
    nodelet
    cv_bridge
    image_transport
    tf
    ddynamic_reconfigure
    diagnostic_updater
    OpenCV
    )

include_directories(
    include
    ${realsense2_INCLUDE_DIR}
    ${catkin_INCLUDE_DIRS}
    ${OpenCV_INCLUDE_DIRS}
    )

target_link_libraries(${PROJECT_NAME}
    ${realsense2_LIBRARY}
    ${catkin_LIBRARIES}
    ${CMAKE_THREAD_LIBS_INIT}
    ${OpenCV_LIBRARIES}
    )

重新测试 realsense-ros 是否安装成功，通过 ros 启动相机，终端输入

roslaunch realsense2_camera rs_camera.launch

终端输入 rostopic list 显示话题

onx001@onx001:~$ rostopic list 
/camera/color/camera_info
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressed/parameter_descriptions
/camera/color/image_raw/compressed/parameter_updates
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/compressedDepth/parameter_descriptions
/camera/color/image_raw/compressedDepth/parameter_updates
/camera/color/image_raw/theora
/camera/color/image_raw/theora/parameter_descriptions
/camera/color/image_raw/theora/parameter_updates
/camera/color/metadata
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/motion_module/parameter_descriptions
/camera/motion_module/parameter_updates
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/auto_exposure_roi/parameter_descriptions
/camera/rgb_camera/auto_exposure_roi/parameter_updates
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/auto_exposure_roi/parameter_descriptions
/camera/stereo_module/auto_exposure_roi/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates
/diagnostics
/rosout
/rosout_agg
/tf
/tf_static
onx001@onx001:~$

这表明 realsense-ros 安装成功！

4.3 修改配置文件

一般来说，安装完成后可能会出现 ros 能接收到话题但是收不到数据的情况。

这时需要修改 ~/catkin_pkg/src/realsense2_camera/rs_camera.launch 文件。

先备份 rs_camera.launch 文件，终端输入

cd ~/catkin_pkg/src/realsense-ros/realsense2_camera/launch
cp ./rs_camera.launch ./rs_camera.launch.bak

然后将 rs_camera.launch 文件内容替换为如下内容

<launch>
  <arg name="serial_no"           default=""/>
  <arg name="usb_port_id"         default=""/>
  <arg name="device_type"         default=""/>
  <arg name="json_file_path"      default=""/>
  <arg name="camera"              default="camera"/>
  <arg name="tf_prefix"           default="$(arg camera)"/>
  <arg name="external_manager"    default="false"/>
  <arg name="manager"             default="realsense2_camera_manager"/>
  <arg name="output"              default="screen"/>
  <arg name="respawn"              default="false"/>

  <arg name="fisheye_width"       default="-1"/>
  <arg name="fisheye_height"      default="-1"/>
  <arg name="enable_fisheye"      default="false"/>

  <arg name="depth_width"         default="640"/>
  <arg name="depth_height"        default="480"/>
  <arg name="enable_depth"        default="true"/>

  <arg name="confidence_width"    default="-1"/>
  <arg name="confidence_height"   default="-1"/>
  <arg name="enable_confidence"   default="false"/>
  <arg name="confidence_fps"      default="-1"/>

  <arg name="infra_width"         default="640"/>
  <arg name="infra_height"        default="480"/>
  <arg name="enable_infra"        default="false"/>
  <arg name="enable_infra1"       default="true"/>
  <arg name="enable_infra2"       default="true"/>
  <arg name="infra_rgb"           default="false"/>

  <arg name="color_width"         default="-1"/>
  <arg name="color_height"        default="-1"/>
  <arg name="enable_color"        default="false"/>

  <arg name="fisheye_fps"         default="-1"/>
  <arg name="depth_fps"           default="30"/>
  <arg name="infra_fps"           default="30"/>
  <arg name="color_fps"           default="-1"/>
  <arg name="gyro_fps"            default="400"/>
  <arg name="accel_fps"           default="400"/>
  <arg name="enable_gyro"         default="true"/>
  <arg name="enable_accel"        default="true"/>

  <arg name="enable_pointcloud"         default="false"/>
  <arg name="pointcloud_texture_stream" default="RS2_STREAM_COLOR"/>
  <arg name="pointcloud_texture_index"  default="0"/>
  <arg name="allow_no_texture_points"   default="false"/>
  <arg name="ordered_pc"                default="false"/>

  <arg name="enable_sync"               default="false"/>
  <arg name="align_depth"               default="false"/>

  <arg name="publish_tf"                default="true"/>
  <arg name="tf_publish_rate"           default="0"/>

  <arg name="filters"                   default=""/>
  <arg name="clip_distance"             default="-2"/>
  <arg name="linear_accel_cov"          default="0.01"/>
  <arg name="initial_reset"             default="false"/>
  <arg name="reconnect_timeout"         default="6.0"/>
  <arg name="wait_for_device_timeout"   default="-1.0"/>
  <arg name="unite_imu_method"          default="linear_interpolation"/>
  <arg name="topic_odom_in"             default="odom_in"/>
  <arg name="calib_odom_file"           default=""/>
  <arg name="publish_odom_tf"           default="true"/>

  <arg name="stereo_module/exposure/1"  default="7500"/>
  <arg name="stereo_module/gain/1"      default="16"/>
  <arg name="stereo_module/exposure/2"  default="1"/>
  <arg name="stereo_module/gain/2"      default="16"/>


  <arg name="emitter_enable"   	default="false"/>
  <!-- rosparam set /camera/stereo_module/emitter_enabled false -->
  <rosparam>
    /camera/stereo_module/emitter_enabled: 0
  </rosparam>
  <rosparam if="$(arg emitter_enable)">
    /camera/stereo_module/emitter_enabled: 1
  </rosparam>


  <group ns="$(arg camera)">
    <include file="$(find realsense2_camera)/launch/includes/nodelet.launch.xml">
      <arg name="tf_prefix"                value="$(arg tf_prefix)"/>
      <arg name="external_manager"         value="$(arg external_manager)"/>
      <arg name="manager"                  value="$(arg manager)"/>
      <arg name="output"                   value="$(arg output)"/>
      <arg name="respawn"                  value="$(arg respawn)"/>
      <arg name="serial_no"                value="$(arg serial_no)"/>
      <arg name="usb_port_id"              value="$(arg usb_port_id)"/>
      <arg name="device_type"              value="$(arg device_type)"/>
      <arg name="json_file_path"           value="$(arg json_file_path)"/>

      <arg name="enable_pointcloud"        value="$(arg enable_pointcloud)"/>
      <arg name="pointcloud_texture_stream" value="$(arg pointcloud_texture_stream)"/>
      <arg name="pointcloud_texture_index"  value="$(arg pointcloud_texture_index)"/>
      <arg name="enable_sync"              value="$(arg enable_sync)"/>
      <arg name="align_depth"              value="$(arg align_depth)"/>

      <arg name="fisheye_width"            value="$(arg fisheye_width)"/>
      <arg name="fisheye_height"           value="$(arg fisheye_height)"/>
      <arg name="enable_fisheye"           value="$(arg enable_fisheye)"/>

      <arg name="depth_width"              value="$(arg depth_width)"/>
      <arg name="depth_height"             value="$(arg depth_height)"/>
      <arg name="enable_depth"             value="$(arg enable_depth)"/>

      <arg name="confidence_width"         value="$(arg confidence_width)"/>
      <arg name="confidence_height"        value="$(arg confidence_height)"/>
      <arg name="enable_confidence"        value="$(arg enable_confidence)"/>
      <arg name="confidence_fps"           value="$(arg confidence_fps)"/>

      <arg name="color_width"              value="$(arg color_width)"/>
      <arg name="color_height"             value="$(arg color_height)"/>
      <arg name="enable_color"             value="$(arg enable_color)"/>

      <arg name="infra_width"              value="$(arg infra_width)"/>
      <arg name="infra_height"             value="$(arg infra_height)"/>
      <arg name="enable_infra"             value="$(arg enable_infra)"/>
      <arg name="enable_infra1"            value="$(arg enable_infra1)"/>
      <arg name="enable_infra2"            value="$(arg enable_infra2)"/>
      <arg name="infra_rgb"                value="$(arg infra_rgb)"/>

      <arg name="fisheye_fps"              value="$(arg fisheye_fps)"/>
      <arg name="depth_fps"                value="$(arg depth_fps)"/>
      <arg name="infra_fps"                value="$(arg infra_fps)"/>
      <arg name="color_fps"                value="$(arg color_fps)"/>
      <arg name="gyro_fps"                 value="$(arg gyro_fps)"/>
      <arg name="accel_fps"                value="$(arg accel_fps)"/>
      <arg name="enable_gyro"              value="$(arg enable_gyro)"/>
      <arg name="enable_accel"             value="$(arg enable_accel)"/>

      <arg name="publish_tf"               value="$(arg publish_tf)"/>
      <arg name="tf_publish_rate"          value="$(arg tf_publish_rate)"/>

      <arg name="filters"                  value="$(arg filters)"/>
      <arg name="clip_distance"            value="$(arg clip_distance)"/>
      <arg name="linear_accel_cov"         value="$(arg linear_accel_cov)"/>
      <arg name="initial_reset"            value="$(arg initial_reset)"/>
      <arg name="reconnect_timeout"        value="$(arg reconnect_timeout)"/>
      <arg name="wait_for_device_timeout"  value="$(arg wait_for_device_timeout)"/>
      <arg name="unite_imu_method"         value="$(arg unite_imu_method)"/>
      <arg name="topic_odom_in"            value="$(arg topic_odom_in)"/>
      <arg name="calib_odom_file"          value="$(arg calib_odom_file)"/>
      <arg name="publish_odom_tf"          value="$(arg publish_odom_tf)"/>
      <arg name="stereo_module/exposure/1" value="$(arg stereo_module/exposure/1)"/>
      <arg name="stereo_module/gain/1"     value="$(arg stereo_module/gain/1)"/>
      <arg name="stereo_module/exposure/2" value="$(arg stereo_module/exposure/2)"/>
      <arg name="stereo_module/gain/2"     value="$(arg stereo_module/gain/2)"/>

      <arg name="allow_no_texture_points"  value="$(arg allow_no_texture_points)"/>
      <arg name="ordered_pc"               value="$(arg ordered_pc)"/>
      
    </include>
  </group>
</launch>

最后终端输入

roslaunch realsense2_camera rs_camera.launch

此时输出 imu 等信息，并且屏蔽了结构光

/camera/accel/imu_info
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth/metadata
/camera/extrinsics/depth_to_infra1
/camera/extrinsics/depth_to_infra2
/camera/gyro/imu_info
/camera/imu
/camera/infra1/camera_info
/camera/infra1/image_rect_raw
/camera/infra1/image_rect_raw/compressed
/camera/infra1/image_rect_raw/compressed/parameter_descriptions
/camera/infra1/image_rect_raw/compressed/parameter_updates
/camera/infra1/image_rect_raw/compressedDepth
/camera/infra1/image_rect_raw/compressedDepth/parameter_descriptions
/camera/infra1/image_rect_raw/compressedDepth/parameter_updates
/camera/infra1/image_rect_raw/theora
/camera/infra1/image_rect_raw/theora/parameter_descriptions
/camera/infra1/image_rect_raw/theora/parameter_updates
/camera/infra1/metadata
/camera/infra2/camera_info
/camera/infra2/image_rect_raw
/camera/infra2/image_rect_raw/compressed
/camera/infra2/image_rect_raw/compressed/parameter_descriptions
/camera/infra2/image_rect_raw/compressed/parameter_updates
/camera/infra2/image_rect_raw/compressedDepth
/camera/infra2/image_rect_raw/compressedDepth/parameter_descriptions
/camera/infra2/image_rect_raw/compressedDepth/parameter_updates
/camera/infra2/image_rect_raw/theora
/camera/infra2/image_rect_raw/theora/parameter_descriptions
/camera/infra2/image_rect_raw/theora/parameter_updates
/camera/infra2/metadata
/camera/motion_module/parameter_descriptions
/camera/motion_module/parameter_updates
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/auto_exposure_roi/parameter_descriptions
/camera/stereo_module/auto_exposure_roi/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates
/diagnostics
/rosout
/rosout_agg
/tf
/tf_static

至此，Realsense 驱动就安装完成了，并且实现了与 ROS 系统的连接！