The hardware and software architecture required to acquire real-time data from the drone has taken form during the fall 2023 semester. The foundation of the system relies on ROS, or the Robot Operating System written in Python and running on Ubuntu 20.04, a Linux distribution. ROS is a library of software tools like a group of application programming interfaces (APIs) that use a common syntax and extend to include more complex tools like Gazebo. ROS is used by DEL to acquire software-in-the-loop (SITL) and hardware-in-the-loop (HITL) data. SITL testing is based on flight controller (FC) firmware namely PX4. SITL testing also uses a ROS-integrated physics simulator called Gazebo. HITL operations are currently based on a Pixhawk FC, telemetry radio, Jetson Nano, and a ground station laptop.
Software Overview
Software-in-the-loop Architecture
ROS provides the frame of the software architecture for real-time data acquisition. ROS allows multiple packages to run simultaneously and communicate. SITL testing is based on a local version of the PX4 FC firmware: a copy of the firmware “simulated” by running directly on the ground station laptop. SITL testing starts by launching the PX4 firmware on a laptop using a provided C++ launch file. The firmware is a “local” version since it is running on thesame laptop with which it is communicating. To visualize the local data, Gazebo can be launched with a stock quadcopter. The drone in Gazebo is essentially “flying” as if the local PX4 were running on said drone. Next, a custom ROS package is launched that communicates with the local PX4. PX4 and other common firmware use the MAVLink, or micro-air vehicle link, communication protocol: during SITL testing the local PX4 package helps communicate MAVLink signals to ROS.
To get real-time data from the local PX4 into Python using ROS, a MAVROS node based on the FC firmware is launched. With a MAVROS node launched in one terminal, other packages can be launched from the same ROS workspace in separate terminals to communicate with MAVROS. DEL uses subscriber and publisher nodes to receive data from MAVROS. The subscriber listens to MAVROS trajectory messages and processes that data with a ‘callback’ function. Processing includes position estimation and data visualization. For SITL testing a publisher node is defined in the same script to control the local PX4 and Gazebo drone. Any kind of remote control could be completed using the Python/ROS system where feedback is read by a subscriber and controls are relayed by a publisher. However, for SITL testing DEL uses an arbitrary control namely a high-level position set point command. Similarly, real trajectory data could be sent to Gazebo during flight to visualize a flight path. The diagram above describes the ROS architecture for SITL testing and software operations in general.
Results and Discussion:
SITL testing provides a cheap testbed for all software components needed to complete the current goals of DEL. SITL testing saves on both financial and schedule costs. Since all of the software used in the final product of DEL operations is open source, the final software architecture is free to use. From a schedule standpoint, the local PX4 gives the ground control laptop access to most of what is given by the real drone PX4. For checks and development, time-consuming flight tests can be replaced by convenient SITL simulations. Altogether, SITL testing has provided DEL with the ability to troubleshoot and refine software in parallel to hardware development. The main DEL Python script was developed using only SITL testing, including ROS communications. Data visualization using matplotlib was also completely developed with only SITL. A significant benefit of using Python and ROS becomes evident: with a different low-level communication tool, the exact same source code can be used for SITL and HITL operations.
Hardware-in-the-loop Architecture
Software Adjustments from SITL Scripts:
Most of the SITL software translates exactly to HITL operations. The first difference is in where MAVLink signals are coming, and what tool processes said signals. For HITL operations, MAVLink messages will be communicated by the FC, and received by QGroundControl (QGC) on the ground station laptop. Although there are multiple hardware configurations described below, they all rely on QGC to process MAVLink signals. QGC is an open-source ground control software mostly written in C++. So, hardware operations start by launching QGC and connecting the FC. Next, MAVROS is launched to accept forwarded messages from QGC. Finally, a custom ROS package is launched which subscribes to MAVROS messages, processes data, and visualizes trajectory. A key change in the HITL source code is to remove the high-level control input used during SITL testing: DEL does not control the real drone from Python. Removing all automatic, software commands avoids accidentally controlling a drone that has propellers installed which is important since DEL uses wired testing frequently. In short, after changing low-level communications, the only change required to move from SITL to HITL testing is to remove a small portion of code for safety. Small additional changes will be mentioned below. Note an alternative to QGC that DEL is researching is mavgen and Pymavgen which may be able to handle MAVLink signals to provide more flexibility.
Wired HITL Testing:
The least complicated hardware system involves connecting the FC directly to the ground station computer using a USB cable. The usefulness of wired HITL testing is that all MAVLink communication types that would exist during flight are available over a wire. Thus, all of the ROS communications code can be refined and adjusted over a wire with very minimal setup. A key change to note involves the connection between QGC and MAVROS. MAVROS uses a uniform resource locator (URL) to find MAVLink signal sources on a given computer system. For example, a certain URL, “udp://:14540@localhost:14557” was used for SITL testing to receive messages from the local PX4. Wired HITL testing uses the URL “udp://:14550@localhost:14557” to receive forwarded signals from QGC. Wired HITL testing can be a first step in setting up a new drone prototype, or a useful step in troubleshooting hardware. The figure above shows that the Pixhawk and ground station laptops will communicate over MAVLink using a USB port and specific serial URL.
Results and Discussion:
Wired HITL testing has enabled significant advancement of current software and software plans. Namely, troubleshooting communications was key in refining the ways DEL sets correct settings with MAVROS. In the future, additional sensors could be tested using a wired connection. Additionally, the Jetson Nano will eventually use a wired connection to communicate with the Pixhawk, so lessons learned can be carried forward.
Telemetry Radio Based:
The base flyable design for real-time data acquisition relies on SiK telemetry radios. As shown in the figure above, there exists a radio on the drone and a ground station computer. The drone radio connects to the FC via the telem1 port using MAVLink to communicate. From the drone to the ground station, the radios communicate using a radio frequency protocol. The ground radio connects to the ground station laptop via a USB port. The ground radio and laptop again communicate via MAVLink where MAVROS is the tool used to connect the ROS workspace with the drone data. Although not yet tested, MAVROS may need to be initialized using a new URL depending on how QGC performs with a telemetry radio instead of a wired FC connection. Telemetry radio-based operations focus on minimal data from the IMU sensor.
Jetson Computer Based:
A second flyable design to be implemented on drone prototype 2 focuses on the Jetson Nano for additional onboard processing and communication over WiFi. Using a flight computer allows for advanced sensors to be used like a depth camera, optical flow sensor, and LiDAR. Some computational tasks such as data processing and perhaps estimation with the EKF will move to the onboard computer. The computer also offers the capability for highly autonomous missions involving visual-inertial odometry (VIO) informed motion planning and swarm operations. The Jetson will communicate with the Pixhawk across the telem2 connection again using MAVLink. Data will be sent to the pilot ground station laptop via WiFi. The received data will be used for data visualization, but most processing will occur on the drone. The Jetson-based framework is diagramed in the figure above.