SSCATeR: Sparse Scatter-Based Convolution Algorithm With Temporal Data Recycling for Real-Time 3-D Object Detection in LiDAR Point Clouds

Drone swarm operations are becoming increasingly more ubiquitous in diverse fields, including infrastructure maintenance. Inspections in and around structures such as bridges and harbor locks can lead the drone to operate in GNSS-restricted or GNSS-denied environments. Therefore, additional fallback systems are required to mitigate mid-air collision risks between swarm members.

The use of LiDAR-based object-detection artificial intelligence has proven successful in the automotive industry. Drones, however, experience much tighter size, weight, and power (SWaP) restrictions, along with the additional consideration of the vertical dimension. As such, there is a need for computationally lightweight approaches for real-time drone-to-drone sensing and detection.

To address this, this work adapts the aerial version of the popular automotive object detection network PointPillars, called Aerial-PointPillars, which reduces latency using a novel sparse scatter-based convolution algorithm with temporal data recycling (SSCATeR). The modifications focus on two sections of the network architecture, the Pillar Feature Net (PFN) and the backbone.

In the PFN, a point cloud frame is converted into a grid of pillars. During feature extraction, these pillars are flattened into a pseudo-image that represents the features of the detection area. Instead of discarding and replacing previous results, this method reduces the frame interval from 100 ms to 10 ms and introduces change maps to reuse previous unchanged results.

The change map records whether a pillar has gained or lost any points in the previous 10 ms. If there is a change, the site is processed as normal; if not, previous results can be reused. The change map and pseudo-image are then passed to the network backbone. Since the backbone comprises multiple layers, the change map is updated whenever a convolution modifies an output site; otherwise, the previous results are reused.

To validate this approach, the researchers tested the algorithm against other networks in three scenarios and on two different embedded boards. Sparse Aerial-PointPillars and PillarNet were used as comparison networks, due to their speed and accuracy. Of the three scenarios, training and test data were collected from two drone swarm missions: one in a natural environment and one in a built environment. The final scenario was an automotive context, using the publicly available PandaSet dataset. The embedded boards used were the Nvidia AGX Xavier and the more powerful AGX Orin.

Depending on the scenario and board used, an SSCATeR-based Aerial-PointPillars is between 1.73 and 3.88 times faster than the other networks, whilst producing identical results to Sparse Aerial-PointPillars and consistently operating in real-time (> 10 ms) on the AGX Orin. The approach is also 6.61 times faster than regular sparse convolution over a single layer.

This work also includes an analysis and evaluation of the impact of LiDAR crosstalk on the network, showing minimal changes in results, making SSCATeR a promising approach for the future of real-time embedded object detection in drone swarms and for its potential adaptation into other point cloud-based object detection networks, such as PillarNet.