In recent years, computer vision (CV) has made remarkable strides, particularly in fields like visual odometry and visual SLAM (Simultaneous Localization and Mapping). Thanks to advancements in GPU technology, these capabilities have been extended to small drones navigating complex environments. While these achievements are undeniably impressive, it’s important to recognize that the conventional CV stack, despite its successes, has significant limitations—especially when faced with adverse conditions.
The Conventional CV Stack: A Quick Overview
At the core of most contemporary computer vision systems is a process that can be summarized as follows: One or more high-resolution cameras capture a video sequence of the environment. These images are then processed using feature detectors, which identify easily tracked texture elements such as corners and edges. These features are parameterized with statistics about their immediate regions, allowing them to be matched between frames—either across views from multiple cameras or sequential frames from a single camera.
From there, projective geometry is used to construct 3D models of the environment, including the position of the robot or device. Outlier elimination and optimization techniques are applied to refine these models, resulting in a final estimate. The mathematics behind these algorithms has been around for decades, but only recently has it become viable for real-time applications, thanks to contemporary GPUs. The recent successes in this field are as much a triumph of raw computing power as they are of algorithmic ingenuity.
However, as impressive as this approach is, it may not be the best solution for every application—particularly when the environment is less than ideal.
Weaknesses in Adverse Conditions
Despite its sophistication, the conventional CV stack has several inherent weaknesses, especially when operating in adverse conditions. These weaknesses stem largely from its reliance on feature detectors.
- Narrow
Obstacles: One of the most significant challenges arises when the system encounters narrow obstacles, such as cables or bare branches. Feature detectors excel at identifying two-dimensional textures, but they struggle with one-dimensional objects. As a result, these narrow obstacles often go undetected, posing a significant risk in environments where precision is critical. - Low
Light Conditions: Another major issue occurs in low-light conditions. An image sensor must make a trade-off between maintaining a high frame rate, which can result in noisy pixels, and using a longer integration time, which can lead to image smear. Both of these scenarios severely impact the effectiveness of feature detectors, rendering them nearly useless in such conditions. - Environmental
Factors: Dirt, condensation, and environmental particulates like smoke or dust can also wreak havoc on computer vision systems. These factors can distort the images captured by the cameras, leading to errors in feature detection and, consequently, in the overall CV stack.
Given these limitations, it’s not surprising that many failures in mobile robot navigation—especially in challenging environments—can be traced back to one or more of these conditions.
Implications for Smaller Platforms
The challenges don’t end with adverse conditions. The scalability of the conventional CV stack is another significant concern, particularly for smaller platforms such as nano drones or insect-scale robots. The heavy computational resources required to execute these algorithms in real-time make them impractical for these smaller, resource-constrained devices. As the demand for smaller, more agile robots increases, the need for alternative approaches to computer vision becomes even more pressing.
Exploring Alternatives
So, what are the alternatives if we wish to use a passive, vision-based approach without relying on a tera-operation GPU? And how can we deliver vision-based navigation capabilities to insect-scale flying robots or fast-moving ground vehicles?
In upcoming posts, I plan to explore some ideas that draw inspiration from the world of insect vision—approaches that could provide the key to overcoming the limitations of conventional CV systems in both adverse conditions and on smaller platforms.
Join the Conversation
I’d love to hear from you. Is your experience with the conventional CV stack similar to mine, or have you encountered different challenges? What types of topics would you like to see discussed in future posts? Please share your thoughts in the comments below—let’s start a conversation about the future of computer vision.