94 mission-specific AI model that can be embedded on an EO/IR gimbal’s SoC. It is worth noting here that the widespread training of AI algorithms using 640 x 512 resolution LWIR image datasets is a major reason why there isn’t stronger demand for higher resolution thermal cameras. As autonomous recognition, classification and gimbal tracking of different objects can now be accomplished with a success rate close to 99% using streams of 640 x 512 thermal photography and video, there is no significant pay-off to be derived from higher resolutions in terms of vehicle intelligence or capabilities; the few remaining percentage points of accuracy can be gained through algorithmic optimisations. Right now, the roadmap for thermal AI lies not so much in further development of the AI algorithms but in identifying new and valuable ways of using them, as well as updates of the software to keep them compatible with the latest SoCs, particularly for multi-role uncrewed systems that need multiple simultaneous analytic capabilities per mission. Edge computing Companies such as Qualcomm and Nvidia stand out among thermal imaging and other sensor suppliers for the SWaPoptimised SoCs they produce. These generally combine a GPU with a CPU, memory, comms, power management and other systems in a tightly integrated form suited for packaging inside a gimbal without it presenting a glaring point of mechanical or thermal vulnerability. High computing power is critical for real-time processing and analysis of thermal monitoring data, given that modern IR scanners and video cameras are capable of frame rates of 30-60 fps or higher. Popular platforms for thermal image processing include Qualcomm’s RB5, based on its QRB5165 processor, which is capable of up to 15 trillion floating point operations per second (Tflops). Among other components it includes eight of the company’s Kryo 585 CPUs, each being a 64-bit system with up to 2.84 GHz clock speed, as well as its Adreno 650 GPU, which is built for up to 343 Gigaflops of performance in 64-bit operations, or up to 1372 Gflops in 32-bit operations. In the future, Qualcomm platforms are expected to remain prevalent in turnkey thermal imaging payloads, particularly with newer generations of the RB5 being released at the time of writing. However, thermal systems developers plan to integrate Nvidia platforms as it issues more powerful SoCs and because some engineers prefer working within the Nvidia ecosystem. For reference, the next generation of the Qualcomm RB5 is expected to perform nearly 50 Tflops, such is the necessary degree of processing power needed for running the kinds of computationally intensive perception software models that are now rapidly becoming a mainstay of thermal imaging products for uncrewed systems. System integration As coupling a SWaP-optimised thermal camera with a powerful edge computing system becomes increasingly important for maximising the capabilities and actionable information that can be derived from a thermal core, it is increasingly common for thermal core manufacturers to engineer their products with an array of interfaces, to make it easier for system integrators to connect and implement edge compute components seamlessly. For example, the MIPI-CSI2 camera interface is gaining widespread popularity as a protocol for connecting image sensors to processors. It costs less than other interfaces, is relatively easy to program, and uses minimal CPU resources or power to implement. This latter property also means MIPI ports produce little heat, so they don’t overburden the thermal core’s cooling system. December/January 2024 | Uncrewed Systems Technology Focus | Thermal imaging Using trained algorithms to intelligently detect and classify objects in IR is becoming a core expectation among uncrewed vehicle integrators and operators (Courtesy of Teledyne FLIR) IR cores built with numerous MIPI, Ethernet and serial ports make integration with processors, storage drives and other sensors far easier (Courtesy of Exosens)
RkJQdWJsaXNoZXIy MjI2Mzk4