Issue link: https://resources.mouser.com/i/1437726
27 INTEL 2021 Neural networks trained using modern deep-learning techniques take image analysis to the next level, being able to recognize objects or people with a high degree of accuracy, for example. Using machine- learning approaches, neural networks can be trained on large data sets, enabling highly accurate decision making. This approach provides greater precision than legacy object-recognition methods, as well as removing the need for painstaking hand-coding of explicit rules. The capabilities with sophisticated software are open-ended. The FLIR ® Firefly ® machine vision camera incorporates an on-camera deep neural- network accelerator based on the Intel ® Movidius Myriad 2 Vision Processing Unit (VPU). The Firefly machine vision camera enables sophisticated machine-vision applications while remaining more cost-effective, simpler to integrate, and more reliable than discrete systems. FLIR ® Firefly ® : Machine Vision + Deep Learning FLIR engineers accelerated the Firefly's development cycle using Intel ® Movidius technology for both prototype development and large-scale commercial production (Figure 1). Rapid prototyping based on the Intel ® Movidius Neural Compute Stick (NCS) and Neural Compute SDK streamlined the early development of machine learning in the camera. The production version of the Firefly uses the tiny, standalone Intel Movidius Myriad 2 VPU to do two jobs: image signal processing and open platform inference. Once satisfied with neural-network performance in the prototyping phase, FLIR engineers took advantage of the VPU's onboard image signal processor and CPU. Utilizing onboard imaging, convolutional neural network (CNN) and programmable compute capabilities of the chip allowed FLIR to minimize size, weight, and power consumption aggressively. This approach provides a single hardware and software target that simplifies prototyping, while also enabling the production version of the full camera to be about an inch square (Figure 2). Placing deep neural-network acceleration directly on the camera enables inference to be performed at the network edge, rather than having to transmit the raw video stream elsewhere for processing. This approach introduces a number of advantages that improve the overall solution, including the following: • Real-time operation. Processing in place eliminates the latency associated with transporting data for off-camera computation, allowing detection and subsequent responses to be made in real time. • Efficiency. Eliminating the need to send raw video data over the network reduces costs related to bandwidth, storage, and power consumption. • Security. On-camera inference enables a simplified, self-contained architecture that reduces the attack surface, and the relatively small amount of data passed over the wire can be encrypted with minimal impact. The FLIR Firefly camera marries machine vision and deep learning by combining excellent image quality with Sony's Pregius, sensors—GeniCarn compliance for ease of use—and an Intel Movidius Myriad 2 VPU for performing deep neural-network inference. Firefly's ultra-compact footprint and low power consumption make it ideal for implementations with space and power constraints, such as handheld and embedded systems. The camera is also equipped with a use port for host connectivity as well as four bi-directional general- purpose input/output (GPIO) lines for connection to other systems. Figure 1: The Pure Firefly camera was tested (left) with the Movidius Neural Compute Stick and prototyped (right) with the Intel, Movidius Myriad 2 vision processing unit. Figure 2: Use of the onboard image signal processor and CPU enable the Firefly* camera—including its onboard processing hardware—to be very small in size.