Issue link: https://resources.mouser.com/i/1442826
Another software approach is to bypass implementing models with generic frameworks and taking a bespoke approach to developing models optimized for a specific hardware target. Optimizations beyond pruning and quantization (e.g., relying on the similarity among adjacent frames in a video stream to quickly find previously detected objects) can extract further performance. Companies like Pilot.AI and Invision.AI have ported their object-detection models to Layerscape, achieving movie-quality frame rates. A system-level approach can also rationalize limited hardware resources available for retrofitting AI and streamline upgrading systems already in the field. For example, a first level of AI classification can be added to an IP camera, smart door lock, or other devices, taking advantage of any available processing headroom and memory. This level can extract features or do other preliminary classification, cascading the results downstream to the associated Layerscape-powered camera headend or home-automation hub to complete the analysis process. If a deployment has insufficient resources, it need not be ripped out and replaced but instead supplemented with an adjunct Layerscape system or module for the AI functions. Figure 2 shows this approach in the context of a roadside unit (RSU). These are systems deployed throughout a smart city to help implement an intelligent transportation system (ITS). They monitor roads and intersections with various sensors and communicate with vehicles and adjacent RSUs. In the Figure 2 example, the vehicles, cameras, and radars pre-classify the data they capture, communicating their findings to the RSU. The RSU tracks and plots vehicles and pedestrians, analyzes their motion and queuing, controls traffic signals, and communicates with other systems—a big load that would be even bigger if the first level of processing hadn't gotten done near the various sensors. Regardless of the software approach taken, it's essential to keep it up to date. This is a crucial function of NXP's EdgeScale suite, which takes advantage of Layerscape's platform trust hardware to securely update AI models and firmware from a cloud-based console. The Layerscape recipe of combining processing and I/O is well suited to supporting AI. We find the Arm Cortex-A72 CPU—the workhorse used in many Layerscape processors with one to 16 cores—performs about as well as a single thread of a server- grade processor or a single core of a PC-grade processor. We've seen this result on benchmarks in the SPEC suite, in networking tasks, and video compression. The Arm Cortex-A53 CPU—the lower-cost stablemate of the Cortex-A72—works well for applications when paired with optimized software and in less-demanding situations. For example, a video surveillance system operating at only 8fps can compress this video in the H.264 format using only a single Cortex-A53 CPU, with cycles remaining for other tasks. An adjacent Cortex-A53 CPU running commercial AI software can identify bodies at this frame rate or faster. Layerscape's abundant USB, Ethernet, and PCI ports can connect to cameras, radar modules, and other sensors generating input to be analyzed. These I/O ports are also essential for LAN and WAN connections. It's hard to imagine a system using AI that doesn't also communicate. Competing processors may have useful multimedia engines but cannot match Layerscape's interfacing options and networking performance. In conclusion, developers need not rely on an expensive coprocessor add-on or think their only option is a competing chip with hardware acceleration but without Layerscape's networking and I/O or cost-efficiency. Nor must one wait to implement AI. Get started today! ■ Figure 1: Mapping an AI-Enabled Application to Layerscape device. 25 AI Figure 2: Cascaded AI Can Play a Role in the Smart City.