Data at the Edge: Why Computing Is Coming Home from the Cloud

Source: Ar_TH/stock.adobe.com

The history of computing can be told through many lenses. Take, for instance, the cat-and-mouse game between hardware and software. New hardware is introduced, and it takes a while for software to take full advantage of its computing horsepower. Then, the software becomes resource-hungry and must wait for the hardware to catch up. Another example is the back-and-forth between local and remote computational capability. While the terminology may change—cloud versus remote and local versus the edge—the general concepts remain relevant. Essentially, edge computing is about maximizing data processing as physically close to the source as possible.

Edge computing is an iteration of computational evolution where data are not just mouse clicks and keystrokes, but rather sensor telemetry, camera images, and video streams, among many other sources. So, in an era of nearly ubiquitous wireless internet access in many areas of the globe, why not just package all the data and send it to the cloud? After all, we have been doing that since 2006, after the emergence of reliable cloud infrastructure, such as Amazon Web Services (AWS), Google Coud, and Microsoft Azure. Answering this question, however, is not so simple.

Local to Remote to Local Again

To answer that question, let’s look at the rise of the internet itself. In the 1990s, there was a latency issue as websites started springing up on the World Wide Web. While a website hosted on a server in the United States might be accessible from Japan, downloading the site to the browser could take a relatively long time. This delay, known as latency, could be exacerbated if your network is very active with other users. Then, if the site became popular, the additional traffic would only make things worse. To reduce this latency and improve the overall experience, locally available content distribution networks (CDNs) were devised to cache copies of websites (or other content like movies or documents) closer to the end users accessing them.

With that foundation, let’s examine what makes edge computing different from what has come before. Around the time smartphones and associated peripherals—like wearables, Internet of Things (IoT) cameras, smart lightbulbs—became common, a fundamental shift occurred in the source and directional flow of data. Instead of accessing static content from a remote server to view locally in a web browser, these “smart devices” started generating lots of data from their multiple onboard sensors.

The problem was that these devices were not actually so smart. Sensors may generate data, but data is not information or knowledge. Data must be processed for it to be useful, and, for smart devices, that processing occurs in the cloud, especially with the advent of computationally intensive artificial intelligence (AI) and machine learning (ML) algorithms. Suddenly, lots of devices with lots of data were pushing lots of bits up to the cloud.

Latency Returns as the Bottleneck

All this brings us back to that pesky little problem of latency, specifically with inferencing. In computing, inference is when a trained AI model is presented with new inputs to produce probabilistic outputs. It is the most end-user-focused aspect of AI/ML technologies and has historically been computationally intensive. Therefore, getting inferencing right is crucial. Initially, the only viable option was to send raw data to the cloud to leverage the computational horsepower of server-grade hardware and then send the outputs back to the local device. But embedded hardware has since become more powerful, and ML algorithms have developed greater efficiency. This convergence has created several noteworthy benefits that make edge computing a viable alternative to sending all that data to the cloud for processing.

• Reduced latency: Edge computing enables real-time or near-real-time responses by eliminating the need to send data to a remote cloud for processing.
• Bandwidth optimization: Processing data locally reduces the volume of data that needs to be transmitted to central servers, conserving network bandwidth.
• Improved reliability: Edge computing continues functioning even with intermittent or limited internet connectivity, making it ideal for remote or mission-critical applications.
• Security and privacy: Sensitive data can be processed locally rather than transmitted to a central location, reducing exposure to potential cyber threats.

New Technology Making Edge Computing a Reality

The emergence of new hardware and software over the past few years has enabled the increased use of edge computing, making it possible to run complex algorithms like neural networks on relatively inexpensive, battery-operated devices. The following are some of the current edge-native hardware:

• Google Edge Tensor Processing Unit (TPU): purpose-built chip for executing TensorFlow Lite models efficiently
• NVIDIA Jetson series: AI-accelerated embedded platforms for robotics, video analytics, and smart cities
• Intel^® Movidius™ Neural Compute Stick: plug-and-play USB accelerator for low-power deep learning
• Industrial edge gateways: rugged micro data centers acting as local processing hubs and interconnects to send data to the cloud and receive updates

Still, hardware is nothing without software. AI workloads at the edge require optimized software frameworks for low-power inferencing, with the following popular frameworks available:

• TensorFlow Lite: lightweight ML inference for edge devices
• ONNX Runtime: supports running AI models across various platforms
• NVIDIA TensorRT: optimized deep learning inference for Jetson devices

However, for crucial operations that can’t wait for the cloud, such as real-time processing for AI, video analytics, and robotics, processing at the edge doesn’t mean that data is never sent to the cloud. There are still good reasons for sending data to the cloud via edge gateways:

• Scalable storage and archiving: Store vast amounts of data long-term without hardware constraints.
• Global analytics and trend detection: Aggregate data across devices to uncover system-wide insights and patterns.
• Advanced model training: Use powerful cloud resources to retrain models with real-world data and deploy the revised ML models to the edge.
• Centralized management: Remotely monitor, update, and manage devices through cloud dashboards and automation tools.
• Disaster recovery and redundancy: Protect data from local failures through secure, replicated cloud backups.

Edge-oriented communication protocols and message brokers, including the following, ensure this communication is done efficiently:

• Message Queuing Telemetry Transport (MQTT): Efficient and low-overhead communication for IoT
• Constrained Application Protocol (CoAP): Lightweight protocol designed for constrained devices
• Advanced Message Queuing Protocol (AMQP): Used in banking and industrial applications for robust messaging
• ZeroMQ and NATS: High-speed messaging systems for real-time processing

To ensure security, performance, and flexibility at the edge, specialized operating systems and runtime environments for edge deployments have emerged. These environments are purpose-built to run reliably on resource-constrained devices, support containerized workloads, and enable secure software delivery and lifecycle management in the field. Some of the widely used platforms include:

• Ubuntu Core: A streamlined, container-ready version of Ubuntu designed specifically for IoT and embedded systems. Ubuntu Core uses snap packages to deliver software updates automatically, ensuring devices can be updated securely and rolled back if needed. Its read-only file system and strict confinement model make it ideal for mission-critical edge deployments requiring reliability and resilience.
• Yocto Project: Rather than a complete Linux distribution, the Yocto Project is a hardware architecture-agnostic set of tools that allows developers to build custom, lightweight Linux operating systems tailored precisely to embedded systems’ hardware and application needs. It is widely used in industrial, automotive, and consumer electronics where complete control over the OS footprint, dependencies, and security is required.
• Azure IoT Edge Runtime: Microsoft’s container-based edge computing platform enables deployment of cloud intelligence to edge devices. It supports containerized workloads, including AI models, Azure Functions, and custom logic—all managed via the Azure portal. Features like offline execution, device twins, and centralized monitoring make it a powerful runtime for enterprises with distributed assets across factories, logistics, and infrastructure networks.

Lastly, to manage the growing complexity of distributed edge computing systems, major cloud providers and specialized platforms now offer orchestrated frameworks that bridge the edge and cloud. These platforms support device provisioning, workload deployment, secure communication, and lifecycle management.

• AWS Greengrass: Amazon’s edge framework extends AWS services to edge devices, allowing developers to run Lambda functions, maintain local messaging and state, and perform machine learning inference using models trained in the cloud. It supports MQTT messaging, offline mode, and seamless synchronization with AWS IoT Core.
• Azure IoT Edge: A fully managed service that brings Azure’s capabilities—like stream analytics, ML, and custom logic—to edge devices via Docker containers. It supports edge module orchestration, code deployment over-the-air, and rich integration with Azure DevOps for continuous integration (CI) and continuous delivery (CD) pipelines in IoT scenarios.
• Google Cloud IoT Edge: Designed to extend Google Cloud’s AI and data services to the edge, this platform includes support for the Edge TPU, TensorFlow Lite, and secure device management. Though its development has slowed in favor of Anthos and Google Kubernetes Engine on-premises (GKE on-prem) distributions, it remains a viable solution for vision- and inference-heavy applications in Google-centric environments.
• Edge Impulse: Unlike the other three cloud platforms in this list, Edge Impulse focuses exclusively on TinyML, a type of ML specifically for microcontrollers and constrained hardware. The platform includes data collection, model training, on-device deployment, and performance optimization tools. TinyML is a favorite among developers building intelligent wearables, sensor networks, and battery-powered devices where every byte and milliamp counts.

Conclusion

Edge computing represents a pivotal shift in how we design, build, and deploy intelligent systems—and for engineers, it opens an exciting new frontier. With the ability to process data locally, in real time, and on resource-constrained devices, edge computing directly brings advanced capabilities like AI, automation, and analytics to the environments where data is generated. Beyond its technological significance, edge computing is also important from legal and business perspectives, with the rise of sovereignty and privacy laws, such as the General Data Protection Regulation (GDPR).

This evolution is driven by increasingly powerful microcontrollers, energy-efficient AI accelerators, and lightweight ML frameworks. Platforms including AWS Greengrass and Azure IoT Edge, along with tools like TensorFlow Lite and NVIDIA Jetson, are democratizing access to technologies once reserved for enterprise data centers. Today, engineers can prototype and deploy sophisticated, intelligent edge applications, from autonomous robots and smart environmental monitors to connected wearables and industrial automation systems.

Ultimately, edge computing allows engineers to break free from cloud dependency, enabling more responsive, resilient, and privacy-conscious designs. It empowers creators to build systems that think locally and act instantly—an approach that is not only technically powerful but also increasingly necessary in a connected, data-driven world. As the edge continues to evolve, it offers an unprecedented opportunity for engineers to shape the next generation of intelligent, decentralized technology.