AI in Image Processing
In the signal-processing pipeline, sitting behind the
camera optics and the sensor, commercialization of
machine learning (leveraging deep neural networks) is
enabling a revolution in the way images are constructed
and information is subsequently extracted from them.
One example can be seen in the use of AI to significantly
improve low-light performance, enabling high-quality
images to be taken in near-dark conditions.
Raw data captured under low-light conditions is known
to challenge traditional signal-processing pipelines.
Electronically raising sensor sensitivity (ISO number)
can add noticeable noise to images resulting in poor
image quality, and applying de-noising to the image
has only limited effectiveness. Other techniques to
improve image quality include extending the exposure
time, although this is often impractical in industrial
applications or cameras on board vehicles.
More recently, an ingenious technique has been
developed that leverages machine learning to greatly
reduce the detectable noise in images constructed from
raw low-light data. A deep neural network is trained
using datasets that contain raw short-exposure low-light
images and corresponding long-exposure reference
images. When the network is fully trained, it is capable
of creating high-quality images by working directly on
raw short-exposure data. This technique is coming into
the market in top-of-the-range smartphones – enabling
delivery of better-looking photographs. It is also
applicable to capturing better images for industrial and
security applications, such as production line inspection
or surveillance systems.
Conclusion
Numerous technical improvements are being developed
throughout modern image processing systems – from
the camera lens at the front end of the system to the
image sensing and image processing apparatus behind
it. Together, these are expected to drive expansion in the
breadth of applications that can be addressed, as well as
increasing system performance benchmarks.
23
INTEL 2020
Global shuttering improves image sharpness when
photographing fast-moving objects or when the camera
is mounted on a moving vehicle. First featured in high-
end still cameras, the technique is now in demand to
enhance the performance of industrial and automotive
vision systems. In global shuttering, the charge value
of all pixels is stored simultaneously to a small in-pixel
memory before being read sequentially into the frame
buffer line by line as before. This results in a clear image,
free of rolling-shutter distortion.
Several challenges have been overcome to create
global-shutter image sensors that achieve elevated
SNR and dynamic range without increasing pixel size to
compensate for the presence of in-pixel memory, which
effectively reduces the pixel area that can be used for
photon absorption. An example of such image sensors
is the 1Mpixel, 1/4-inch format ON Semiconductor
ARO144.
Global shutter pixels feature high quantum efficiency to
ensure fast charging while at the same time remaining
insensitive to charging effects not related to the image,
such as crosstalk resulting from electron diffusion. In
addition, optical shielding is applied close to the sensor
to exclude stray illumination from the pixel surface.
Figure 2: The ARO144 image sensor from ON Semiconductor.
Learn More
ADLINK TECHNOLOGY
PCIE-GIEIMX AI FRAME
GRABBER
