The Plateauing of Display Technology

Video displays are everywhere, from cell phones to movie screens, and made of many different materials. Increasing both the amount of pixels on the screen and the rate at which the pixels are refreshed has been a continual process in the display industry, but there is a limit to how sharp and how quickly displayed an image can be before the user will no longer notice improvement.

Table 1: Various display resolutions and their applications. 4K and 8K may vary somewhat in horizontal and vertical resolution due to multiple standards.

Larger, sharper displays require fitting more pixels on a screen at once. For the past decade, high-definition screen resolution has been 1080, while movies are projected onto giant theater screens at 2K resolution. However, 4K displays have started to make their way into homes as televisions or computer monitors, and the forefront of resolution technology is 8K.

Figure 1: A comparison of screen resolutions for 8K UHD, 6K UHD, 5K UHD, 4K UHD, FHD and SD. By Libron (Own work) [CC0], via Wikimedia Commons.

Content: Keeping Pace?

Not a lot of content is available for the highest resolutions. Content is usually created in ultra-high-definition (UHD) in order to make the visual effects and editing easier (they can zoom, crop, and digitally correct errors and still make it look good when bouncing down to high-definition 2K or 1080p). Until recently, 8K has been rare; used only in instances where heavy post-production is required. It’s also used to digitize high-quality film prints, probably most famously with the 2012 scan of Lawrence of Arabia1.

In June 2016, Guardians of the Galaxy Vol. 2 wrapped up production in 8K digital, the first feature created in the emerging format, but a film in 8K is unlikely to ship to theaters as such.² Eventually content will arrive for 4K, and 8K content will follow, but by then consumers may realize that increasing to 8K improves quality only for very large TVs; ones they may not realistically want in their living room. Note that on the chart below³ one would have to be watching from a distance of 5 feet with a screen of ~90 inches to reach the limits of 4K, let alone 8K. For many, living room set ups and screen size preferences ensure that 1080p is good enough. Nevertheless, 8K TVs may become commonplace soon enough, as technological progress does not halt easily; however the march to higher resolution might end with 8K for the typical home.

Figure 2: This chart shows that with an 84-inch screen, 4k resolution will only be completely apparent until a distance of about 5.5 feet or less from the screen. With a 55-inch screen, it’s 3.5 feet or closer. Chart printed with permission by Carlton Bale from "Does Resolution Matter?"

Speeding up frame rates is another area where manufacturers push boundaries and standards. All LCD displays have a certain refresh rate, measured in hertz (Hz), which determines the number of times per second the screen can refresh its image. Frame rate, on the other hand, is the number of times an image is sent to the screen to be displayed. Film has largely maintained a standard of 24 frames per second (fps), but directors have experimented with higher. Peter Jackson famously shot his Hobbit films at 48 fps, which caused some critics to accuse the film of possessing “the soap-opera effect,” where there’s less motion blur than movie-goers are used to seeing on film.4 Modern LCD screens reduce motion blur using motion interpolation, a smoothing technique that adds an extra frame in between frames by “guessing” what an intermediate frame would look like and then inserting it, thus increasing the frame rate artificially.5 This feature has had similar mixed reactions. Walk into any consumer electronics store to witness the “uncanny valley-esque” look of interpolated frame rates.

Gamers Get Real

One industry reaping great benefit from high frame rates is video gaming. Players are responding to visuals on screen, so high frame rates feel smoother, but require much more computing power to render new images rapidly, which is why console games have a capped frame rate while PC games allow adjustment based on gamers’ hardware. For consoles, 30 fps is the standard, while 60 fps is considered optimal. Frame rates of 90 fps are used by Oculus Rift and HTC Vive, the current leaders in virtual reality hardware.6 When watching a movie, one is a passive observer to the action; but when playing a video game, one is participating to a certain degree. Furthermore, when a person has a virtual world strapped to their face, their brain, thinking that it is looking at the real-world, expects a refresh rate that responds to movement in real-time. As such, inconsistencies with the frame rate have been blamed for causing motion sickness.7 This is only a theory, however, since “virtual reality sickness” and how frame rate correlates are unresolved and remain a highly-studied topic.

Faster processing means potentially faster render times and frame rates, but there’s a limit to how fast humans can percieve frame rates. One could compare perception of frame rates with game performance; the faster the figures on the screen are moving, the faster the image needs changing. In 2006, a study by Claypool, Claypool, and Damaa tested players’ performances in a first-person shooter at different frame rates. The study seems to conclude that the improvement in a player’s performance does not increase much between 30 and 60 fps.8 Of course, this study was 10 years ago, so modern games with more complicated graphics may affect the results of a study today. Many gamers would disagree with the spirit of the study, in that frame rate is not only about performace but about “feel,” as well. One can achieve a certain clarity that almost makes the game feel hyper-realistic and three-dimensional at times, which pairs well with high resolution graphics. But although 60 fps seems like a good baseline for player performance and feel, it ultimately comes down to each individual’s eyes.9 While there is a limit to how fast human eyes can perceive a frame rate, each person is different. If we assume that most players start seeing diminishing returns after 60 fps, which seems to be the case with many casual gamers, then display manufacturers are relatively safe when it comes to refresh speeds.

Figure 3:

 The effects of frame rate and frame resolution on user performance. Score vs. Framerate (512 x 384 resolution), Source: The Effects of Frame Rate and Resolution on Users Playing First Person Shooter Games, Mark Claypool, Kajal Claypool and Feissal Damaa. 2006, http://web.cs.wpi.edu/~claypool/papers/fr-rez/paper.pdf

On the forefront of display technology is 165 Hz and faster-refreshing 1080p monitors.10 Monitors at 120 Hz used extrapolation to claim faster refresh capability, but cutting edge displays can finally support the highest refresh rates. Faster monitors are always being developed, but today’s selection of display technology can support incredibly smooth gameplay.

The next challenge is to make monitors that refresh at these fast speeds but also at higher resolution. 4K screens require much more processing power and faster graphics cards to refresh a greater number of pixels while synchronizing the incoming frame rate with the monitor refresh rate. This a matter of communications bandwidth. Since incredible amounts of data need to be transferred to these screens at high speeds, for bigger screens, innovative software and interface hardware are required to accomplish equally fast refresh rates. DisplayPort is one audio/visual interface used for high resolution, high refresh rate transmissions, currently supporting 8K at 30 Hz at its best.11

Where Will Display Tech Take Us Next?

Modern screens are already so sharp as to seem lifelike, and refresh rates are fast enough on reasonably-sized screens for quick game rendering. Content may not yet be available to take full advantage of the best displays, but it will come soon, and in great numbers. So, super high refresh rates on ultra high resolution screens aside, where does the display industry go next? In spite of this “sensory wall” the display industry is approaching, innovation will always soldier on. Maybe no one will care when 16K comes along, but new paradigms will eventually revolutionalize display technology. Curved screens and backlighting techniques can provide variation in LCD, OLED, and AMOLED markets. But TV networks, still catching their breath after the upgrade to HD, are starting to roll out infrastructure to broadcast 4K content while internet streaming services have slowly been adding 4K content as people buy compatible TVs.

Maybe vector-based graphics will leave pixelated displays in the past. With the rise of smart TVs, a consumer might be watching a movie and have the iMDB page (an online resource for broadcast and celebrity content) automatically show up on their tablet; an example of interactivity between entertainment devices in the home. In a few years, don’t expect to be able to distinguish the leaps and bounds technology has delivered within the pixels and smoothness of displays, but do expect something truly unexpected and novel to come from the industry.