Osegi
Hardware·2 min read

Glasses-free 3D display turns 2D video into 3D and tracks your eyes

A Japanese restaurant is testing a glasses-free 3D sign that converts ordinary video to 3D live and follows your eyes. It only works well for one viewer at a time.

A customer stands in front of a glasses-free 3D digital signage display showing osechi New Year food in a Japanese restaurant.
Image: ITmedia NEWS

Stand in front of the screen and a plate of osechi, the traditional Japanese New Year food, seems to float out toward you. No 3D glasses. Move left or right and the angle of the dish changes with you. The source video is ordinary 2D, and AI is converting it to 3D as it plays.

That is the glasses-free 3D display that SoftBank, SB Creative and the Japanese restaurant chain Tonden showed to the press on November 12 at a Tonden branch in Saitama. It is a trial of digital signage, the electronic menu boards and ads in shops. SoftBank says it is the first time it has tested this 3D display in Japan.

How does the glasses-free 3D display work?

Three technologies are involved. First, a lenticular lens: rows of thin, half-cylinder lenses that bend light so each eye gets a different image. Second, a depth estimation AI model that guesses how far away things in a video are and generates left-eye and right-eye images in real time. In a shot of cherry blossoms with Mount Fuji behind, it works out that the blossoms are in front and pushes them forward.

Third, AI eye tracking. A camera on the display finds the viewer's eyes and adjusts the picture to match, which keeps the 3D image sharp even if you are a little off-center.

SoftBank's Yuji Kumon said the main point is that video no longer has to be made specially for 3D displays. Existing 2D footage works as it is.

The catch: one viewer per screen

The eye tracking follows one person per display. With several people watching, not everyone sees 3D. A reporter watching a colleague from the side saw the image warp like a mirage at some angles. In a busy restaurant, how many customers can actually see the effect is an open question.

What happens next

The three companies want to know whether 3D grabs more attention than flat video and raises interest in the food. Visitors answer three yes-or-no questions with stickers on a board. At 3 p.m. on November 12, all stickers for "did the osechi look 3D" and "would you look at a 3D video in a shop" were positive. On "did your interest in osechi rise", most said yes, but a few said it had not changed.

SB Creative plans to build the conversion into BeyondCAST, its cloud service for managing video across many displays. Companies could then upload 2D clips and play them in 3D. SoftBank and SB Creative will weigh commercial launch after the final results.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:softbank3d displaydigital signageaisb creativetonden

More from this day

See all →

More in: Hardware

See all →