BlanchTouch Turns Walls and Desks Into Touch Surfaces for AR

How it works

The headset camera captures the hand and fingertips as the user interacts with their surroundings. The AI model detects the colour change, a hand-tracking system works out where the finger sits on the surface, and the movement becomes a digital command for virtual buttons and menus. The method needs no markers or special sensors on the wall or desk, and no calibration of each surface beforehand.

The work targets common problems with AR and mixed reality controls. Typical input means waving hands in the air or using separate controllers, and holding arms up for long periods is tiring. Mid-air gestures also lack the physical feedback of touching a real surface. With BlanchTouch, users can rest their fingers on a desk or wall while working with a virtual interface.

Test results

The researchers tested the system on a range of surfaces and materials and found it detected finger contact with consistent accuracy. Participants were able to complete interaction tasks on ordinary surfaces. According to the paper, presented at the IEEE Conference on Virtual Reality and 3D User Interfaces in 2026, the average error in locating a touch was 5.33 millimetres, with a recall rate of 97.88%.

The technique could be useful in offices, classrooms and workplaces, where nearby surfaces could operate virtual keyboards and menus and reduce reliance on conventional controllers. It does not mean any ordinary screen will become a touchscreen after installing an app: for now, it requires an AR or mixed reality headset with hand-tracking cameras.