This mixed reality application was developed for VWX Agency, with La Roche-Posay as the end client, to communicate information related to atopic dermatitis through an immersive, controller-free experience.
I was responsible for the complete development of the application, including experience design, mixed reality UI/UX, hand-tracking implementation, animation, interaction programming, scene logic, video integration, optimization, testing, and final deployment for Meta Quest 3.
The main creative and technical challenge was to transform the user’s own hands into an interactive medical information interface. Using native hand tracking, users could navigate a floating spatial menu through finger gestures without relying on physical controllers.
When users raised and observed their palms, futuristic interface elements appeared directly over the hands. Some UI components were anchored to the palm, while others were attached to individual fingers, allowing the visual elements to follow the movement of each anatomical tracking point in real time.
This created the impression of a dynamic digital scan rather than a conventional floating menu.
Once the hand reached the required position, an automatic sequence was triggered. Animated graphics appeared around the palm and were followed by floating information panels presenting content related to atopic dermatitis, scarring, and the broader impact of dermatological conditions on patients’ quality of life.
The experience included clinical and social information addressing subjects such as confidence, anxiety, sleep disorders, and attention difficulties in both adults and children.
A secondary section of the application allowed users to access video-based patient stories and related informational content directly from the mixed reality interface. This required the integration of spatial menus, video playback, hand-based interaction, scene transitions, automatic triggers, and floating content panels within the same standalone application.
Particular attention was given to the stability and readability of the interface, since the visual elements had to remain correctly positioned while the hands and fingers were constantly moving.
The final application was presented and used during a live Derm Lab event, where attendees and specialists experienced the hand-tracked mixed reality system on Meta Quest 3.
Role: Solo XR Developer & Technical Artist
Agency: VWX
End Client: La Roche-Posay
Project Type: Mixed Reality Medical Experience
Subject: Atopic Dermatitis
Target Platform: Meta Quest 3
Development Areas: Experience design, mixed reality UI/UX, hand tracking, palm and finger UI anchoring, gesture-based interaction, animation, spatial interface design, automatic trigger systems, scene logic, video integration, testing, performance optimization, and standalone deployment
Key Features: Controller-free gesture interaction, floating spatial menu, palm-anchored UI, individually tracked finger elements, automatic hand-scanning sequence, clinical information panels, integrated patient-story video, passthrough mixed reality, and live event deployment
Deliverables: Complete source project and functional Meta Quest build
A controller-free mixed reality experience developed for VWX Agency, with La Roche-Posay as the end client, to communicate information about atopic dermatitis.
I designed and developed a custom interface anchored directly to the user’s palms and fingers.