Scene Engine: Automated Data-to-Scene Bridge
A longitudinal human digital twin simulation pipeline. Generate interactive, high-fidelity 3D vignettes from raw transcripts and audio data using generative physical AI and WebGPU.
Core Features
A fully headless pipeline for procedural 3D vignette generation.
gRPC-driven vocal performances using the NVIDIA Audio2Face-3D microservice for highly accurate, low-latency lip sync and facial mesh deformation.
Mathematical blending equations mix explicit transcript emotional markers with automatically detected audio emotions, smoothed using a VectorizedOneEuroFilter to eliminate jitter.
A headless pipeline using OpenUSD APIs and usd2gltf to seamlessly convert complex USD stages into web-optimized glTF 2.0 and GLB formats.
A Next-gen TypeScript and React frontend utilizing WebGPU to load massive GLB models and drive an unlimited number of facial blendshapes at a consistent 60+ FPS.
Tech Stack
Combining cutting-edge graphics algorithms with scalable web infrastructure.
OpenUSD
Python-based data translation and asset conversion pipeline.
Python 3.10+
NumPy and SciPy powering emotion blending and temporal filtering.
WebGPU
React & Three.js/Babylon.js delivering next-gen browser graphics.
NVIDIA DGX
Audio2Face-3D inference and Multi-GPU path tracing.