How Virtual Try-On Works

When a shopper clicks "Try On," the browser requests camera permission via getUserMedia(). Once granted, the video stream feeds into a face-tracking model that runs roughly 30 times per second, detecting 468 facial landmarks and head rotation (pitch, yaw, roll) per frame.

Those landmarks drive the product overlay: eye-corner and pupil positions determine frame or shade placement and scale, and head rotation is applied as a transform so the product tilts naturally as the shopper moves. The composited result — live video plus product overlay — renders to a canvas or via CSS transform, and (optionally) session events are sent to your analytics dashboard.

Tryonixs uses Google's open-source MediaPipe Face Landmarker model. It's a machine-learning model trained to output normalized (x, y, z) coordinates for 468 points on the face, plus a transformation matrix used to derive head rotation. The model runs via WebAssembly, with GPU acceleration used when the browser supports it and a CPU fallback otherwise (important for consistent behavior on iOS Safari).

For categories that need pixel-level skin segmentation — like foundation shade matching — a second model (BiSeNet, open-source, MIT-licensed) classifies each pixel into one of 19 facial regions, so shade color applies only to skin and not eyes, lips, or hair.

Every model above runs entirely in the shopper's browser. No video frame or image is uploaded to Tryonixs servers for face detection or rendering — only anonymized analytics events (like "session started" or "product X tried on") are sent, and only if you've enabled analytics tracking. This is an architectural choice, not just a policy: the models are open-source and self-contained, so there's no server-side inference step to route data through in the first place.

Detection targets roughly 30 FPS to balance tracking smoothness against CPU/battery load, with jitter smoothing (a One Euro Filter) applied to position, scale, and rotation so the overlay doesn't visibly shake between frames. Models are lazy-loaded — only the models a given category needs are downloaded, and only when a shopper actually opens try-on, not on page load — and cached in memory for the rest of the session, then disposed of on close to free memory.

1. Sign up and get your public API key. 2. Load the SDK script (or add Shopify/WooCommerce snippets). 3. Add try-on buttons with product IDs and transparent PNG frame/shade images. 4. Allowlist your domain in the dashboard. 5. Check your dashboard analytics after launch to see which products drive the most try-on engagement.

Run a live demo with your own product assets, on mobile Safari specifically (it's the strictest browser environment). Ask whether face data is processed client-side or uploaded to a server. Check whether the pricing model charges per AI inference/session or is flat-rate. Confirm platform support for your actual storefront (Shopify, WooCommerce, or custom). And check whether the vendor's core AI models are open-source and auditable, or a closed-source black box.