GESTO 2000: the camera toy that never phones home

A rebuild of gestomeme.com — the Windows 98 desktop where you teach a webcam your own hand gestures, mint them into reaction cards and duel a friend. The premise is kept and the thing that was broken is gone: no model, no WASM and no font is fetched from a CDN. Every number below was measured off the shipped index.html and its 21 MB of assets, not planned.

App shell

140,498 bytes

Source

2,945 lines

Self-hosted assets

21 MB

Requests to a CDN

Languages

3 繁 / 简 / EN

Selftest

18/18

Why the original stalls

gestomeme.com loads its MediaPipe bundles and .task models from storage.googleapis.com. That host is blocked in mainland China — which is exactly where the original's Simplified Chinese interface was written for. On that network the page does not fail loudly; it sits on 「正在加載模型…」 forever. The fix is not a mirror: it is shipping the bytes with the app.

What it actually does

Watch, then teach

There is no built-in gesture library. You pick an emoji, hold a pose for 1.5 s and the app keeps only the frames where the detector agrees on the hand count — a frame with a second hand wandering in is dropped rather than averaged into the template.

Median, not mean

The template is the per-dimension median of those frames, with the median absolute deviation beside it. One blink of a bad landmark cannot drag a median the way it drags a mean.

A tolerance per gesture

thr = clamp(0.075 + 3.6 × MAD, 0.09, 0.30). A still recording earns a tight threshold; a shaky one earns a forgiving one. This is the fix for the single global slider that made every gesture either trigger-happy or deaf.

Mirror-invariant matching

L1 distance over 42 hand dimensions, also scored against the mirrored template, so a left-handed and right-handed attempt at the same pose are the same gesture. Faces are matched on 52 blendshape scores — expression, not geometry.

Fire on agreement

The best match has to win four consecutive frames, then a 1.6 s cooldown holds. Without that handshake a single stray frame mints a reaction card, and the gallery fills with garbage.

Request path

first paint
   └─ GET index.html                 140,498 B  (inline CSS + JS, no framework)
   └─ GET ./vendor/vision_bundle.mjs    136,870 B  MediaPipe tasks-vision
   └─ GET ./wasm/vision_wasm_internal.wasm  9.4 MB  SIMD runtime
   └─ GET ./models/hand_landmarker.task     7.8 MB  21 hand landmarks
   └─ GET ./models/face_landmarker.task     3.8 MB  478 landmarks + 52 blendshapes
                    │
                    └── all five from THIS origin. Same host as the page.
                        No storage.googleapis.com, no jsDelivr, no fonts.

Two feature vectors

Hand · 42 dimensions

21 landmarks, translated so the wrist is the origin
divided by palm size (wrist → middle knuckle), so distance from the lens stops mattering
x and y kept, z dropped — depth is the noisiest axis on a laptop webcam
scored against the mirrored vector as well, so hand laterality is free

Face · 52 blendshapes

MediaPipe's ARKit-style coefficients — jawOpen, browDown, mouthSmile…
a face at a different distance produces the same vector, which landmark distance cannot promise
runs at ~11 Hz, not 30: the face model costs about 3× the hand model, and an expression does not need frame-perfect timing

Measured, not assumed. With the face model on every frame the preview ran at 2 fps in software rendering; throttling it to every 90 ms and caching the result between detections put the same scene at 14–18 fps with no change in recognition scores.

The duel has no server

Two browsers need one SDP blob each way. There is no signalling server holding it: the offer is JSON → CompressionStream('gzip') → base64, which is short enough to paste into a chat window, and the answer comes back the same way. After that the connection is peer-to-peer and the site is not in the path. It is a deliberate trade — a manual code exchange instead of an account and a signalling box — and it keeps the whole app deployable as static files.

What broke, and what it cost

Symptom	Cause	Fix
Skeleton floated off the hand, labels read backwards	The video and the overlay canvas were both CSS-mirrored with `scaleX(-1)`. Mirrored pixels are fine; mirrored text is not.	Mirror only the video, and flip the x axis inside the canvas draw — labels stay legible.
Every window opened at 0,0 in the top-left corner	`clampIntoStage()` ran over closed windows on resize, writing `left:0px` into their inline style. `show()` then read a truthy `"0px"` and skipped parking entirely.	Bail out unless `data-open="1"` — a hidden window keeps its saved slot.
Preview dropped to 2 fps once a hand appeared	Face landmarker + blendshapes on every frame, on a software rasteriser.	Face detection throttled to ~11 Hz with the previous result reused in between.
Camera window cut its own empty state in half	A fixed 520 px window against a 4:3 video plus a 110 px note block.	Height derived from the real stack (570 px) instead of guessed once.
Duel codes forced a horizontal scrollbar	Base64 in a textarea with default `width` set the fieldset's min-content width.	`width:100%` + `box-sizing:border-box` + `word-break:break-all`.

Verified by driving it

Headless Chrome, a synthetic camera fed from a still image of a hand (--use-file-for-fake-video-capture), and the app's own test hooks:

hand detected → 1 hand, 21 landmarks, overlay drawn
gesture taught → 42 dimensions, auto tolerance 0.10 from a perfectly still recording
live score on the same pose → 0.97, card auto-captured → 1 card in the gallery
SELFTEST PASS — 18/18, zero console errors: i18n completeness, matcher, duel codec, a real RTCPeerConnection datachannel loopback, minesweeper placement/cascade/win, the window manager and card rendering
390 × 844: five real tabs, no horizontal overflow, no clipped panels

What is different from the original

	gestomeme.com	GESTO 2000
Models	fetched from `storage.googleapis.com`	served from this origin, first paint included
Training	one pass, one global threshold	median template + per-gesture tolerance from your own jitter
Hand features	raw landmark distance	42 dims, wrist-origin + palm-normalised, mirror-invariant
Face features	landmarks	52 blendshapes
Duel	needs a signalling path	one gzip+base64 SDP code each way, no server
Language	简体 only	繁體 / 简体 / EN, every string a triple
Layout	fixed pixels	window manager (drag, resize, minimise, cascade) + real mobile tabs

Honest limits. Accuracy was verified with a synthetic camera, not a real webcam — the detector, the feature path, the template and the card pipeline are all exercised, but nothing here proves how well it behaves under your room's lighting. 14–18 fps is a software-rendering number in headless Chrome; a normal GPU-backed browser is faster, and a decade-old laptop may not be. Gestures live in localStorage and cards in IndexedDB, so they are per browser and per device — clearing site data deletes them. The duel requires both people to paste a code each way; there is no contact list and no discovery. And it is a rebuild of an idea, not a fork: none of the original's code or art is reused.

Play GESTO 2000 Request path Portfolio