01
Thermal overlay
Move your pointer (or finger) along the leg. A perfusion heat-map fades in where you look — and every annotation lands in the margin, never on the body itself.
Perfusion scancanvas sim
hover / drag over the leg
probe: —
Why margins matter here: the cool zone on the lower leg reads “low perfusion — check diabetic foot”, but the label and arrow live beside the limb. The center of the body stays clean for hands, wounds and eye contact.
02
Contactless vitals
A camera signal — ambient light reflected by the skin — is enough to extract a pulse. No electrodes, no cuff, no touch. The waveform below is simulated from that principle.
Camera PPGsignal sim
runs automatically
HR … bpm · SpO₂ …% · camera-based, no touch
How it will work: the glasses' RGB camera tracks micro-changes in skin color caused by blood flow (photoplethysmography), computes heart rate and oxygen saturation on-device, and flashes the number at the edge of the lens — a glance, not a screen between you and the patient.
04
Margin HUD
The attention budget: the center of vision belongs to the patient. Everything the agent wants to say lives in the periphery — and only one object may ask for attention at a time. Click any margin chip (or press the button) to hand it the budget.
Attention budgetHUD sim
click a margin chip, or cycle
budget holder: vitals chip · center: reserved for the patient
The rule the page itself practices: marks live in margins. A HUD that paints numbers over a face is a HUD that gets taken off. One object at a time; the rest wait their turn, dimmed.
A
What the glasses do
Four capabilities, one frame — each demonstrated above.
Thermal overlay
clinician-facing
- Live perfusion heat-map over skin, revealed by gaze/pointer, not plastered on
- Cool zones flagged in margin notes — diabetic foot, circulation, inflammation
Contactless vitals
clinician-facing
- Camera-based PPG: HR, SpO₂, respiration — no electrodes, no touch
- Numbers at the lens edge; alerts escalate to the margin, never the center
Shared marks
agent-mediated
- Anchors pinned to body-space, not screen-space — stable as the patient moves
- Multi-viewport sync: what the doctor marks, the nurse sees, same second
- Signable receipts — who marked what, when, on whose record
Margin HUD
agent-mediated
- Attention budget: center belongs to the patient, always
- One attention request at a time; everything else queues dimmed in the periphery
B
What we can program
MARG is built on an OEM Android-based glasses platform. We own the whole software lane — this is what our code touches.
Firmware lane
our code
- OEM Android-based SDK — full application layer, display compositor for both lenses
- Sensor drivers — thermal sensor, ToF depth, IMU fusion, mic-array beamforming
- On-device vision models — body tracking, anchor stability, PPG extraction; quantized, no cloud round-trip
- Update by signed deltas — firmware and models ship as cryptographically signed incremental updates; unsigned code never boots
Why Android-based matters
our lane
- Mature toolchain: Kotlin/C++, NDK, A/B partitions, OTA infrastructure
- Real-time display pipeline we can throttle — the attention budget is enforced at compositor level, not by app politeness
- Existing OEM hardware lanes mean we program, not fabricate — faster to clinical validation
D
The pair: glasses + tablet
One exam, two screens, two depths. The glasses capture and point; the tablet deepens and shares. Neither replaces the other — they are the pair.
Glance on the lens
glasses role
- Capture & anchor — cameras and sensors read perfusion, vitals and body position while the doctor keeps hands free and eyes on the patient
- Voice inquiry — "examine the left leg", "show the last photo of this lesion" — answers arrive as quiet margin confirmations, never covering the patient
- Point — the mark, the alert, the number: one object at a time at the edge of vision
Dive on the tablet
companion role
- The full picture — complete thermal map, exam history, side-by-side comparisons — content too rich for the lens margin, shown where it belongs
- Shared with the patient — the doctor turns the screen: the patient sees their own data; trust grows in the conversation
- Extended battery & compute — heavy processing can run on the companion, keeping the glasses light enough for a full shift
Any Android device the doctor already owns
no dedicated hardware
- Just an app — the MARG companion is a regular Android app: install from the store, pair with the glasses over encrypted link, done
- Works on what is already in the pocket — phone, tablet or the clinic's existing devices: the doctor buys the glasses; the second screen is the one they already have
- Offline-first sync — paired devices share the same signed marks in real time; if the link drops, each keeps working and reconciles later
Same truth on both screens
house doctrine
- One mark, every device — what the doctor marks by voice appears identically on the lens margin and the tablet screen, same second
- Signed both ways — voice commands and marks produce receipts: who asked, what was shown, when — on whose record
- Fits every delivery mode — hospital on-prem (clinic-owned tablets, data never leaves), hybrid, or clinic SaaS with each doctor's own device
E
Every sensor, honestly tiered
Not a wish list — a physics list. What fits in the frame today, what fits with tradeoffs, and what belongs in a companion clip-on.
In the frame today · proven
tier A < 80 g
- Stereo RGB cameras — lesions, wounds, pallor, video-PPG dermatology, primary care
- Thermal LWIR — perfusion, inflammation, thrombophlebitis diabetic foot flagship
- ToF depth — wound volume, edema, range of motion
- IMU · mic array · ambient sensors — anchors, voice UI, and the calibration that keeps thermal & PPG honest
Feasible with tradeoffs
tier B
- IR eye tracker — concussion screening, Parkinson's saccades, cranial nerves neurology gold
- mmWave radar — patient respiration + heart rate through clothing, no touch
- Multispectral bands — tissue oxygenation, graft viability, burn depth
- UV-A · polarized light — lesion margins, melanoma screening adjunct
- Mini LiDAR — posture, scoliosis, joint alignment
Companion clip-ons · physics says not in-frame
tier C
- Ultrasound probe (USB-C) — the killer app: image overlaid anchored on the body part itself
- Digital stethoscope — murmur AI, anchored to chest location
- OCT module · dermoscope · single-lead ECG — retina, mole mapping, AFib screen
Specialization packs
same frame, tuned
- Primary care — full-body triage, vitals, wound, skin
- Diabetes / vascular — perfusion, oxygenation, DVT, foot ulcer tracking the university flagship
- Dermatology · Neuro-ER · Cardiology · Ortho · Pediatrics (cute mode, zero touch)
- Each pack = signed sensor modules + on-device models, hot-swappable per clinic profile
Honesty bar: nothing here "sees inside the body". We overlay registered imaging from companion sensors, anchored to the patient — the honest, regulated version.
F
Hard to hack, harder to trust when hacked
"Unhackable" does not exist. The goal: a stolen or jailbroken MARG is worth nothing — no keys, no data, no clinical trust, no clone value.
Silicon root of trust
layer 1
- Secure Element — keys born on-chip, never exportable de-solder the chip, the keys die with it
- Verified boot chain — every link signs the next; one modified byte = no medical mode
- Anti-rollback — patched holes stay patched
Attestation
layer 2
- Every boot, every pairing — glasses prove firmware is genuine to the companion app and clinic server
- Jailbroken = rejected — the app refuses to show clinical overlays from a device that fails attestation
- No TEE, no deal — a hard OEM requirement
Data & runtime
layer 3–4
- Per-exam encryption, keys in the Secure Element, destroyed on remote wipe
- Raw frames zeroized after inference — nothing to steal at rest
- Sensor-spoof physics check — a fake "healthy" overlay inconsistent with body geometry is rejected
- Marks signed in Secure World — a compromised OS cannot forge a doctor's signature
Fleet & lifecycle
layer 5
- Stolen = revoked = clinical brick within one heartbeat of report
- Signed research mode — bug bounty gets a no-patient-data build, curiosity channeled away from jailbreaks
- The real moat — hardware is copyable; the signed trust layer, the fleet, the evaluation history — are not. A cloner gets empty silicon.
Regulatory dividend: these layers are exactly what FDA 510(k) / CE-MDR cybersecurity guidance requires — designed to pass, not retrofitted.
G
Can it be built? Yes — here is the honest math
Deep scan of 33 makers, factory paths and funding instruments — every number sourced in the study dossier. Nobody sells what MARG needs; the gap is real, and the physics is proven.
The market gap, verified
Nobody sells this yet
market scan · 33 makers
- No shipping glasses combines thermal + ToF + RGB under 80 g with a lock we control
- Closest consumer (RayNeo X3 Pro 76 g, Rokid 49 g): no thermal, no ToF, app-level SDK only
- Medical (Augmedics, Medivis): approved software over someone else's glasses
But the physics is proven
existence proofs
- UTRACK ships AR + 384×288 thermal in one frame — thermal-in-frame is integration, not physics
- RealWear integrates an official FLIR module in industrial glasses
- FLIR Lepton 3.5 weighs 0.9 g — sensors fit; nobody has assembled the set
The build path
we are the OEM of record
- Optics + compute from existing lanes (Vuzix Ultralite OEM / Thundercomm AR1 reference)
- Thermal from FLIR Lepton, depth from ST ToF — the integration RealWear already proved
- Secure-boot root keys provisioned to us — the vault is ours (section F)
The cost ladder — proof before money
A · Bench rig
€370–580 · 2 weeks
- Pi 5 / Jetson Orin Nano + FLIR Lepton 3.5 + ToF + camera
- Proves the clinical pipeline: thermal/RGB registration, radiometric temperature, edge inference
- Zero hardware-commitment money spent
B · Wearable mock
€720–1,250 · +2–3 weeks
- Adds real AR viewer + 3D-printed frame — what the clinician actually sees
- Demo-grade evidence for doctors, universities, IRB, investors
C · ODM pilot build
€22–35k · MOQ 300–500 · 7–12 mo
- Platform re-skin by Tier-2 Shenzhen ODM — pilot clinical units, manufacturability
- Full-custom NRE (€220–350k+) only after clinical signal justifies it
Who pays — and the regulatory chess
Funding, ranked by speed
non-dilutive first
- Business Finland Sprint — €75–100k, decision ~2 months, sized for the functional prototype
- BF Deep Tech Accelerator / NIY — up to 3×€400k or €1M over 6–12 months
- Eurostars with a European university/hospital — 50% of SME costs, no cap, pays for the alliance itself
Medivis' wedge, not Augmedics' war
regulatory strategy
- Medivis: software over off-the-shelf glasses → 510(k) at $84–221k, cleared in 7 weeks on $2.3M seed
- Augmedics: custom hardware first → $10M+ burned before clearance
- MARG route: software on COTS glasses first (education pilot), own hardware when the grant pays
Every number sourced in the study dossier: market-deepscan, odm-prototype-path, funding-allies (2026-09-27). Built while validating.
H
The first prototype: what it already does, and for whom
Scenarios A+B (bench + wearable, under €1.3k) already carry real exams. Honest framing: screening adjunct and education — not diagnosis. Diagnosis comes with the regulated build.
Exams the first prototype can run
Thermal perfusion mapping
thermal + RGB
- Diabetic foot screening — temperature asymmetry between feet, the best-evidenced use in the literature
- Circulation screening adjunct — cold zones, Raynaud, PAD follow-up
- Marked in margins, tracked over visits, signed each time
Wound & inflammation documentation
RGB + ToF + thermal
- Wound photo + volume measurement (ToF depth) — healing tracked with numbers, not adjectives
- Inflammation surveillance — hot spots around incisions and joints over time
- Same anchor every visit — the photo lands in the same body-space point
Contactless vitals
camera PPG
- Heart rate, SpO₂, respiration — no electrodes, no cuff, no touch
- Numbers at the lens edge during the exam, full curve on the tablet
Lesion & mole tracking
RGB stereo
- Standardized capture angle every visit — comparison finally apples-to-apples
- Shared marks on suspect lesions — "which mole did you mean?" ends here
Education mode — the university pilot
with any Android tablet
- CT / X-ray / ultrasound DICOM → 3D — the real aneurysm, the real fracture, rotatable and sliceable in class
- Professor's shared mark seen by every student at once; every mark signed
- Anonymized cases from the university itself — data never leaves the campus
Aesthetic medicine
thermal + RGB + ToF · tier 1-ready
- Perfusion mapping pre/post — fillers, toxins, threads: vascular compromise caught as a heat signature BEFORE the skin changes — the nightmare complication becomes visible early
- Recovery tracking — post-laser, post-peel, post-procedure: inflammation heat map + photo + volume at the same anchor every visit
- Standardized before/after — same angle, same light, same anchor: the comparison patients ask for, finally honest
- Clinic marketing, signed — thermal "before/after" images are striking AND factual — a differentiator no competing clinic has
- Scar & healing documentation — volume by ToF, pliability context: plastic surgery follow-up with numbers
Motion & posture baseline
ToF + IMU
- Range-of-motion measurement with marks anchored to joints
- Posture screening adjunct — numbers for physio follow-up
Who we go to first — the tiers
Tier 1 · Prototype allies
education + screening pilots
- Brazilian federal medical universities (Bruno's lane) — anatomy & radiology teaching via PD&I convênio: no regulatory burden, real users, real cases
- UEF / Kuopio university hospital — diabetes & wound-care research groups: Kuopio is a diabetes research capital; thermography of the diabetic foot is their literature
- Doctors: endocrinologists & diabetic-foot teams, wound-care specialists, dermatologists, pilot GPs
Tier 2 · Pilot clinical
ODM build (€22–35k tier)
- Vascular clinics — perfusion + ulcer tracking at scale
- ER triage — contactless vitals queue, hot-spot flagging
- Physio & ortho — ROM numbers; occupational health screenings
Tier 3 · Specialization packs
regulated product
- Cardiology (stethoscope companion + murmur AI), Neuro-ER (eye-tracker version), Ophthalmology (OCT companion), Pediatrics (cute mode, zero touch)
- Each pack = the section E matrix, tuned — one frame, many clinics
Why Tier 1 is Tier 1: strongest published evidence (diabetic-foot thermography), zero regulatory barrier (education lane), universities bring grants, validators and cases — and Bruno is already at the door.
C
What they must have
The non-negotiables — the frame we will accept from hardware, and the privacy bar we hold ourselves to.
Sensors
hardware floor
- RGB + thermal + ToF — color, heat and depth fused into one body-space anchor model
- IMU — head tracking so marks stay glued to the world, not the lens
- Mic array — beamformed voice for hands-free dictation and mark placement
Wearability
hardware floor
- < 80 g — a full shift must forget it's there
- All-day battery — thermal + vision workloads budgeted to last the shift, not the demo
Privacy
our bar
- On-device first — video and vitals processed on the glasses; raw frames never leave by default
- HIPAA / GDPR-grade — signed data lineage for every mark, every update, every access
House rule this page practices: characters stay cute and kid-friendly, and nothing — not a heat-map, not a label, not a HUD — covers a face or the center of a body in a medical demo. Marks live in margins. That's not a limitation; it's the product.