We gave a gym app eyes — 30 FPS rep counting, zero wearables, fully on-device.

Gym tech relies on hardware. We were asked to remove it.

A two-stage on-device pipeline, purpose-built for the constraints.

We designed a pipeline that separates what the person is doing from how many times they've done it — two distinct inference problems that each have a right-sized answer. Decoupling them meant we could tune each independently without touching the other, and swap components as the hardware targets changed.

• 01Replaced raw video with skeleton sequences as the model input. Running pose estimation first strips away everything the gym environment does to confuse a model — bad lighting, moving backgrounds, varied clothing. The result is an input that's stable across locations and people.
• 02Ran action recognition asynchronously on a background thread. The recogniser sees a rolling window of skeleton frames and emits a stable exercise label 4–6 times per second — fast enough to react, slow enough not to thrash. It never blocks the camera thread.
• 03Used a cycle-detection approach with linear — not quadratic — time complexity. Most repetition counters compare every frame against every other frame, which gets expensive fast. Our counter uses a smarter representation of motion cycles that stays fast even during a 45-minute session.
• 04Built a graceful degradation layer for low-end devices. On chipsets below a defined benchmark, the system steps down to a lighter pose model and adjusts the frame budget — staying useful at 24 FPS rather than dropping to something unusable.
• 05Fine-tuned the entire pipeline on client-collected gym footage. Benchmark datasets capture controlled lab conditions. We collected ~40k clips of real gym-environment reps and used them to close the gap between academic accuracy and production accuracy.

98% accuracy. Real gyms. Mid-range phones. No cloud.

The app passed the client's internal "shelf test" — propped 8 feet away against gym equipment, counting through 12 exercises across six testers without manual recalibration. Thermal headroom after a 45-minute continuous session stayed within spec on every test device. The beta hit their internal accuracy threshold (≥95% within ±1 rep) in the first week of user testing.

How the system is structured under the hood.

We don't publish model names or exact architectures publicly — the stack is a competitive advantage for the client. What we can share is the structural logic, which applies to any project in this class.