Skip to the build notes
ALEX SCHAR / FLETCHERAll updates
← All build updates

Desk interaction

Aug 30, 2026 · 1:38 video

Giving hand tracking its own accelerator

I install the AI HAT so tracking and spoken responses do not have to compete for all the same processing on the Pi.

WHERE THIS TEST ENDED

The board is fitted and the new program is the next test. This episode states the performance problem; it does not measure the improvement yet.

  1. Camera frames
  2. Accelerated tracking
  3. Hand positions
  4. Dashboard interaction

Two jobs competing on one small computer

I describe tracking slowing down while Fletcher speaks. In the video I report roughly 24–28 frames per second on its own and 12–14 during speech. Those are the figures I give for the problem in this episode, not a new measurement taken for this page.

The goal is to keep interpreting hand movement while the assistant processes a question and plays its reply through the projector.

Fit the accelerator, then change the software

I connect the HAT using the small ribbon connection, realize I have initially picked up an older board, and replace it with the intended one. I then put the assembly back above the desk to run the new program.

The hardware is only one half of that change. The tracking software has to use the accelerator for the intended workload. Fitting a board does not, on its own, establish a faster or more accurate interaction.

Test the promised improvement separately

I describe a division between tracking the hands and processing responses, along with the goal of handling both hands. The episode ends before the new test result.

The later lighting update is an important follow-up: even with an accelerator, a camera still needs a usable image of the hand. Compute capacity, visibility, and calibration each need their own check.

About this breakdown

Matched to this original TikTok, including its spoken captions. The result above describes this episode; later work is identified separately.

Breakdown checked September 6, 2026.