Computer Vision · Multimodal

Teaching a system to read bean behavior during a roast.

Roast logs, chamber-aligned computer vision, and audio combine into a machine-readable description of fluidization, lift, packing, and phase behavior in a fluid-bed roaster — not just “a fan event happened here.”

Detector MAE2.40s
Fan transitions <5s4/4
Bean-state features4
Detector passes31

The signals are showing real bean-state structure.

Not just “motion happened here” — chamber-relative measurements of how the bean mass is actually behaving.

Best detector so far

Aligned multimodal, phase-aware is the current benchmark at 2.40s MAE on the reference run, with all four fan changes landing inside 5 seconds.

Alignment changed the measurement

Movement mean rose to 0.052 on the latest run once the region-of-interest followed the chamber instead of a fixed crop. Chaos and bed-state numbers shifted too — the old measurements weren’t wrong, they were just unaligned.

Phase-aware picking matters

On the stress-test run, the phase-aware aligned pass cut MAE from 30.40s to 6.84s by stopping the picker from treating power changes like fan changes.

The bean-state features are the asset

Fan timing is still just a proxy. The stronger result is cleaner chamber-relative measurements for bed height, fluidization, coherence, and lift — usable across roasts and roasters.

How the system turns video and logs into something a model can learn from.

A deliberate pipeline. Each step is a choice about what to measure and what to ignore.

01

Align the chamber

Each roast is registered to the roaster, not to raw frame pixels.

02

Track bean-state

Motion becomes bed height, fluidization, coherence, and lift.

03

Compare to the log

Bean behavior is lined up with bean temp, RoR, fan, power, and roast events.

04

Surface transitions

Phase-aware detection highlights moments that look operationally important.

Bean-state variables that matter more than the raw detector score.

Fan timing is a proxy check. The bean-state readings are the real product.

Bed Height

Vertical reach of the moving bed.

A direct read on whether the coffee is settled, lofted, or riding too high.

Fluidization

How freely the beans circulate.

Healthy fluidization usually sits in the middle, not at either extreme.

Coherence

How tightly the bean mass moves together.

Higher coherence means a denser, more packed bed with less separation.

Lift

How much activity reaches the upper chamber.

Useful for spotting airflow changes that are subtle in the raw video.

How the measurements move from run to run.

Aligned, chamber-relative readings — movement and chaos shift with the workflow, not just the bean.

Movement mean

aligned ROI · normalized
Reference run
0.063
Stress test
0.050
Latest run
0.052

Chaos mean

aligned ROI · normalized
Reference run
1.464
Stress test
1.670
Latest run
1.674
Detector: aligned multimodal, phase-aware Capture: video + audio + log, frame-aligned