2. We combine both on Cactus; rolling entropy method helps with per-token signal for early exit, while the probe runs on complete output.
3. Users hate this in production, drains battery on edge devices, but ultimately primarily works ok-ish on 7B+ models, we will share the results too.
2. We combine both on Cactus; rolling entropy method helps with per-token signal for early exit, while the probe runs on complete output.
3. Users hate this in production, drains battery on edge devices, but ultimately primarily works ok-ish on 7B+ models, we will share the results too.