The GGUF quantisations do run on most recent hardware, albeit at increasingly co... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

Filligree on Oct 3, 2024 | parent | context | favorite | on: FLUX1.1 [pro] – New SotA text-to-image model from ...

The GGUF quantisations do run on most recent hardware, albeit at increasingly concerning quality tradeoffs.

tripplyons on Oct 3, 2024 [–]

I haven't noticed any quality degradation with the 8-bit GGUF for Flux Dev, but I'm sure the smaller quantizations perform worse.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact