I ran into some issues that are more extreme versions of Qwen's thinking loops while reasoning. It is great at the small puzzles I set for it but it did some frankly insane things on a PHP coding task I set it.
It also had some issues that might be parsing/chat template stuff, tool calling oddities. I will try it again, I did try it pretty much the day it shipped and it's possible there are more improvements in their llama.cpp fork since.
It would be churlish to be overcritical, mind you — the PrismML ternary stuff is an advance — but it feels like it should be applied at training. I figure we will see that, somewhere, quite soon.
Did you try the BottleCap ThinkingCap Qwen post-train with the reduced thinking overhead?
Not tested much but it is not noticeably worse than the underlying Qwen 3.6 27B in Q4_K_M, which in my experience is kind of a first for a Qwen fine-tune of this nature. They are almost always worse.
I think it does use fewer tokens while reasoning, which is potentially useful. I need to do more testing, because any performance advantage over the 27B is useful for me on an M1 Max.
It also had some issues that might be parsing/chat template stuff, tool calling oddities. I will try it again, I did try it pretty much the day it shipped and it's possible there are more improvements in their llama.cpp fork since.
It would be churlish to be overcritical, mind you — the PrismML ternary stuff is an advance — but it feels like it should be applied at training. I figure we will see that, somewhere, quite soon.
Did you try the BottleCap ThinkingCap Qwen post-train with the reduced thinking overhead?