Do you have a source for this? I'm pretty sure responses api is only there to obfuscate reasoning, but that they're still keeping reasoning traces in the backend.
> Input and output tokens from each step are carried over, while reasoning tokens are discarded.
Keeping reasoning tokens around is better for caching and for remembering past insights, so you might reasonably wonder why we designed it this way. The main benefit of dropping reasoning tokens is that you can fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step. This was a larger consideration with our earlier reasoning models that had shorter context windows (~200k), longer thinking times (up to ~100k per message), and poor compaction. However, now that we've shipped longer context windows, we've trained our models think much more efficiently, and we've made compaction way better than it used to be, the balance of factors is changing. Tune in Thursday!
> fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step
This is something I never understood. Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction. Was it to optimize caching? Well I guess it doesn't matter now that you hinted that this choice was made because of prior limitations and may change very soon
> Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction.
Models are typically trained (at longer conversations/more turns) either with or without the reasoning still in the conversation. If you train a model with those, then using it without them, the model will perform a lot worse, same vice-versa if you train without but then end up using the model with them.
That's why you'll see some models have it and others don't, and trying to use them another way, will make them worse, they weren't trained like that.
So why aren't the models trained with both? I'm guessing that sort of permutation in the training would lead to double the amount of training time being needed, as you know effectively will have two variants of every session you train on, with and without the reasoning.
Well, obviously its true as you can see in the silbing comment. But still interesting how experiences differ. I wonder did you ever use codex in more demanding workflows, because that at least something like that is going on is obvious if you use it for a few hours
You answered at more or less the same time as an OpenAI engineer confirming this is the case, explaining why they did it, and pointing to their docs that also very explicitly explain this "made-up nonsense". You should really consider why you felt so certain about something you were plain wrong about.
