Relax prefill parser to allow space.#21240
Conversation
|
Fixes #20356 |
|
We should move the space to every parser creation that needs this. Not every model requires it. |
|
@aldehir wouldn't that clutter the code without measurable performance benefits? |
|
That's why I added |
|
I tried GLM 4.7 flash and Qwen 3.5 27b with reasoning and tools. Seemed to work fine, no loose think or /think tags spotted, no chat template errors. I noticed new reasoning budget messages that weren't there a few weeks ago (but this is unrelated to this pr):
|
|
Yeah, I keep on forgetting about |
* Relax prefill parser to allow space. * Move changes from prefix() to parser generation * Only allow spaces if we're not having a pure content parser next
* Relax prefill parser to allow space. * Move changes from prefix() to parser generation * Only allow spaces if we're not having a pure content parser next
Overview
As in title.
Additional information
Prefill parser was strictly requiring the reasoning marker at the very start of the message, which interfered with models that liked to insert eg. a newline there.
Requirements