I'm amused by how the whole reasoning model thing feels like a formalization of the old "think step by step" prompting hack, which was discovered against GPT-3 two years after that model was first released.
My favorite trick for controlling the reasoning level is the hack where you look at the output token stream and spot the token for "the model has concluded reasoning"... and then replace that with the tokens for "wait, but" and force it to keep going!
embedding-shape 1 days ago [-]
> My favorite trick for controlling the reasoning level is the hack
First described here I think, specifically for Qwen models: https://arxiv.org/abs/2501.19393, although it seems unclear how much it actually improves the final response unless you create a new fine-tuned model with this in mind, and also if it's possible to replicate for other models. Still, got inspired and gonna give it a try with DiffusionGemma although slightly differently
diwank 1 days ago [-]
[flagged]
sva_ 1 days ago [-]
I recommend his book, "Build a Reasoning Model (From Scratch)", which is also linked in the article.
LLMs don't actually reason, they just create the illusion of reasoning by repeating things over and over.
solumunus 4 hours ago [-]
It has the same effect as reasoning. Its reasoning in a practical sense if not a philosophical one, and that’s what most people care about.
tenuousemphasis 17 hours ago [-]
So do most people.
cyanydeez 20 hours ago [-]
mmm, practically, llamacpp solved this with reasoning budget for tokens and a customized message.
ive tailored my local models to match agent with a message that pushes to use subagents and context compression.
it works pretty smoothly if theres proper vector and scope to the project. theres probably a way to also get it to record memories but ive not seen a memory system that includes spatial type reasoning which would create memories associated with file paths down to AST graphlike leaves.
then we could include remembering details about the path.
llamacpp also has a header for setting these that an intelligent harness could tailor per round message and budget. ideally you start with a small budget and expand based on some complexity criteria.
alas, i want to build things not related to AI.
connerpro 1 days ago [-]
[flagged]
DekryptLabs 1 days ago [-]
[flagged]
kimonsodu 1 days ago [-]
[flagged]
tal-onn 15 hours ago [-]
[flagged]
Rendered at 21:28:55 GMT+0000 (Coordinated Universal Time) with Vercel.
My favorite trick for controlling the reasoning level is the hack where you look at the output token stream and spot the token for "the model has concluded reasoning"... and then replace that with the tokens for "wait, but" and force it to keep going!
First described here I think, specifically for Qwen models: https://arxiv.org/abs/2501.19393, although it seems unclear how much it actually improves the final response unless you create a new fine-tuned model with this in mind, and also if it's possible to replicate for other models. Still, got inspired and gonna give it a try with DiffusionGemma although slightly differently
https://sebastianraschka.com/books/#build-a-reasoning-model-...
ive tailored my local models to match agent with a message that pushes to use subagents and context compression.
it works pretty smoothly if theres proper vector and scope to the project. theres probably a way to also get it to record memories but ive not seen a memory system that includes spatial type reasoning which would create memories associated with file paths down to AST graphlike leaves.
then we could include remembering details about the path.
llamacpp also has a header for setting these that an intelligent harness could tailor per round message and budget. ideally you start with a small budget and expand based on some complexity criteria.
alas, i want to build things not related to AI.