NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
Laguna S 2.1 (poolside.ai)
Lwerewolf 1 days ago [-]
Testing it now. At the very least, competitive with DS4-Flash indeed. On my small (and per Sol's words, _very_ semantically dense) C test codebase, it found things that only gpt-5.2 managed to find back in the day, but also made a stupidly incorrect initial observation that a memfd_create()/mmap was used for IPC (funnily enough - sol missed that as well in its review, until I pointed it out). Re: the claims vs deepseek v4 - both flash and pro are expected to get a "general availability" release very soon (i.e. well-"post-trained"), so things can change in a... well, flash, as per usual in the current environment.

Anyways, keep 'em coming.

ilc 1 days ago [-]
What harness/quant did you use for testing?
Lwerewolf 1 days ago [-]
nvfp4 mlx, literally barebones pi.

edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings.

embedding-shape 22 hours ago [-]
> edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings.

Been playing around for a few hours with the poolside/Laguna-S-2.1-NVFP4 + poolside/Laguna-S-2.1-DFlash-NVFP4 + vLLM, been seeing the same behaviour. Usually new model releases are plagued with issues at release though, best to wait 1-2 weeks then retry, or better yet, investigate yourself :) Personally I haven't found any obvious issues.

embedding-shape 12 hours ago [-]
Update: Seems quite literally they have bugs on the hardware I'm trying to run this with:

From Poolside CEO Eiso Kant on Twitter:

> Learning we have some bugs on the RTX6000. We’re on it. Team has worked non stop last days and it’s getting late for a lot of the inference folks, so might be until tomorrow till we have a solution. - https://x.com/eisokant/status/2079693050796785720

Update2: I'm now running poolside/Laguna-S-2.1-NVFP4 with vLLM 0.23.1rc1.dev1378+gd6dbdb9b0 (FlashInfer 0.6.14) and seeing slightly better results in regards to the looping. I can't see any specific changes that would affect this though, strangely enough.

embedding-shape 6 hours ago [-]
Update3, summarized from RTX6kPRO Discord:

> The BF16 checkpoint doesn't exhibit any of the complaints [...] with our current quants, the models tend to choose the wrong logits sometimes [...] why we'll need a requant [...] We're not aware of any bugs in any runtimes themselves [...] We have two remaining things that we're trying to tackle: some people are reporting thinking being too hard to trigger (^^) , and others are saying it thinks too much. We've seen much more of the latter internally

Lwerewolf 5 hours ago [-]
Deleted earlier, didn't see you post, pasting here:

/* Just started testing with the gguf (with gpu offload, m5 max 128gb), q4_k_m, running seemingly well. Speed is initially slightly faster than antirez/ds4 - decode tok/s in the 30s, prefill ~400-ish. Expected, given the slightly smaller size. Looks to be working fine, but too early to tell. Definitely likes to "think". */

Anyways, guessing that discord might be focusing on the nvfp4 stuff. I've noticed spelling mistakes in the thinking traces, tool calls have been fine so far.

Lwerewolf 2 hours ago [-]
Well, just ran said gguf on the GeneralsX codebase with a pretty open-ended "Explain this codebase to me, and the general game loop." prompt, and...

  Let me also look at the GameLogic::update() to see the rest of the update flow, especially the object update loop.

  Actually, I think I have enough information now. Let me also check the GameLogic::UPDATE to understand the full update flow.
...repeating forever.

Since they mentioned that they're working on new quants, guess I'll wait. From earlier tests on work stuff, it's definitely capable.

5 hours ago [-]
sosodev 23 hours ago [-]
What inference server are you using? They have a custom branch for llama.cpp, but I wouldn't be surprised at all if it still needs fixing.
Lwerewolf 23 hours ago [-]
This: https://github.com/Blaizzy/mlx-lm/tree/pc/add-lg ...and this is what I should probably wait for (not sure why it's in vlm): https://github.com/Blaizzy/mlx-vlm/tree/pc/laguna-s-nvfp4 ...or perhaps I should've just used the gguf with the provided llama.cpp instead of trying to run the nvfp4-mlx from the get go, but where's the chaos in that :)

Running deepseek flash on something locally now, this will have to wait a bit. I still stand by my initial quick assessment - looks capable. Some people on r/localllama also reported loops. We'll see in ~10 hours. Hopefully I haven't terribly mislead people.

19 hours ago [-]
mchusma 1 days ago [-]
Incredible. This is definitely the launch of the day. Just crushing Google's releases.

The pricing here is incredible. This is the first US release that's competitive with DeepSeek V4 Flash. Very excited about this.

river_otter 1 days ago [-]
Hey, this model is not a joke! Exciting, we already got a usable PR of work out of it.

https://github.com/mozilla-ai/otari/pull/348

mft_ 1 days ago [-]
Looks impressive, and this size fits achievable home hardware.

That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.)

Edit: someone in the process of doing so: https://huggingface.co/vcruz305/Laguna-S-2.1-GGUF

Archit3ch 21 hours ago [-]
> if someone would kindly quantise this down for the 64GB paupers

The Q4_K_M is 75GB. I'm exactly at 64GB, and I wouldn't quantize it further. Instead, do partial weight residency and stream the rest from SSD.

yogeshp 24 hours ago [-]
They have also published smaller 33B model called Laguna XS 2.1, its Q4 gguf is 20GB.

https://huggingface.co/poolside/Laguna-XS-2.1-GGUF/tree/main

mft_ 23 hours ago [-]
Thanks for flagging. From the few benchmarks I can find, it looks there or thereabouts with Qwen 3.6-35B-A3B, or maybe a touch below. I'm interested to compare a model that is a big jump larger with pretty impressive benchmarks, but more heavily quantized to fit.
mft_ 7 hours ago [-]
This is interesting: https://github.com/tanishq-dubey/macos-laguna-s2.1

Someone has benchmarked a wide range of different quants.

nunodonato 12 hours ago [-]
unsloth is always here to save the day https://huggingface.co/unsloth/Laguna-S-2.1-GGUF
andy99 12 hours ago [-]
Seems it’s not fully supported in mainline llama.cpp yet https://github.com/ggml-org/llama.cpp/pull/25165

In the huggingface link they mention building for CPU and for CUDA, does anyone know if that means it wouldn’t be possible to build targeting Vulkan?

andy99 9 hours ago [-]
Replying to myself, seems this PR was merged into main and it the model does work with a Vulkan backend on my Framework desktop, I’m getting about 220 tok/s prompt processing and 21 tok/s output on the 4-bit quant. This is really a sweet spot imo on this machine between maximizing ram use and still having decent speed due to the expert size. This looks really promising.
regularfry 9 hours ago [-]
What's your hardware? I've got 64GB RAM and a 4090 here, wondering if it's worth a play.
andy99 8 hours ago [-]
It’s the Strix halo (AMD) with 128 GB shared memory. The 4bit quant is ~75GB.

Unfortunately I don’t know about the best way of running on an Nvidia gpu, you could try llama.cpp and offloading as many layers as possible into the gpu and using RAM for the rest, not sure if that would slow it down too much though.

regularfry 4 hours ago [-]
I got usable token rates (10-20tps from memory, so marginal) with the Qwen A10B a while back, well before all the new speculative speedups landed in llama.cpp. There's an unmerged branch which allegedly supports this, but I don't know how well yet. Looks worth investigating but I might give it a few days to see what bugs get shaken loose.
verdverm 24 hours ago [-]
The tool I've been using, llm-compressor, can quant models that do not fit in memory (use the sequential pipeline)

https://github.com/vllm-project/llm-compressor

my setup to help you on your way: https://github.com/verdverm/quantr

Though it seems these will not be needed as Poolside has published quants & dflash with their models.

alfiedotwtf 12 hours ago [-]
Nice! Do you know of any tools that do this for tensorrt models?
verdverm 4 hours ago [-]
Sorry, I don't, still newer to the quantizing side of things since many are already produced by others.
kamranjon 1 days ago [-]
Whoa whoa whoa, 118b params, 8b active MOE, long context reasoning, open weights - music to my ears. Hadn't heard of this lab before but I am very excited, will definitely try this out tomorrow - this is a real sweet spot I think in terms of model size and performance.
svclaws 1 days ago [-]
If the numbers are legitimate then our prayers have been heard
SwellJoe 1 days ago [-]
This is exactly the kind of model that's been needed in the middle. Realistically self-hosted, Good Enough intelligence, MoE so it's fast on limited bandwidth systems like Strix Halo and DGX Spark.

For a while there's been nothing to run on my Strix Halo that's notably better than what I can run on my dual 32GB GPU desktop (Gemma 4 or Qwen 3.6 dense models), but this seems likely to be the step up in size that actually works better than those.

aubanel 23 hours ago [-]
Really impressive signal that this 128B model can beat DeepSeek V4 (1.6T) on most coding benchmarks!

Also, I really like Poolside's habit to compare not only to other top models in its weight class (others don't do it, looking at you Mistral), but also to the very top open-weight models, even much bigger ones like the 2.5T Kimi-K3!

benjiro29 23 hours ago [-]
!! Be careful when testing the model.

A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinking has a issue with the default configuration.

Important - make sure that THINKING is enabled. By default it wasn't although I was passing the flag --default-chat-template-kwargs '{"enable_thinking": true}' in vllm recipe. The generation_config.json file that is included has by default max_new_tokens as 32k which seems to be cutting off thinking altogether so increase it. At first I was very disappointed with the output I was seeing, but once thinking is enabled, the code quality seems to be MUCH better. More real world testing to be done.

https://www.reddit.com/r/LocalLLaMA/comments/1v2pg99/laguna_...

d2p 13 hours ago [-]
Looks like the default chat template was updated on HF to enable this by default shortly after you posted this :-)
7 hours ago [-]
voxgen 23 hours ago [-]
Even the official provider on OpenRouter seems to have this issue. Hope it's an easy fix for them.
nshotton 22 hours ago [-]
Thanks for posting this, it made a huge difference tweaking the recipe.
Iolaum 1 days ago [-]
Model Looks amazing!

Even more important, subjectively, is that this model will run very well on Strix Halo (e.g. Framework Desktop), DGX Spark kinds of devices. Looking forward to Unsloth dynamic mtp quants.

P.S. Looking at the HF release they already offer Q4_K_M and DFlash drafter for speculative decoding!

verdverm 24 hours ago [-]
I hope all models going forward come with a dflash drafter so we don't have to train one up separately.
river_otter 1 days ago [-]
I love this. Is it possible to give a feel of how this stacks up to the good old Opus 4.5 in coding quality? For me that was the turning point where agentic coding in Claude Code etc became usable. Have we hit that threshold?
megavon 1 days ago [-]
Having played with it for like 3 hours now....I'm probably moving from CC to this
river_otter 1 days ago [-]
I am about 1 hour into using it with pi.dev. Do you have thinking on high? It is doing good but at one point i had to stop it and say 'you're overthinking this' haha
megavon 24 hours ago [-]
Yes full send mode on thinking. I have moved on from watching my agents and I don't really care how it thinks. I look at the end result and so far this thing has been blowing me away. No way this is as good as it is this small and fast. Outside Fable, this might be the best thing I've ever used.
kamranjon 1 days ago [-]
What quant are you using?
river_otter 1 days ago [-]
fingerprinter 24 hours ago [-]
One hour in, no more Codex for me. This thing rips.
mark_l_watson 9 hours ago [-]
For me, poolside.ai “came out of nowhere” a week or so ago when I discovered their local coding harness ‘pool’ and their smaller 33G MOE model that runs fast and is effective on my old 32G mac mini. Really good work!!

I need to evaluate their large hosted model.

spelk 22 hours ago [-]
This is fantastic work, really impressive is an understatement. I really hope this sets a new DeepSeek-esque standard and starts another the death knell for companies continuing to cosplay as frontier labs (like Cohere).
megavon 1 days ago [-]
This is INSANE. How did they do this?
eisokant 1 days ago [-]
"What we've done in this model is not necessarily add more intelligence, but improve the behaviors that lead to a more capable model: more verification, less taking things for granted, not declaring victory early, and being more persistent.”

+

https://poolside.ai/assets/laguna/laguna-m1-xs2-technical-re...

Lwerewolf 1 days ago [-]
Almost like a built-in heavyweight harness.
aitchnyu 12 hours ago [-]
Are all AI labs in Google's weight class in crawling and ranking the Web's content? I know OpenAI has contractor subject matter experts in all topics.
1 days ago [-]
1 days ago [-]
kamranjon 1 days ago [-]
"It went from the start of training to launch in under nine weeks..."

This is pretty impressive.

luciana1u 17 hours ago [-]
64GB RAM used to mean you could run three VMs and a database. now it means 'maybe the 120B model fits if I close everything else'
khurs 9 hours ago [-]
chat as listed on that page: https://chat.poolside.ai
loolhahalmao 23 hours ago [-]
happy the US has some counterweights to the Chinese labs, just need about half a dozen more.
platinumrad 17 hours ago [-]
Does it refuse to work on "cyber"?
kouteiheika 10 hours ago [-]
It's an open-weight model so it literally doesn't matter whether it refuses by default or not, because it's pretty trivial to uncensor[1] any open-weight model and make it not refuse.

[1]: https://github.com/p-e-w/heretic

tosh 1 days ago [-]
open weights and

similar performance to deepseek v4, inkling at size of nemotron 3 super (!)

resonious 15 hours ago [-]
How much does it cost? I even made an account and I cannot find pricing anywhere...
asar 12 hours ago [-]
In / Out Price

$0,10 / $0,20per 1M

from openrouter

samelldev 7 hours ago [-]
impressive benchmarks for the active params. i'll load it up and test
iraldir 1 days ago [-]
Amazing model at this size if true, that's quite crazy!
docheinestages 23 hours ago [-]
Any estimates of the performance (prompt processing and decoding tokens/s) on consumer hardware like Macbook Pro M-series?
axus 22 hours ago [-]
You will not, directly or indirectly: copy, modify, or create derivative works of the Site or any Product;
drob518 18 hours ago [-]
Immediate reaction is that it seems to be a bit behind Meta Muse Spark 1.1 performance at approximately the Deepseek v4 Flash price point. That's quite good given Muse Spark benchmarks a lot better than Deepseek v4 Flash (assuming benchmarks mean anything, which they don't).
markasoftware 17 hours ago [-]
have only tested a few prompts, but it failed my favorite non-coding question that dsv4 flash aces. Benchmarks look excellent though (don't they always!)
polski-g 18 hours ago [-]
This thing is great, twice as fast as DS4Flash and slightly smarter too. I swapped most of my sub-agents to this model.
_mrinalwadhwa_ 22 hours ago [-]
Has anyone tried it on a mac yet?
_mrinalwadhwa_ 17 hours ago [-]
Was able to run it on Apple M3 Max (128 GB)

host: Apple M3 Max, 128 GB model: Laguna-S-2.1, 118B-A8B MoE, Q4_K_M (75 GB), DFlash speculative decoding server: http://127.0.0.1:8000, llama.cpp, ctx 64K, 8-bit KV cache

  mode: max thinking
   #  tokens   tok/s  dflash
   1     600    14.4     11%
   2     600    26.1     27%
   3     600    17.8     18%
   4     600    14.0     16%
   5     600     9.3     15%
  --------------------------------
  median  14.4   mean  16.3   min   9.3   max  26.1   tok/s

  mode: no thinking
   #  tokens   tok/s  dflash
   1     190    10.0     20%
   2     109    26.7     65%
   3      95    29.6     72%
   4      93    32.8     81%
   5     382    14.0     30%
  --------------------------------
  median  26.7   mean  22.6   min  10.0   max  32.8   tok/s
alfiedotwtf 12 hours ago [-]
Anyone know if this is supported in ik_llama.cpp?
throwdbaaway 12 hours ago [-]
It works, thanks to https://github.com/ikawrakow/ik_llama.cpp/pull/1911, which got merged in early June. However, there might still be some issue with the chat template.
alfiedotwtf 6 hours ago [-]
Awesome, thanks!

Edit: it amazes me how fast ik_llama.cpp moves

suprjami 12 hours ago [-]
Just merged into main llama.cpp a few hours ago:

https://github.com/ggml-org/llama.cpp/pull/25165

carimura 24 hours ago [-]
Congrats Poolside team!!
verdverm 19 hours ago [-]
initial impressions, great model for coding, probably swapping it out for qwen 27b for a while to long-term test, more sycophantic than any I've run locally myself
danr4 23 hours ago [-]
holy shit its accelerating fast
literallyroy 1 hours ago [-]
[dead]
Archit3ch 21 hours ago [-]
[dead]
1 days ago [-]
reindeer2 10 hours ago [-]
I've been following work on the second-order effects that ripple through the system for a while. This is the first treatment I've seen that the framing reveals an assumption that isn't explicitly defended.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 20:37:01 GMT+0000 (Coordinated Universal Time) with Vercel.