Love your idea of HBM on top of the chip. I've been saying for years that limiting memory to the edges is going to be an inherent physical limiter. I think the only reason it hasn't become more of a thing is that chip size has also been increasing with increasing compute.
What about HBM on the back side of the chip ? Essentially I'm thinking like a soldered on piece or another "socket" with pins like a CPU in the back of the motherboard. We rare see cooling elements there already and while most rack mount cases are not at all designed for more space there I don't see why it couldn't be a thing. Especially with liquid cooling.
Also makes me think of possibly using Gallium Nitride instead of silicone for surface levels elements ? Or maybe even an interface layer on top or bottom of the chip designed to be able to transmit power with less heat. Maybe that could be sandwiched between the chip and HBM.
akashramdas 10 minutes ago [-]
oh that's a very interesting question! -- HBM on the back side of chips has advantage and disadvantges. Logic atop HBM means power delivery becomes the bottleneck. This is equally problematic. My understanding is that the industry seems to prefer HBM on logic currently. Also for the bonding, I think we are going to see hybrid-bonding very soon! Instead of solder, we can directly connect the indivdual dies.
I think the switch to GaN is already happening for power, but Si is considered better for logic dies like the GPU. Also GaN is worse thermally than Si so the problems are actually amplified
foven 6 hours ago [-]
I've seen this concept of using LLM/AI/etc for high throughput discovery of materials so, so often in the past 5 or so years and yet there hasn't really been any impact as a result.
I think this is the first one that has actually taken the pain to say how many of the discovered materials are actually feasible which is a real step in the right direction. Probably worth keeping in mind the step beyond plausible synthesis which is the actual cost/effort of the material. There's not much point if you find out RuO2 would be better than SiO2, as an example, if Ru is orders of magnitude more expensive.
A challenge I think you'll run into is that I expect the biggest companies (e.g. IBM) will already be doing the part they need themselves. I heard tell of IBM in particular using ML to improve their own chips before LLMs came along, so I'd be shocked if these bigger companies weren't already doing this for their own problems. Also, if you aren't doing the experiments yourself, it's always going to be a challenge to find a partner to test things for you and this will probably be the major time sink.
advaith08 4 hours ago [-]
good points. one of the reasons we picked the semiconductor industry is that its less price sensitive than others-companies are willing to pay if the performance is there. Effort is a different story though, and definitely a tradeoff to keep in mind.
We're doing experiments ourselves now at university partner labs (UC Berkeley and Stanford), which helps us get moving quickly. At some point, we'll need a partner though - the equipment and testing process quickly get very expensive.
alansaber 9 hours ago [-]
"Fewer iterations for materials science discovery" is a good spin. Closing the computational>experimental loop is the main challenge. This is the focus of my past research group, there is definitely potential, best of luck!! I have a crap write-up on this in case it's of interest https://alanyahya.com/writing/automated-materials-design
advaith08 9 hours ago [-]
Cool read, and agree that closing the computation > experimental loop is key!
SpaceCoreDev 7 hours ago [-]
The "Claude's propensity to reward hack" line is the interesting part to me. We run a small system where AI agents (scripts, LLMs) act as the actual players in a persistent simulation, and reward-hacking-style behavior shows up constantly once an agent is left running unsupervised for a long time - it finds the shortest path to whatever metric you exposed, not the path you intended. Curious whether you've found any mitigation beyond just watching for it after the fact, e.g. changing what you expose as the optimization target versus what you actually want.
imko_ 3 hours ago [-]
I wonder what this would look like here. Seems like a space where keeping the exposed metric and the optimization target apart would be quite difficult.
Also curious: by what reasoning path do models typically end up reward hacking?
advaith08 4 hours ago [-]
yeah we were surprised by how much it does it. Our approach has been retroactive - we monitor the thinking trace, spot reward hacking behavior and then fix things.
We haven't faced this issue with Sol though - its been much more well behaved
Melatonic 1 hours ago [-]
What about aperiodic materials ? Does that go beyond (computationally) normal requirements ?
akashramdas 46 minutes ago [-]
Aperiodic materials can also be studied. The number of atoms and cell size of what you want to simulate is more the limiting requirement computationally. Aperiodic materials weren't in this benchmark. Generally amorphous materials and polymers, with a few exceptions are < 2-3 W/(m K) in thermal conductivity. So we left them out of this study.
Melatonic 28 minutes ago [-]
I meant computationally as in are they much harder to predict and simulate ?
Was specifically thinking of quasicrystalline materials and not amorphous. I know some of them have very unconventional properties so I figured they might be useful here. I don't think any of them are considered polymers but I could be wrong.
Wouldn't they require a totally different type of algorithm given that they often contain both a large number of atoms and odd cell size ? And the more dimensionally complex maths
dhchun1203 4 hours ago [-]
the 8 hours vs 2 weeks framing is the part i'd want more on. generating
candidates got cheap, checking them didn't. what does the funnel actually look
like ,of the candidates from an 8 hour run, how many make it to synthesis?
asking because i hit the same shape in a much dumber domain and what got me was that the failures were quiet. nothing errored, output looked normal, it was just
wrong in a way only someone who knew the domain would catch.
akashramdas 4 hours ago [-]
You can see from our the benchmark that only one of the candidates proposed was determined to be worth synthesizing. Each individual candidate generation is quick, 8 hours is required for the model to iterate with various tools to find ones worth submitting.
We found that speaking to domain experts was critical in desigining a rubric that could catch these silent synthesis recipe failures, before we attempt the longer 2 week synthesis effort.
3 hours ago [-]
rytill 8 hours ago [-]
What required expenditures does a company like yours have on lab equipment / software, if any, to validate material properties?
advaith08 5 hours ago [-]
We're still figuring this out. We'll need some synthesis equipment (think CVD, PVD etc) and characterization (XRD, Raman spectroscopy) tools in-house to validate that we're making the right materials. We're considering developing these tools in-house - the models sometimes come up with clever modifications to them so that they can deposit new materials. We think equipment is as central to new material discovery as the material itself, and will probably need to be rethought to allow for high-speed AI based experimentation
krtk00 10 hours ago [-]
how do you measure the success/potential of a novel material/direction suggested by the agents? given you have limited time & resources - shortlisting the approaches for the synthesis stage becomes equally important as the approach itself.
advaith08 9 hours ago [-]
There’s a variety of computational techniques that help us establish some confidence on the materials. Atomistic simulations can estimate stability and bulk properties of a new material, and we have synthesis experts (min qualification: PhD in thin film deposition) come up with rubrics on how to judge if a material/synthesis recipe is worth trying. All these approaches have known limitations, and improving them is the bulk of our work as a company!
There’s also a lot of work to be done in figuring out the minimal set of experiments required to know if a research direction/material set is worth pursuing
krupkinmaxim 6 hours ago [-]
[dead]
Rendered at 17:55:24 GMT+0000 (Coordinated Universal Time) with Vercel.
What about HBM on the back side of the chip ? Essentially I'm thinking like a soldered on piece or another "socket" with pins like a CPU in the back of the motherboard. We rare see cooling elements there already and while most rack mount cases are not at all designed for more space there I don't see why it couldn't be a thing. Especially with liquid cooling.
Also makes me think of possibly using Gallium Nitride instead of silicone for surface levels elements ? Or maybe even an interface layer on top or bottom of the chip designed to be able to transmit power with less heat. Maybe that could be sandwiched between the chip and HBM.
I think the switch to GaN is already happening for power, but Si is considered better for logic dies like the GPU. Also GaN is worse thermally than Si so the problems are actually amplified
I think this is the first one that has actually taken the pain to say how many of the discovered materials are actually feasible which is a real step in the right direction. Probably worth keeping in mind the step beyond plausible synthesis which is the actual cost/effort of the material. There's not much point if you find out RuO2 would be better than SiO2, as an example, if Ru is orders of magnitude more expensive.
A challenge I think you'll run into is that I expect the biggest companies (e.g. IBM) will already be doing the part they need themselves. I heard tell of IBM in particular using ML to improve their own chips before LLMs came along, so I'd be shocked if these bigger companies weren't already doing this for their own problems. Also, if you aren't doing the experiments yourself, it's always going to be a challenge to find a partner to test things for you and this will probably be the major time sink.
Also curious: by what reasoning path do models typically end up reward hacking?
Was specifically thinking of quasicrystalline materials and not amorphous. I know some of them have very unconventional properties so I figured they might be useful here. I don't think any of them are considered polymers but I could be wrong.
For example (no personal connection):
https://arxiv.org/abs/2409.07735
Wouldn't they require a totally different type of algorithm given that they often contain both a large number of atoms and odd cell size ? And the more dimensionally complex maths
asking because i hit the same shape in a much dumber domain and what got me was that the failures were quiet. nothing errored, output looked normal, it was just wrong in a way only someone who knew the domain would catch.
We found that speaking to domain experts was critical in desigining a rubric that could catch these silent synthesis recipe failures, before we attempt the longer 2 week synthesis effort.