NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
▲Claude was trained on protein sequences at scale (xinmingtu.cn)
t0mek 5 hours ago [-]
In "Empire of AI", the author mentions that when OpenAI included public GitHub repositories in its training data, the resulting model achieved better results even in fields unrelated to programming.

Aside from the obvious applications in biochemistry, could protein sequences also be used as a new training data source to improve overall model quality? If that's true, it's quite fascinating that we can take highly complex data we don't fully understand and use it to improve LLM training, simply because it contains so many significant structures and relationships.

phkahler 5 hours ago [-]
Software is used in all fields, and github issues and PRs are going to contain detailed reasoning about all sorts of things when adding features or fixing bugs.
fritzo 3 hours ago [-]
Right

> GitHub repositories ... in fields unrelated to programming

About as surprising as training on books and improving in fields other than writing.

origini 3 hours ago [-]
We have seen it happening in math and physics.

We have seen that LLMs are quite good in reading binary and changing binary directly.

Could be possible.

5 hours ago [-]
eddiewithzato 13 hours ago [-]
Claude just blocks from anything in this field. Can't even work with my own DNA without triggering opus 5.5 50% of the time
_ink_ 12 hours ago [-]
I expect this to be another revenue stream. My guess is that there will be an accreditation step and afterwards you get access to an unlocked model. Just as in security. And possibly it will come with a price tag.
chriskanan 7 hours ago [-]
I'm told by colleagues in the life sciences and pharma space that it isn't just "safety" but more like artificial market segmentation. If one has to pay extra for guardrail removal, then that's market segmentation. Anthropic makes most money from company whales. Want the guardrails gone? Pay a lot extra over baseline. But Mythos probably had PEFT adapters, so that at least makes it a bit different. But still there is not nearly enough transparency about these things.
MisterMunchkin 3 hours ago [-]
And oopsie whoopsie, looks like we accidentally guardrailed out all of the startups that might challenge your pharmaceutical company. Thanks for being such a loyal customer :)
5 hours ago [-]
FredFS456 11 hours ago [-]
jwr 11 hours ago [-]
This already exists with Mythos. You can apply to get approved for cybersecurity testing, which is relatively easy if your need to just test your own software. But Mythos is accessible only through the API, not through your subscription, so you will be paying quite a bit for it.
LunaSea 5 hours ago [-]
As long as there is no possibility for large companies to self-host Mythos to scan local code I don't think the current Mythos offering is reasonable.
rmonvfer 11 hours ago [-]
Are you sure about this? I’m verified for cybersecurity uses and I can use mythos with no issues with my subscription.
Chu4eeno 8 hours ago [-]
You need to pay for extra usage credits for Mythos, there's no included usage of it in a subscription.

It was included in the subscription up to monday (for mythos/fable).

captn3m0 7 hours ago [-]
> It was included in the subscription up to monday (for mythos/fable).

It seems to be counting against my Fable limits.

bortimus 5 hours ago [-]
Have access to Mythos, use it as an alternative to Fable when I'm out of Fable credits. Seems to not count against it, or at least have its own limits.
r_lee 11 hours ago [-]
or you have to have your organization approved so then everything will be done through approved "partners" I.e. consulting companies who add their own markup ontop of everything. we're already seeing this with Mythos
amelius 7 hours ago [-]
Why do you think it's not national security/health related?
DonsDiscountGas 6 hours ago [-]
Claude has no way of knowing whether the DNA you're working with is yours. And you probably wouldn't want to change that.

My own life science experience is that Fable 5 was completely unusable at first but 5.1, 5.2, and Opus 5.5 rarely trigger the safeguards.

noduerme 10 hours ago [-]
What does "triggering it" look like in this case? Does it want verification that it's really your own DNA?
dlougheed 8 hours ago [-]
what does "working on your own DNA" even mean?
dekhn 2 hours ago [-]
I have a BAM file containing my personal genome sequence. In principle I might want to use Claude to analyze it (I do not- I don't think it would be very useful ).
timcobb 6 hours ago [-]
Not OP but guessing they sequenced their DNA and would like to work it up or analyze it
svnt 4 hours ago [-]
In Nov/Dec 2025, just prior to the training cutoff for the first model to show this capability (Opus 4.7) I was working on solving arc-agi using claude code and was exploring using protein data and other adjacent data to improve performance.

In January it seemed very clear the subscription models behind claude code became less competent, there was a whole series of posts about it. I stopped working on it as a result.

LarsDu88 37 minutes ago [-]
I just tried this, and the ability of Opus to recognize functional residues from arbritrary protein sequence is quite extraordinary. A skill I don't think any humans have outside of focused understanding of specific proteins.
highfrequency 4 hours ago [-]
It would be cool if Claude were not explicitly trained on protein sequences at scale, and this were an emergent property from better fundamental understanding of chemical structure.
mlmonkey 3 hours ago [-]
Does shuffling still result in a viable protein? I'm no biochemist, so this is a naive question. Do proteins have some (chemical) constraints on how the "letters" can be arranged?
dguest 3 hours ago [-]
Shuffling still gives you a viable amino acid sequence but it might fold up into garbage or just fall apart.

Real proteins have a number of structures which are composed from sequences and which make up the larger functional proteins. In the extreme case if you have a good idea what the protein does and an unprecedented way to predict its folding you could probably guess most of the sequence having never seen it.

You can think of it as computer code. It's a terribly written language that does no static checking or error handling, and in which every conceivable hack is used. But since many proteins do similar things they also reuse the same substructure.

mlmonkey 2 hours ago [-]
[Displayingmy ignorance here] So is it possible that the model has learnt which letters are valid continuations? Like, for example, the rule in English "I before E except after C". Could the model just have figured out some simple rules from a small number of examples and is just applying them?
zodwick 12 hours ago [-]
Is this a valuable skill ? Sorry, bit of a noob question ig but have been seeing a lot of buzz around protein sequences in general
gilleain 12 hours ago [-]
Do you mean 'skill for a human' or 'Skill for an LLM'? :)

I used to do bioinformatics, although more of the structure side than sequence. Like an information or understanding, it depends on what outcome you want that might benefit from the skill.

I suppose that predicting the next amino acid in a sequence would be useful for, say, reconstructing a protein sequence from fragments. Or constructing evolutionary trees of sequences by predicting the common ancestors?

zodwick 11 hours ago [-]
I'm not sure how to phrase this correctly sorry, But I suppose the question I have is what comes after say "reconstructing a protein sequence from fragments" -> like what how does one go from this to developing a drug or enzyme. Is being able to predict the next amino acid useful in this process ?

Linking to ML space, is this analogous to knowing the model's weights byheart ? Traditionally this has been not useful to humans and machines, but recently some LLMs have started showing capability to directly edit the model weights to alter the behavious of smaller models. Is that a completely wrong analogy ?

Also I guess what I am partly curious is on how this data ended up in the model. Was it part of the pretraining corpus or was there an RL environment which specifically rewarded this behaviour

gilleain 11 hours ago [-]
Ok the next steps beyond predicting a sequence would then be modelling the structure - which requires something like AlfaFold or 'traditional' (as it were) software for molecular mechanics and dynamics.

How useful the predicted sequence is for the predicted structure - and predicted function - no idea! There are already several layers of prediction here.

Sorry, I definitely do not know enough about LLMs to understand the analogy to weights. You can predict the effects of mutations on sequences, I think? As in, if I change this Q to an E will it make the enzyme faster (??). Perhaps you could LLM-generate a plausible member of a family of enzymes with a particular function, given fragments of an existing protein (???).

No idea how the data got in the model. It seems like the study that Xinming and Chen have done was designed to eliminate sources from the general corpus of knowledge - like any old science papers - b choosing obscure and rare sequences that might only be found in a sequence db.

zodwick 4 hours ago [-]
Unfortunately I don't have any expertise in bioinformatics and hence can't provide good feedback. Almost as if we need something with good understanding of both fields :)

jokes apart I do hope the LLMs ability to being a master at multiple unrelated fields unlock valuable insights

On the data getting in the model - I suspect: as the models have gotten bigger, the labs have started including more info from sequence databases of some sort

dfdydx 11 hours ago [-]
I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about.
zodwick 5 hours ago [-]
good point on not needing labeling, hence proly makes sense to throw it in in pretraining and see if useful capabilities emerge. Funny how a lot of this is still a black box
throw310822 10 hours ago [-]
They both depend on how much generalisation you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics.
dlougheed 8 hours ago [-]
an LLM-predicted evolutionary tree seems monumentally useless, since it'll be unstable and you have no guarantees/metrics on any property or quality of the tree itself (parsimony, likelihood, estimated TMRCA, etc.)
zodwick 4 hours ago [-]
out of my comfort zone, but out of curiousity; are these parameters ( parsimony, likelihood, estimated TMRCA, etc.) verifiable easily ? If so I wouldn't dismiss capabilities of LLMs in predicting these. All tasks that can we RL'd will be RL'd imho.
gilleain 8 hours ago [-]
Totally fair, I'm well beyond my comfort zone in bioinformatics :) I should stick to structure.
5 hours ago [-]
kazinator 13 hours ago [-]
Gemini should be good at naming the next sugar in a polysaccharide. :)
troupo 12 hours ago [-]
You, however, will be banned for "biosecurity" research. Or, as Amodei puts it, "you can't have opensource models because with them you can't ban users"
kypro 6 hours ago [-]
What's the steel man argument for giving all of humanity access to advanced bio research and capabilities?

I think I'm naive sometimes when it comes to trade-offs between freedom and security, but even I'm confused at why it would be a good thing for the whole of humanity having access to this stuff...

My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model with these capabilities so it might be safer if we brought forward the risks while they are still management so people take them extremely seriously.

If we go 5-10 years and then the weights of an ultra-capable model is leaked that may be game over.

wavemode 3 hours ago [-]
> What's the steel man argument for giving all of humanity access to advanced bio research and capabilities?

Well that question is very loaded and slippery-slope-ish. To bring things down to practical terms, the real question is "should we pass laws restricting open models". (And probably with an added caveat "in the US", since lawmakers can't control what AI researchers overseas do.) Which has a lot more substance for debate on both sides.

wry_xy 3 hours ago [-]
The knowledge and capabilities are latent regardless of which humans if any have access to them. Same for their risks.

We can "protect" ourselves by limiting who has access, but:

- how often do those "trustworthy" elites use their abilities for abuse?

- it might leak

- someone else could independently obtain the capabilities

- an "act of god" might hit us without human intent

In any event, we're less prepared, because we assumed it couldn't happen. And it's not just bio.

And that's ignoring deeper questions of enforcement or censorship or surveillance.

I won't fight for a world where people are intentionally kept dumb, where the powerful are intentionally kept powerful, and where we intentionally keep ourselves vulnerable.

troupo 6 hours ago [-]
> What's the steel man argument for giving all of humanity access to advanced bio research and capabilities?

What's the steelman argument for giving this research to Anthropic? Or any research that they are now gating?

> My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model

Yes. Because most of this research is either public, or should be public (but hidden behind bullshit fees by gatekeepers like Elsevier)

octoberfranklin 12 hours ago [-]
Curious: how can there be a correct prediction for the next element of a "shuffled" protein? The shuffled proteins don't occur in real life, so what makes one answer correct and another incorrect?
tstactplsignore 7 hours ago [-]
It is a negative control in the experiment. Shuffling is a better control than random because it accounts for the raw rates of amino acids in each protein. For example W is very rare, so Claude could beat a truly random generator just by guessing not W. But the signal seen on real sequences is much stronger than that, so it tells you that Claude is doing something more than just guessing 'not W' on real sequences.
octoberfranklin 4 hours ago [-]
But the signal seen on real sequences is much stronger than that

But the shuffled sequences aren't real sequences!

Sorry if I'm being daft here, but I still don't understand what the source of truth is for the shuffled sequence tests.

wavemode 4 hours ago [-]
To make an analogy - in one test the researchers took a real number from nature and asked the AI what it was, and the AI guessed it correctly a decent amount of the time. In another test the researchers just rolled a die and asked the AI what they rolled, and it couldn't guess correctly. The "source of truth" is just whatever the die roll was.

It may seem obvious that the AI fails in the control experiment, but the purpose is to rule out the possibility that information about the correct answer is leaking into the model context somehow. Or that the model is relying on some statistical technique like frequency analysis.

octoberfranklin 1 hours ago [-]
Oh! So a lower score on the "shuffled" sequence is more validating of the experimental hypothesis.

I wish the article had stated this clearly. It seems to imply the opposite, which is of course false.

4 hours ago [-]
anonymous_sorry 12 hours ago [-]
It is a rather odd control. Is it just to confirm that Claude isn't reading the sequence off disk?
gilleain 12 hours ago [-]
Perhaps. They explain why:

> Shuffling keeps the letters and destroys their order, so the difference in accuracy measures what the model knows about the sequence itself.

So by analogy, if you know a word in English starts with 'Q', you could reasonably predict the next letter would be 'u'. By shuffling, you might encounter the letter Q later on, and not successfully predict a 'u'.

Put another way, the shuffled sequences can only use frequency analysis for prediction, while the unshuffled ones have local and global order signals.

octoberfranklin 11 hours ago [-]
Yes but what is the source of truth for the next element of a shuffled sequence?

For a natural sequence, the observed molecule in nature is the source of truth.

gilleain 11 hours ago [-]
We are definitely going beyond the borders of my understanding of sequences here, but I would guess the source of truth is just the expected frequency distribution of amino acids? So 'G' (glycine) is most frequent, and something like 'W' (tryptophan) is least frequent (dependent on organism?).

However I thought the point was that the _difference_ between the shuffled prediction and the unshuffled is the measure here. So shuffled is the baseline, and the gap above that is how good the prediction is.

pcrh 7 hours ago [-]
It's hard to see what the point of this is?

When would one want to predict a full-length unknown protein sequence from only a fragment? Any fragment of any useful length can be back-translated to the DNA sequence and that can be tested against genomic databases (a process called tblastn). And if your protein/gene is not found there, you have no way of knowing whether or not the predicted sequence has any basis in reality.

Testing the model against shuffled sequences is also kinda cheating, as these are no longer random pool of amino acids, but one whose source already exists in the databases.

tstactplsignore 7 hours ago [-]
I think the point is probably to improve reasoning about protein design work. If Claude has an innate understanding of protein sequence variation then it is able to just eyeball potential issues in protein designs. To be clear, I'm pretty sure Anthropic would ask claude to design proteins using task specific with conventional tools, but they then want it to be able to look at results and reason about them, akin to an expert. Not sure whether it'll actually help but I think that is the logic.

[0] https://assets.claude.com/30bf50e22a01388bb29bf077ee3f244531...

da_chicken 6 hours ago [-]
Probablye because that's only part of the problem.

Doesn't anyone remember the Borderlands Science! mini-game that was in Borderlands 3?

https://www.nature.com/articles/s41587-024-02175-6

suddenlybananas 10 hours ago [-]
What's more surprising is that the other models aren't
arbot360 9 hours ago [-]
Goes to show how far Anthropic is going with unconventional RL tasks / synthetic datasets compared to others. The isn't really any natural human task that would require you to do this. This makes sense for their philosophy of sublimating everything into the model though, anything that can be done on a computer should be able to be approximated inside the model eventually.
HackerThemAll 10 hours ago [-]
No doubt that all of them, except maybe Gemini, surely were trained on SciHub and all books + data they could download from torrents and other sources, as well. That could include leaked personal data and proprietary copyrighted material, from anywhere maybe even including Darknet. Add to that public patent registry and the book buyout and scanning campaigns that occur.

I am certain that all these companies have stored and backed up all they could find, so that it can't be taken away from them.

spwa4 8 hours ago [-]
DNA is a language, or at least a protocol. Just a very difficult one. All meaning is locked up in the shape of the Lennard-Jones 12-6 magnetic flux isosurface that is generated when you string the letters together from amino-acids. Probably justified to call that a bit more indirect than English.

And electrostatic interactions.

And some methylation.

And some phosphorisation (but now we're getting deep into somewhat rare grammar)

Yeah, DNA is a language that is way too complicated for human minds to understand, although plenty of humans "get the gist" so to speak after looking at it for a few hours, but ... here's some bad news: it's only somewhat too complicated for LLMs. So why wouldn't you train LLMs on them? Frankly, it's the one language we have "texts" from that is actually difficult to understand for LLMs and ... well, it's definitely improving their understanding.

(also we have quite a bit of DNA text that isn't from the internet or books, it's freely downloadable and unlike everything else LLM companies steal, actually not covered by copyright)

dekhn 2 hours ago [-]
"Lennard-Jones 12-6 magnetic flux isosurface " isn't a thing.

LJ potential is, but it doesn't have a magnetic flux or an isosurface.

Filipc 9 hours ago [-]
What I find most interesting here isn’t necessarily that Claude can predict protein sequences, but what this tells us about how knowledge is acquired by large models.

The shuffled-sequence control is particularly compelling. It suggests that the models aren’t simply recalling amino acid frequencies or reproducing patterns from scientific literature. They’re capturing structural information embedded in the sequences themselves.

There’s an important distinction, though, between learning statistical representations of protein families and actually understanding the underlying biology. Next-amino-acid prediction is a fascinating proxy, but it doesn’t necessarily demonstrate that the model has learned the mechanisms governing protein folding or function.

I’d be particularly interested in seeing whether these capabilities transfer to genuinely novel protein families, rather than previously unseen members of known families. The post-cutoff experiment is a good step in that direction, although it still tests generalization within existing families.

Either way, this raises an interesting question about foundation models: how much latent scientific knowledge are we inadvertently creating simply by exposing them to sufficiently large and diverse datasets?

Perhaps we’re underestimating what can emerge from learning the statistical structure of the natural world.

yesornoisyes 5 hours ago [-]
I wonder how they would do on novel protein sequences?

I mean, that’s the holy grail, no? In fact, I’m genuinely surprised the other labs don’t train on protein sequences. It’s both an excellent source of high quality information and potential breakthroughs.

gigiamoroso 11 hours ago [-]
[flagged]
thin_carapace 11 hours ago [-]
stop being anti vax
supersimpleguy 7 hours ago [-]
[dead]
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 19:27:03 GMT+0000 (Coordinated Universal Time) with Vercel.