You guys might want to look at Alphadidatic [0] and it might be worth your time to see if that model published 5 months ago still generalizes. Also, all the prompts are tuned for Claude 4.6 and I needed to throwout or rewrite all my agents, prompts, skills for Claude 5 which fortunately seems like it handles recursive self-improving agents natively.
I can't justify spending $1k - $2k a month for real time options data and compute for what is in my 401k. I guess the question I have is would your tool help me trade these strategies and more important test them on a few hundred a month?
Very interesting. I am curious: Is the "test" part actual live trading, or is it also a backtest?
I am asking because I see the agents can do websearch, so how is Alphadidatic preventing the agents from just looking at the asset's performances and cheat?
Back to our product. It is not 2C yet. So we are building the RL Envrionments for finetuning LLM, not providing a personal trading agent. However we can also build a professional trading harness using the setup we have. If you are interested, let's keep in touch: michael.zhang@edotenv.com
jfrbfbreudh 22 hours ago [-]
This is very cool, but who is the ideal customer here? I used to work at one of the top tier shops and we had multiple teams whose entire responsibility was building and maintaining our simulation environments.
Mzzzzz 21 hours ago [-]
We are selling to frontier labs, e.g. OAI and Ant, and fintech companies that are trying to build agentic trading system but do not have those internal quant setups, e.g. Coinbase, Kalshi etc.
cromwellian 1 days ago [-]
I'm skeptical frontier LLMs can actually do well (e.g. alpha 5%+) without fine-tuning, especially on historical market data. Presumably you support fine-tuned models?
RuiWang0811 24 hours ago [-]
cofounder here - LLMs can do some model training, they train on ML competition data after all. But they do struggle with low signal to noise ratio of market data. But that’s exactly what our environments will teach.
ak_111 1 days ago [-]
if the data is not synthetic, how do you ensure that the LLM hasn't learnt about this data for example from training on the Financial Times.
Mzzzzz 1 days ago [-]
We do a 2 step anonymisation:
1. Mask all symbols, timestamps etc. So the agents cannot infer the assets/time periods.
2. Mathematically transform numerical values and returns. E.g. the market return targets are not the raw market returns, but neutralised and manipulated. So even the agents have certain bullish/bearish biases, it cannot make use of it, as we use the transformed values.
ah i thought so, interesting. I think the challenge is to do 2 while still keeping it realistic, which actually gets very close to synthetic data generation.
RuiWang0811 24 hours ago [-]
we do affine transformations of the data, so all return/ pnl measures are still the same as with untransformed data. The transformation doesn’t change the conditional distribution of the data, which is what alphas ultimately measure
hmokiguess 24 hours ago [-]
One thing I always think about whenever someone talks about solving investment is "and then what?"
Say there's a crystal ball, wouldn't everyone use such crystal ball? Wouldn't crystal ball become illegal? Wouldn't crystal ball nullify the effects of things?
What am I missing, can someone from this field educate me on how this stuff scales?
11 hours ago [-]
RuiWang0811 23 hours ago [-]
this seems to be a common misconception, our envs use market data, but the goal is not (only) trading. Market data just happens to be a good source of hard data science tasks.
Re trading: I’d argue there is no such thing as solving investment nor is there “the one profitable strategy”. Every decision from personal risk appetite to trading horizon changes what is the optimal strategy for you and there are multiple strategies that make money.
Also note that even the most profitable alphas are no crystal balls. Someone else mentioned 5% correlation to future return - depending on horizon and data such level of correlation can make 9 figure PnL and is by no means easy to achieve
hmokiguess 23 hours ago [-]
What's the margins that makes this worth chasing then? That's the part I maybe don't quite understand, why would you pour a lot of money and resources into something that is stochastic at best?
Mzzzzz 23 hours ago [-]
Quant trading is an extremely high margin business itself. Quant shops are printing billions and have on average much higher profits per employee than tech companies. So it is definitely a business worth doing.
On the other hand, you could also view quant research as some very hard research problems, so training LLMs on these problems could also enhance their general research capabilities.
hmokiguess 8 hours ago [-]
Right, that's quant trading, that's not this business. Enhancing their research seems like they are your customer and you would be eating from their margin.
How would you sell to them, you would deliver what? Faster time to decision? Wouldn't quants be your competition, given their work is building tools like this one?
RuiWang0811 6 hours ago [-]
No we sell our own research to AI labs as RL envs. Realistic RL envs grows in demand as labs seek better data train better models.
It’s a complimentary business. Simply put: we sell envs to labs, labs make better models, firms buy these models to make more profit. Everybody wins.
Yeah a competitor for us would be fellow quants doing the same thing. But even then every quant firm trades differently (and good ones all make money) so envs can still be sufficiently different.
hmokiguess 5 hours ago [-]
So what's an example today of "envs" like yours that these firms buy and how you differ from them? Why you?
RuiWang0811 4 hours ago [-]
there are loads of env businesses for coding tasks, enterprise tasks, computer use etc etc. we offer different envs from a niche industry, which just so happens to be a very hard data science task & where the data doesn't saturate. In order to build these you'd need niche expert knowledge.
I looked through the transcript/output of the model/run linked but didn't find anything that showed much, if any, alpha. Maybe I missed it?
Mzzzzz 23 hours ago [-]
It is the raw trace, so it is the most complete records but hard for human to read. We showcased some features they found in this research blog post: https://edotenv.com/blog/alpha-autoresearch
RuiWang0811 24 hours ago [-]
not sure about your background, the trace shows the feature engineering the LLMs did
jjallen 5 hours ago [-]
The comment in the parent said "kind alphas the agent found". It linked to a job then a trial. All of the comments I read there except for one said things like "The final alpha check is still fitting; the prior lagged backtest’s exact zero metrics make clear that model was not acceptable economically". One said "successful backtest’s" but did not expound on what that means or anything. I was looking for the alpha it found and not the feature engineering.
My backgroud is in finance but do not look at things so quantitatively.
Here is another comment/output: "correlation −0.007 and directional accuracy 0.498". Wouldn't directional accuracy have to be greater than .50 to be profitable?
Forgive my ignorance I am interested in this though. I have asked Claude/ChatGPT to show me where the alpha is that the model found as well so I can learn.
jjallen 5 hours ago [-]
Spent more time looking at this because I was still interested in the alpha it found and found this:
Oh my Current Thing. This this enough current things?
RuiWang0811 21 hours ago [-]
languagelearner, I think you need to spend more time learning languages
modgate 17 hours ago [-]
Strong agree that static evals saturate — the decay property of markets is the genuinely useful part: historically, quant alpha decays on the order of 30-50% per year as capital crowds in, so a live market eval is self-difficultating, exactly what model comparison needs once benchmarks plateau. The hard part I'd flag is comparability: market paths are stochastic, so two runs of the same model can land on wildly different difficulty depending on the realized path — without controlling for that, the eval measures luck more than the model. Options that work: fixed-seed regime paths with resampled baselines, or bootstrapped difficulty metrics (percentile of PnL against a distribution of random strategies) instead of raw return. The second hard part is reward shaping for RL: sparse PnL rewards over multi-day horizons give terrible exploration, so you'll likely need shaped intermediate rewards (execution quality, information state) to get gradients flowing at all. Also worth publishing: variance across seeds for each model — that's the number that tells users whether a 2% delta is signal or noise. Are you planning to ship fixed seed regimes, or is path stochasticity part of the point?
RuiWang0811 5 hours ago [-]
We use real historical market data for the environments. There is no parametric modelling involved.
The decay property refers to alpha that we give the agent for trade in the env - they are generated as tools.
feelingsonice 23 hours ago [-]
I'm not fully clear on this. Is this a quant trading benchmark for LLMs or a RL env?
Mzzzzz 23 hours ago [-]
It is both. We can use the same setup for both RL and Benchmarking.
feelingsonice 23 hours ago [-]
Is it STRICTLY for LLMs & quant research or does it do generic trading simulation?
Curious as to whether you guys have shown that post-training has actually improved performance of agents in your environments.
RuiWang0811 5 hours ago [-]
We have experiments showing that agents at least can learn from the environment by overfitting on train. But we do not yet have full post train runs, mainly due to time. But follow our blog/X where we’ll regularly update our research
Rendered at 21:01:53 GMT+0000 (Coordinated Universal Time) with Vercel.
I can't justify spending $1k - $2k a month for real time options data and compute for what is in my 401k. I guess the question I have is would your tool help me trade these strategies and more important test them on a few hundred a month?
[0] https://github.com/adam-s/alphadidactic
I am asking because I see the agents can do websearch, so how is Alphadidatic preventing the agents from just looking at the asset's performances and cheat?
Back to our product. It is not 2C yet. So we are building the RL Envrionments for finetuning LLM, not providing a personal trading agent. However we can also build a professional trading harness using the setup we have. If you are interested, let's keep in touch: michael.zhang@edotenv.com
In addition, we did not observe such behaviour in our traces. An example: https://hub.harborframework.com/jobs/af0299f9-a3bb-44ea-8ced...
Say there's a crystal ball, wouldn't everyone use such crystal ball? Wouldn't crystal ball become illegal? Wouldn't crystal ball nullify the effects of things?
What am I missing, can someone from this field educate me on how this stuff scales?
Re trading: I’d argue there is no such thing as solving investment nor is there “the one profitable strategy”. Every decision from personal risk appetite to trading horizon changes what is the optimal strategy for you and there are multiple strategies that make money.
Also note that even the most profitable alphas are no crystal balls. Someone else mentioned 5% correlation to future return - depending on horizon and data such level of correlation can make 9 figure PnL and is by no means easy to achieve
On the other hand, you could also view quant research as some very hard research problems, so training LLMs on these problems could also enhance their general research capabilities.
How would you sell to them, you would deliver what? Faster time to decision? Wouldn't quants be your competition, given their work is building tools like this one?
It’s a complimentary business. Simply put: we sell envs to labs, labs make better models, firms buy these models to make more profit. Everybody wins.
Yeah a competitor for us would be fellow quants doing the same thing. But even then every quant firm trades differently (and good ones all make money) so envs can still be sufficiently different.
My backgroud is in finance but do not look at things so quantitatively.
Here is another comment/output: "correlation −0.007 and directional accuracy 0.498". Wouldn't directional accuracy have to be greater than .50 to be profitable?
Forgive my ignorance I am interested in this though. I have asked Claude/ChatGPT to show me where the alpha is that the model found as well so I can learn.
"cumulative_after_cost_return": -0.004003033519454746"
https://hub.harborframework.com/jobs/af0299f9-a3bb-44ea-8ced...
¯\(ツ)/¯
Oh my Current Thing. This this enough current things?
The decay property refers to alpha that we give the agent for trade in the env - they are generated as tools.
For now it is only facing LLM/Agent.