The self‑improving part of AgentRun is very interesting to me. In my project https://loopseo.org I faced difficulties, in keeping autonomy while also having controllable behavior. I would like to know what safety and oversight methods you use for this situation.
brendanator 9 hours ago [-]
Sourcery CEO here. Wanted to comment on the claims of 1.6x Greptile review comments.
With Greptile there's a precision-recall trade-off with code review comments. Greptile is at the precision end of the scale, so every comment that they make is accurate. We've seen on GitHub that over 80% of the comments they make get addressed. At the other end of the spectrum is Sourcery [1], which produces over 2x as many comments as Greptile, and this agent run example has 1.6x.
Shameless self-promotion: we believe that outputting a lot of comments, even if they're nitpicky, is the right way to go for coding agents because they can triage all of the comments very rapidly and decide which to address and which are not relevant.
I have done this with my own agent frameworks. They have their own issue trackers and record notes there, so the next agent to work an issue can pick up and go. In addition to providing the specific context for individual tasks, the notes then get swept into per-project libraries that distill the information for future planning.
If it is not written down, the agents can’t know it, so the rule is to write everything down. File an issue before starting work, update with a note at every step. One task per issue. You know… the stuff we would do for each other, if we all didn’t hate Jira and its ilk so very very much.
stronglikedan 6 hours ago [-]
> we believe that outputting a lot of comments, even if they're nitpicky, is the right way to go for coding agents
Is this what they mean by "tokenmaxxing"?
jo_in_europe 9 hours ago [-]
How is this different to a simple harness? For example, I have a harness, made in 1.5 days, that has an executive agent answering the "what done looks like" task, then chunking the work, creating a dependency graph, and assigning it to sub-agents (which can be recursive, ie. the sub-agent becomes an executive for its own workers). And there's an independent "reviewer" for each work chunk, before potentially merging and kicking off any tasks whose dependencies are satisfied. Interfaces with remote hosts, GPU machines from different providers. So what would your thing give me that mine can't do?
How do you tell when production runs start leaving the compiled workflow?
Kubit CEO here. That's the reason why we visually group runs by execution path after labeling every span, so the new branches will popup quickly.
whirlwin 1 days ago [-]
Would like to hear from everyone: How are you dealing with PRs in 2026? It's our biggest bottle neck.
We are considering looking into attestations, e.g. through the SLSA spec: https://slsa.dev/ to ensure build provenance and verification before it is deployed.
philipp-gayret 21 hours ago [-]
For my own projects: Agents have access to managed tools which enforce quality metrics, build, test, and deploy all more or less in parallel and get it ready to go into git. Examples of metrics are: test coverage, allowed dependencies, linters and the usual. These tools run on the same machine where the software will be deployed (for performance reasons). These tools submit attestations for everything they do. I made my own system for it, it keeps track of everything. The whole process including docker build and deploy to a test env is 10s or less. (I find 10s is reasonable to achieve for all projects I've worked on). Secondly I run a daily process for all projects that runs another set of quality metrics ; picture for example dependencies out of date, tests getting too slow and similar topics. These become work for the agents to pick up. Review is me checking if the app does what it should at best. If it passes all the quality metrics, I don't care much for the code.
For clients (who are not that AI-pilled): They're liking CodeRabbit and developers are not reviewing as strictly anymore. Quality metrics are obtained via CI/CD and attached to PR's. If it's all green then most developers would give it a LGTM
killix 18 hours ago [-]
[flagged]
jcarver 21 hours ago [-]
You have two options:
1. Agentic merge queue. Lower risk PRs enter a queue and are auto-merged by a specialized agent. Higher risk PRs are still reviewed by humans. We're currently doing this for Dependabot PRs via a product we built: https://patchwave.ai/ , but the approach generalizes to any kind of PR.
2. Stop doing PRs. Review moves upstream into a multiplayer agent thread that's visible to the team - tag a teammate to pop in and look when you need feedback. PR checks move upstream into this thread as well, the agent just runs them as part of verifying its work. Push to main, ci checks run and deploy as normal from there.
We're at 1 now, but actively working towards moving to 2 and want to help other teams do the same.
DeepOntology 5 hours ago [-]
My own personal recipe:
1. No pull requests
2. I have separate strategy, code-writing, and commit agents
3. A reader that never saw the builder's reasoning checks each fix first
4. Full test run must be green on the exact commit
5. Verification / feature walk on staging
On the app I'm building (and its platform dependencies) I've made it through ~1200 commits and ~300k LOC without more than an occasional trivial regression/having to roll back... so, I think my strategy is working.
biggestriverman 16 hours ago [-]
What I do is while tasks are being assigned to agents I work the the manager agent to ensure the tasks are correct and actually correspond to the goal of the project. The manager agent reaches out with blocking questions it cannot figure out itself. Then for review pre-release the code is verified against the assigned task, checked for bugs and security risks. Once all found issues are fixed it goes out.
If your team is using subscriptions, it is a lot more cost effective (and higher quality) to simply use a local orchestrator to automatically deal with PR reviews. We built https://github.com/codealong/codealong in part to help with this.
miguelrios 1 days ago [-]
very cool. thanks for sharing!
mfrye0 1 days ago [-]
Very cool. I was waiting for something like this to pop up.
Is it possible to be added as a data source? I see you're mostly pointing the agent to easy / publicly available sources like the UK.
We have ~200 government registrar integrations now and 500M entities in the system. We essentially have better coverage than OpenCorporates at this stage. https://savvyiq.ai
miguelrios 1 days ago [-]
Do you have an MCP? I am adding our internal tool proxy which can be used to plug in any MCP into it.
mfrye0 1 days ago [-]
Yeah, but it's out of date. About to replace it with a new version that has SSO support and other updated APIs. Will be available next week.
bmiedlar 22 hours ago [-]
I've noticed on some of my own research and testing that even with caps some agents will work around the per-call time limit using background processes. Do your caps cover compute in the detached processes?
miguelrios 21 hours ago [-]
uh, interesting. I will take a look into this.
anatolii1309 13 hours ago [-]
[flagged]
Rinkesh2010rpp 1 days ago [-]
Cool project, I myself have a similar autonomous agent, in python
At rinkesh2010rpp/sudarshana. But I liked your mission control. Will for sure take some inspiration.
Also, what I have is ability to find other agents on run time, running on Modal, and a safety layer that checks every actions taken by the agent.
and happy to talk more.
Thanks
miguelrios 1 days ago [-]
Very cool. Feel free to borrow from us what's helpful and send feedback our way!
Rinkesh2010rpp 19 hours ago [-]
Thats what I am gonna ask my claude code to do lol.
The core concept is to have a frontier or very smart model run as an agent first for N times. The model get tools, skills etc and instructions. It does the job. At the end it leaves a note (like a retro) explaining what it did and what it could have done better.
As the agent runs, it also writes workflows to do the parts of the job it knows it can do in a more deterministic way (based on previous nodes and thinking traces of other runs etc). We made a dsl for this that has routing, classification, code nodes, etc to build these workflows.
For repetitive tasks (which is our focus) we find that after ~100 cases, the agent most times just runs a workflow end to end instead of looping towards a solution. Workflows are cheap, so the job ends up being way cheaper than an agent in a good model.
K0balt 13 hours ago [-]
Interesting, this is basically what happens organically in my “office” agentic structure (just a bunch of documents that govern an agentic team with defined roles in Claude code). They end up writing python tools to do 90 percent of the repetitive tasks. I also give them access to A farm of specialised local models to use as well as some reasonably capable small LLMs.
I am publishing a library of templates tomorrow with many more.
jstummbillig 1 days ago [-]
I think I'll need like at least another 3-4 months of new models before I can deal with AI UI/UX design.
embedding-shape 1 days ago [-]
It hasn't gotten better since a year ago, I don't have high hopes. People need to get better at making smaller decisions and having more opinions, not just prompt "Create page for dashboard".
miguelrios 24 hours ago [-]
This is fair feedback. We spent a ton of time in the UI for our hosted product and the main website (agentrun.ai) but very little in the OSS console. We’re spending more time on this now and making it better.
With Greptile there's a precision-recall trade-off with code review comments. Greptile is at the precision end of the scale, so every comment that they make is accurate. We've seen on GitHub that over 80% of the comments they make get addressed. At the other end of the spectrum is Sourcery [1], which produces over 2x as many comments as Greptile, and this agent run example has 1.6x.
Shameless self-promotion: we believe that outputting a lot of comments, even if they're nitpicky, is the right way to go for coding agents because they can triage all of the comments very rapidly and decide which to address and which are not relevant.
[1] https://sourcery.ai
If it is not written down, the agents can’t know it, so the rule is to write everything down. File an issue before starting work, update with a note at every step. One task per issue. You know… the stuff we would do for each other, if we all didn’t hate Jira and its ilk so very very much.
Is this what they mean by "tokenmaxxing"?
https://knowhow.ml/harness1.png
https://knowhow.ml/harness2.png
https://knowhow.ml/harness3.png
Kubit CEO here. That's the reason why we visually group runs by execution path after labeling every span, so the new branches will popup quickly.
We are considering looking into attestations, e.g. through the SLSA spec: https://slsa.dev/ to ensure build provenance and verification before it is deployed.
For clients (who are not that AI-pilled): They're liking CodeRabbit and developers are not reviewing as strictly anymore. Quality metrics are obtained via CI/CD and attached to PR's. If it's all green then most developers would give it a LGTM
1. Agentic merge queue. Lower risk PRs enter a queue and are auto-merged by a specialized agent. Higher risk PRs are still reviewed by humans. We're currently doing this for Dependabot PRs via a product we built: https://patchwave.ai/ , but the approach generalizes to any kind of PR.
2. Stop doing PRs. Review moves upstream into a multiplayer agent thread that's visible to the team - tag a teammate to pop in and look when you need feedback. PR checks move upstream into this thread as well, the agent just runs them as part of verifying its work. Push to main, ci checks run and deploy as normal from there.
We're at 1 now, but actively working towards moving to 2 and want to help other teams do the same.
1. No pull requests 2. I have separate strategy, code-writing, and commit agents 3. A reader that never saw the builder's reasoning checks each fix first 4. Full test run must be green on the exact commit 5. Verification / feature walk on staging
On the app I'm building (and its platform dependencies) I've made it through ~1200 commits and ~300k LOC without more than an occasional trivial regression/having to roll back... so, I think my strategy is working.
Is it possible to be added as a data source? I see you're mostly pointing the agent to easy / publicly available sources like the UK.
We have ~200 government registrar integrations now and 500M entities in the system. We essentially have better coverage than OpenCorporates at this stage. https://savvyiq.ai
Also, what I have is ability to find other agents on run time, running on Modal, and a safety layer that checks every actions taken by the agent.
and happy to talk more.
Thanks
If you are interested, check out The agent: https://github.com/rinkesh2010rpp/sudarshana The A2a Hotplug which gives agents ability to add a2a agent in runtime https://github.com/rinkesh2010rpp/a2a-hotplug Security layer for your agent https://github.com/rinkesh2010rpp/taster-ai
The core concept is to have a frontier or very smart model run as an agent first for N times. The model get tools, skills etc and instructions. It does the job. At the end it leaves a note (like a retro) explaining what it did and what it could have done better.
As the agent runs, it also writes workflows to do the parts of the job it knows it can do in a more deterministic way (based on previous nodes and thinking traces of other runs etc). We made a dsl for this that has routing, classification, code nodes, etc to build these workflows.
For repetitive tasks (which is our focus) we find that after ~100 cases, the agent most times just runs a workflow end to end instead of looping towards a solution. Workflows are cheap, so the job ends up being way cheaper than an agent in a good model.
https://agentrun.ai/docs/self-improvement
https://agentrun.ai/docs/learning-reference
Is it more than an ACP connection to other agents?
Do the agents have to exist already created in the harness or can you make agents inside AgentRun?
I am publishing a library of templates tomorrow with many more.
You can read more about how we turn agents into workflows here and examples of using Parallel's decisions API here: https://agentrun.ai/blog/you-may-not-need-an-agent-after-all