How are you all different than the other few CUA APIs in this batch and previous batches?
nkov47 1 days ago [-]
We don't just provide end-to-end API, we're a SOTA harness where you can bring in your own OpenAI or Claude keys and run it on and also we're the only modular API in the market - we give you the ability to control any part of CUA and charge by the type of call as you can see in https://coasty.ai/docs.
throw03172019 24 hours ago [-]
That’s really nice actually. Do the screenshots and playback and logs get stored on our systems as well?
nkov47 23 hours ago [-]
Yes, you have access to all your logs and screenshots and playback!
throw03172019 21 hours ago [-]
Sure but is that stored on our infra or yours? (Work in HIPAA regulated industry)
nkov47 21 hours ago [-]
We're SOC2 and HIPAA compliant and have zero data retention policies through our enterprise platform where it's all covered and we don't keep any data that you don't want us to!
owebmaster 12 hours ago [-]
What do you think that makes your solution "SOTA"? That's quite the interesting claim, which is obviously false.
nkov47 3 hours ago [-]
We've hit 82.8% on OSWorld Verfied(on the official page) and with our latest internal testing we're hitting 85.6% which we've posted (https://github.com/coasty-ai/coasty-osworld).
nkov47 1 days ago [-]
With our API, you can also create a very customizable blend of deterministic workflows and AI calls for CUA, so for example, Coasty for recovery can kick in when something unexpected like a dialog box or popup happens during your deterministic workflow runs.
localplugins 12 hours ago [-]
The most interesting thing about this is buried in a reply rather than the post: the ability to blend deterministic workflows with AI calls, so the agent kicks in on recovery when a dialog or popup breaks the scripted path.
That's your actual differentiator and I'd lead with it. Right now the post frames it as observe-decide-execute-observe, which reads as "another CUA agent," and the top comment is immediately asking how you differ from the other CUA APIs in the batch. But deterministic-by-default with AI on exception is a genuinely different shape from both of the alternatives you name. RPA breaks when the screen changes; a pure agent is nondeterministic on the 95% of steps that never needed judgment in the first place. Doing the boring path deterministically and paying for a model only when reality deviates is the thing that would make me try this.
I came at the same conclusion from a completely unrelated domain (I build asset generators, no computer-use anywhere near it) and the tool only got good the day I stopped letting the model do the parts that had to be identical every run. Generation is sampling. Same prompt, same model, different run, different result. The skill is knowing which steps genuinely need judgment and refusing to spend a sampling operation on the ones that don't. Sounds like you've built exactly that control surface and then under-sold it.
Genuine question on the handoff, since that's where I'd expect this to get hard: when the deterministic path hits something unexpected and the agent takes over for recovery, how does it hand control back? Does it have to return the UI to a known state that the script recognizes, or does the script resume wherever the agent left off? That reconciliation seems like the interesting engineering, and it's also where I'd expect the failure modes to live.
nkov47 3 hours ago [-]
This is a great read of it, and honestly we probably did undersell that part.
The handoff works through checkpoints. The deterministic workflow defines the state it expects before resuming, and the agent’s recovery job is to get the UI back into one of those known states, not just “fix whatever happened” and keep going.
If it can reconcile back to a checkpoint, the script resumes deterministically from there. If it can’t, the run pauses rather than pretending it knows where it is.
That reconciliation layer is definitely where a lot of the hard engineering lives.
owebmaster 10 hours ago [-]
Bro make your bot less verbose, this is terrible
jkwang 13 hours ago [-]
The checkpoint and invariant model is a strong fit for these workflows. Having approval gates plus a replayable event log makes the agent's decisions much easier to audit than a simple end-to-end task API.
nkov47 3 hours ago [-]
Exactly. We’ve found that for real production workflows, “did the task finish?” isn’t enough. You need to know what the agent saw, why it acted, what changed, and where a human approved something consequential.
The checkpoints and event log are really about making failures inspectable instead of mysterious.
StarPA 9 hours ago [-]
Congrats on the launch bro.
I use computer-use agents daily as an end user (browser automation, even wireless ADB to install builds on my phone), so genuine question: how does your API handle the diffrence between reversible and ireversible actions ?
Clicking around a page is one thing, but submitting a form, sending a message or confirming a payment is another — is there a mechanism for the agent to pause and hand back to the human before those, or is that left entirely to the caller?
Asking cause in my exp that boundary is where trust in these agents is won or lost.
nkov47 3 hours ago [-]
Thanks!!! completely agree that this is where trust is won or lost.
We support explicit human approval gates in workflows, so you can have the agent prepare everything, then pause before steps like submitting a form, sending a message, or confirming a payment. The workflow moves to `awaiting_human`, and the caller approves or rejects it before execution continues.
Autonomous runs can also pause and hand over the live machine when the agent encounters something requiring human input. For teams that want stricter control, our lower-level API exposes each predicted action so the caller can inspect or approve actions before executing them.
Today, the irreversible boundary is primarily defined by the developer rather than us trying to universally infer what is consequential. We think explicit policy is safer because “irreversible” varies a lot by application.
15 hours ago [-]
owebmaster 12 hours ago [-]
It's so funny to see YC back tens of generic similar low quality projects
nkov47 3 hours ago [-]
I mean we've spent a year of our life on this(ironing out infra for more than half of the time), so i wouldn't say super low quality but again YC accepted us based on how our business is growing! Thanks for the feedback though!
sneefle 11 hours ago [-]
we run screen-driven agents against web forms in production and the failure mode that took us longest to find wasn't navigation, it was commits that don't commit. a react controlled select can render the right value after a click while the framework's internal state never updated, so every pixel says done and the submitted payload says null. vision-only verification passes because the screen genuinely looks correct.
curious how you handle that class without DOM access. screenshot-after-action catches missing UI feedback, but when the UI itself is lying about form state the only reliable tells we found were downstream: the confirmation page, an outbound request, an email arriving. do your verification events ever consume anything besides pixels, or do you lean on the human approval gates for the risky commits?
ibrobaba 2 hours ago [-]
The invariant we use is the downstream outcome, not the visual state. For a controlled select, that can be the request payload, confirmation state, or persisted record. I’m building AnnotateQA around capturing a real browser failure and creating that failing Playwright reproduction before any code changes, with a candidate PR left for human review. If you have a recent case, I’d be glad to try one.
madikz 9 hours ago [-]
Great observation on the "commits that don't commit" failure mode. We ran into the exact same class of bug building document extraction for regulated financial workflows — turns out vision-only verification has a blind spot that shows up whether you're parsing a screen or a PDF.
In tax preparation, documents have a lot of "the screen lies" equivalents: a PDF renderer can display perfectly aligned columns while the underlying text layer has misaligned fields. We learned this the hard way when an OCR pipeline extracted "123 Main St" beautifully from a W-2 image, but the XML metadata had a completely different address. The screen looked right, the data was wrong.
Our approach was a second verification layer structurally independent from extraction: deterministic rules that reconcile AI output against the source document's known structure. For a W-2, that means cross-walking every box number against the IRS's published schema and flagging deviations — same principle as your "downstream tells" but formalized into a rules engine.
The key insight: in regulated domains, the verification layer must consume something structurally different from what extraction consumed. If both use the same pixel-based representation, they share the same failure modes. Using DOM state, API call logs, or document metadata schemas as the verification source catches the cases where "pixel says done, payload says null."
Defining explicit "verification invariants" (similar to Coasty's approach) has been the most practical defense — things like "total across all line items must equal reported total" or "taxpayer name must match across all documents in the return." Simple arithmetic checks that don't need AI but catch the expensive failure modes that vision alone misses.
Curious how Coasty handles multi-source reconciliation — e.g., when the data on screen needs to be verified against a separate document (like a PDF guidance doc) rather than just against itself?
nkov47 3 hours ago [-]
Yep, this is a real failure mode, and screenshot-after-action alone does not solve it. A field can look populated while the application never commits the underlying value.
We try to define verification around the actual outcome of the task rather than the immediately preceding UI state. Depending on the workflow, that can mean checking the confirmation page, reopening the submitted record, verifying a status change elsewhere in the application, matching a generated reference ID, or confirming that a downstream artifact was created.
We can also verify against separate inputs such as a PDF, CSV, or another screen for example, confirming that the values entered into a portal match the source document and then checking the resulting record after submission.
Today, our core agent is screen-driven, so we do not claim that pixels can prove every hidden state transition. For high-consequence commits, the workflow should use explicit downstream invariants and, where those are unavailable, a human approval or review step. Longer term, we think verification should be able to consume whatever independent evidence the environment exposes, including network or application-level signals, rather than treating vision as the only source of truth but it will be quite slower.
jyswee 23 hours ago [-]
[flagged]
Talordata29 15 hours ago [-]
[dead]
dnkzm 1 days ago [-]
[dead]
Rendered at 20:38:36 GMT+0000 (Coordinated Universal Time) with Vercel.
That's your actual differentiator and I'd lead with it. Right now the post frames it as observe-decide-execute-observe, which reads as "another CUA agent," and the top comment is immediately asking how you differ from the other CUA APIs in the batch. But deterministic-by-default with AI on exception is a genuinely different shape from both of the alternatives you name. RPA breaks when the screen changes; a pure agent is nondeterministic on the 95% of steps that never needed judgment in the first place. Doing the boring path deterministically and paying for a model only when reality deviates is the thing that would make me try this.
I came at the same conclusion from a completely unrelated domain (I build asset generators, no computer-use anywhere near it) and the tool only got good the day I stopped letting the model do the parts that had to be identical every run. Generation is sampling. Same prompt, same model, different run, different result. The skill is knowing which steps genuinely need judgment and refusing to spend a sampling operation on the ones that don't. Sounds like you've built exactly that control surface and then under-sold it.
Genuine question on the handoff, since that's where I'd expect this to get hard: when the deterministic path hits something unexpected and the agent takes over for recovery, how does it hand control back? Does it have to return the UI to a known state that the script recognizes, or does the script resume wherever the agent left off? That reconciliation seems like the interesting engineering, and it's also where I'd expect the failure modes to live.
The handoff works through checkpoints. The deterministic workflow defines the state it expects before resuming, and the agent’s recovery job is to get the UI back into one of those known states, not just “fix whatever happened” and keep going.
If it can reconcile back to a checkpoint, the script resumes deterministically from there. If it can’t, the run pauses rather than pretending it knows where it is.
That reconciliation layer is definitely where a lot of the hard engineering lives.
The checkpoints and event log are really about making failures inspectable instead of mysterious.
I use computer-use agents daily as an end user (browser automation, even wireless ADB to install builds on my phone), so genuine question: how does your API handle the diffrence between reversible and ireversible actions ?
Clicking around a page is one thing, but submitting a form, sending a message or confirming a payment is another — is there a mechanism for the agent to pause and hand back to the human before those, or is that left entirely to the caller?
Asking cause in my exp that boundary is where trust in these agents is won or lost.
We support explicit human approval gates in workflows, so you can have the agent prepare everything, then pause before steps like submitting a form, sending a message, or confirming a payment. The workflow moves to `awaiting_human`, and the caller approves or rejects it before execution continues.
Autonomous runs can also pause and hand over the live machine when the agent encounters something requiring human input. For teams that want stricter control, our lower-level API exposes each predicted action so the caller can inspect or approve actions before executing them.
Today, the irreversible boundary is primarily defined by the developer rather than us trying to universally infer what is consequential. We think explicit policy is safer because “irreversible” varies a lot by application.
curious how you handle that class without DOM access. screenshot-after-action catches missing UI feedback, but when the UI itself is lying about form state the only reliable tells we found were downstream: the confirmation page, an outbound request, an email arriving. do your verification events ever consume anything besides pixels, or do you lean on the human approval gates for the risky commits?
In tax preparation, documents have a lot of "the screen lies" equivalents: a PDF renderer can display perfectly aligned columns while the underlying text layer has misaligned fields. We learned this the hard way when an OCR pipeline extracted "123 Main St" beautifully from a W-2 image, but the XML metadata had a completely different address. The screen looked right, the data was wrong.
Our approach was a second verification layer structurally independent from extraction: deterministic rules that reconcile AI output against the source document's known structure. For a W-2, that means cross-walking every box number against the IRS's published schema and flagging deviations — same principle as your "downstream tells" but formalized into a rules engine.
The key insight: in regulated domains, the verification layer must consume something structurally different from what extraction consumed. If both use the same pixel-based representation, they share the same failure modes. Using DOM state, API call logs, or document metadata schemas as the verification source catches the cases where "pixel says done, payload says null."
Defining explicit "verification invariants" (similar to Coasty's approach) has been the most practical defense — things like "total across all line items must equal reported total" or "taxpayer name must match across all documents in the return." Simple arithmetic checks that don't need AI but catch the expensive failure modes that vision alone misses.
Curious how Coasty handles multi-source reconciliation — e.g., when the data on screen needs to be verified against a separate document (like a PDF guidance doc) rather than just against itself?
We try to define verification around the actual outcome of the task rather than the immediately preceding UI state. Depending on the workflow, that can mean checking the confirmation page, reopening the submitted record, verifying a status change elsewhere in the application, matching a generated reference ID, or confirming that a downstream artifact was created.
We can also verify against separate inputs such as a PDF, CSV, or another screen for example, confirming that the values entered into a portal match the source document and then checking the resulting record after submission.
Today, our core agent is screen-driven, so we do not claim that pixels can prove every hidden state transition. For high-consequence commits, the workflow should use explicit downstream invariants and, where those are unavailable, a human approval or review step. Longer term, we think verification should be able to consume whatever independent evidence the environment exposes, including network or application-level signals, rather than treating vision as the only source of truth but it will be quite slower.