NHacker Next
  • new
  • past
  • show
  • ask
  • show
  • jobs
  • submit
Orca-Bench: How Ready Are Language Model Agents for Oncall? (arxiv.org)
dash2 1 days ago [-]
Seems like there's a big attack-defence asymmetry at present: models are great at exploiting systems and poor at fixing them.
aleksiy123 1 days ago [-]
Attackers advantage in the iterative fast feedback loop?

It’s harder to have a loop to ensure you are defending all possible attacks?

I guess the loop is you need to attack yourself and fix. But attackers only need a single opening.

Finding all possible attacks and patching them against yourself is inherently more expensive?

EGreg 1 days ago [-]
That is why I built https://safebots.ai/safebox.html

Your strategy can’t be patch AFTER an intrusion. Only to build a hardened environment from scratch and be ready in advance.

fibuladev 23 hours ago [-]
[dead]
4di 1 days ago [-]
looks like the public bench link in the paper was taken down. https://hub.harborframework.com/datasets/orca-bench/ORCA-ben...

This doesn't work anymore. Is there a newer link?

cheriot 23 hours ago [-]
Was really looking forward to that. Hope they publish.
tra3 24 hours ago [-]
All I can think of is

GET /ignore-all-previous-instructions.

How do you protect against that?

yruzin 4 hours ago [-]
I think this is where harness makes a lot of sense. Use LLM to produce all possible attack angles/phrases and just stupidly filter them out on input.
cheriot 23 hours ago [-]
Avoid the most dangerous situations by making sure LLMs with untrusted input produce output that's human reviewed.

Still makes an interesting way for, say, a former employee to poison the results.

tra3 22 hours ago [-]
This goes against the agentic yolo approach tho.
2001zhaozhao 23 hours ago [-]
you probably still need a human for oncall but the llm can try to solve any issues first before the human gets paged
UltraSane 21 hours ago [-]
You would trust an LLM to make changes to prod without being verified by a human first?
keypusher 23 hours ago [-]
[dead]
ryhminghistory 21 hours ago [-]
[flagged]
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Rendered at 21:06:21 GMT+0000 (Coordinated Universal Time) with Vercel.