I feel this is not the trajectory we want to go with automation. Perhaps better checks and balances within the automation, or "thresholds" that trip breakers would be a good approach?
Numerous prompting will cause prompt fatigue, similar to pressing yes on a dialog boxes.
LLM, like fire is a powerful tool. Some people play with fire and achieved great things, some play with fire and got burned. A number of them achieved great things and got burned. We need to understand that and learn from our mistakes.
Maybe it's just me, but if I had to approve each command of the agent, that'd remove 90% of the benefits of using an agent in the first place. Almost the whole point is that I can fire off a prompt, it can do whatever and then I come back later.
Instead, wrap the agent in a way so it cannot destroy stuff in the first place. And if you still want it to "be able to destroy databases in production", do so by copy-pasting stuff out of the isolated environment. I've run codex as root, as "dangerously as possible" with zero approvals, since the launch of the TUI, and never hit a snag, because the agent literally don't have access to snag things up.
Agents WILL make mistakes, it's up to you to set things up in a way that you don't get utterly fucked when that eventually happens. Avoiding adding 10s of MCPs tools, avoiding authenticating with all platforms, services and databases and not giving it access to all directories on your computer solves 99% of the issues people are having, and there are numerous of simple ways of doing this.
I feel this is not the trajectory we want to go with automation. Perhaps better checks and balances within the automation, or "thresholds" that trip breakers would be a good approach?
Numerous prompting will cause prompt fatigue, similar to pressing yes on a dialog boxes.
LLM, like fire is a powerful tool. Some people play with fire and achieved great things, some play with fire and got burned. A number of them achieved great things and got burned. We need to understand that and learn from our mistakes.
Maybe it's just me, but if I had to approve each command of the agent, that'd remove 90% of the benefits of using an agent in the first place. Almost the whole point is that I can fire off a prompt, it can do whatever and then I come back later.
Instead, wrap the agent in a way so it cannot destroy stuff in the first place. And if you still want it to "be able to destroy databases in production", do so by copy-pasting stuff out of the isolated environment. I've run codex as root, as "dangerously as possible" with zero approvals, since the launch of the TUI, and never hit a snag, because the agent literally don't have access to snag things up.
Agents WILL make mistakes, it's up to you to set things up in a way that you don't get utterly fucked when that eventually happens. Avoiding adding 10s of MCPs tools, avoiding authenticating with all platforms, services and databases and not giving it access to all directories on your computer solves 99% of the issues people are having, and there are numerous of simple ways of doing this.