Yeah but this is the thing, that's just text. If I tell someone "you can't post on HN anymore", whether they won't is entirely up to them.
Permissions in context or text are weak, these tools - especially the ones that operate on untrusted input - need to have hard constraints, like no merge permissions.
To be clear - the text I pasted is config for the Github actions workflow, not just part of a prompt being given to a model. The authors seemingly understood that the LLM could be prompt-injected run arbitrary code so put it in a workflow with read-only access to the repo.
Permissions in context or text are weak, these tools - especially the ones that operate on untrusted input - need to have hard constraints, like no merge permissions.