GitHub Copilot turning five should have been a victory lap for AI-assisted development. In many ways, it still is. Whatever your view on the tooling, Copilot helped prove that AI could become part of normal engineering work.
However, the anniversary has also exposed a more uncomfortable reality. Recent criticism around Copilot’s reliability and usage-based pricing points to a bigger issue many engineering teams are now running into. AI has made it much easier to generate code, but the rest of the software delivery lifecycle hasn’t moved at the same pace.
As the surrounding delivery system struggles to cope, testing strategy, security reviews and governance all start to slip. This is where a lot of the engineering leaders I’ve spoken to are starting to hit a wall with autonomous software development.
Faster code is only part of the story
The first wave of AI coding was largely about individual productivity. The next wave will be about agents taking on more of the delivery process itself.
For developers, that has obvious appeal. Nobody wants to spend more time on repetitive delivery tasks than they need to. But this is also where confidence starts to drop.
Most engineering leaders aren’t against autonomy. They can see the value in removing repetitive work from developers and giving teams more space to focus on higher-value engineering. The hesitation comes from an understandable concern about what happens when the agent gets it wrong.
An agent with too much freedom can create real risk. It might make a change in the wrong place, expose sensitive data, approve something it shouldn’t, or leave behind a decision or lines of code that nobody can explain. That’s why, despite all the talk about autonomous software delivery, most engineering teams are still operating in a halfway house. AI can suggest and assist, but humans still have to shepherd most of the work through the system manually.
Autonomy needs boundaries
The answer isn’t to slow down AI adoption, but to make autonomy safe enough to trust.
That starts with scope. Agents shouldn’t roam across the SDLC with broad credentials and vague permissions. They need distinct identities and clear boundaries around what they can read, change, and approve. Engineers should treat them less like clever scripts and more like junior team members with specific responsibilities.
The second point is policy. If a human change needs to pass security, compliance, quality and deployment controls, an agentic change should go through the same gates. The policies don’t need to be reinvented, but they do need to be enforced consistently. That might mean restricting which models can be used in production workflows, blocking certain classes of action, or requiring human approval before an agent can deploy or touch sensitive environments.
Third, agents need to be observable. I wouldn’t be comfortable with an engineer making production changes without a traceable record of what happened, and the same applies for AI. Teams need to know what triggered an agent, what context it used, what it decided, what it changed, what it cost and what the outcome was. Without that audit trail, you only have guesswork.
Finally, autonomy has to fit into the tools developers already use. The more agentic workflows require engineers to jump into separate systems, the less likely they are to become part of normal delivery practices.
The best implementations will meet developers in their existing workflows while still running under central governance, security, and audit controls.
Making autonomous development a reality
For me, that is the real lesson from five years of GitHub Copilot. Writing code faster was only the first step for agentic workflows. The real challenge is whether the rest of the delivery process can keep up with the pace of AI without creating new risk.
What have been your experiences of incorporating agents into the software delivery lifecycle? Do you have any other best practices for making automated software development a reality? Please do add them to the comments below.