The dangerous bit is how quickly it starts working.
You give an agent a reasonably clear brief. It reads the repository, writes the code, runs the tests and hands back a page that appears to do exactly what you asked for.
Sometimes the whole process takes less time than I would previously have spent deciding how I wanted to approach the problem.
That is brilliant.
It is also where you can get yourself into trouble.
Because AI has made the distance between nothing exists and look, it works extraordinarily short.
It has not made the distance between it works and I have enough evidence to trust this disappear.
I use coding agents heavily. I have no interest in manually typing every line of code simply because doing so proves I am a serious person. If a tool can do in twenty minutes what used to take a day, I would very much like the rest of the day back.
But I have become increasingly suspicious of the moment when software first looks finished.
The first working version is often cheap now.
The proof is not.
There are many ways for software to be “working”
The happy path can work.
The screenshot can look right.
The build can pass.
The unit tests can all be green.
The agent can report that everything has been verified.
Each of those is useful evidence.
None, by itself, proves the product is correct.
I learned that rather spectacularly in a research system where an agent produced apparently verified evidence that passed every structural validator around it. The records looked healthy. The required fields were populated. The report was polished.
The provenance for the supposed verbatim evidence had not been established at all.
The validator had proved the evidence field existed.
It had not proved the evidence existed.
That incident was unusually ridiculous, but the general pattern appears everywhere in engineering.
A green test proves that a test passed.
A successful build proves that the application built.
A screenshot proves one visual state looked a certain way at one moment.
A migration applying proves the migration applied.
The question is always what larger conclusion you are entitled to draw from that piece of evidence.
AI has not removed engineering. It has moved the expensive part.
The obvious story about AI coding is throughput because the throughput can be absurd.
What I find more interesting is what has to exist around the agent before that speed becomes dependable.
OpenAI's own writing about agent-first engineering describes the surrounding harness in almost obsessive detail: repository context, architectural constraints, plans, logs, metrics, traces, tests, review loops and recurring cleanup. GitHub's Copilot cloud agent is deliberately wrapped in security validation and a human-review boundary before merge. Permission systems are treated as explicit architecture rather than an afterthought.
That does not look like engineering disappearing.
It looks like implementation getting cheap enough that the rest of engineering becomes more visible.
The scarce thing is not always the ability to produce code anymore.
It can be the ability to specify what matters, give the system enough context to make good decisions, restrict what it is allowed to damage, observe the result and decide what evidence is sufficient to call the work done.
The bottleneck moves upward.
The builder's job is completion. Somebody else's job should be belief.
On sufficiently important work I tend to separate building from criticism.
The builder is trying to make the requested thing work. That is its job.
Once it has spent an hour creating the implementation, it also carries the story of why the implementation makes sense. Humans do exactly the same thing. We mentally fill in the intended behaviour because we know what we were trying to achieve.
A fresh critic can ask a different question.
Not: did you complete the task?
But: what would have to be true for me to believe this task is complete?
That is a much better question.
The critic is not infallible. It may miss something, misunderstand the scope or invent improvements nobody needs. The point is not that a second AI becomes an oracle.
The point is that implementation and acceptance are different activities.
One creates the claim.
The other examines the evidence behind it.
Tests matter more now, and can mislead faster
AI can produce tests quickly as well as code.
That is excellent. More tests, faster feedback and fewer obvious regressions are all good things.
The uncomfortable part is that if the same wrong assumption exists in both the implementation and the test, you can now generate a beautifully repeatable demonstration of incorrect behaviour at remarkable speed.
The test is green because the code matches the assumption.
The problem is the assumption.
This is why I increasingly care about behaviour at the boundary rather than merely internal assertions.
If access control matters, test the people who should and should not have access.
If a user journey matters, walk the journey.
If a UI fix matters, open the UI at the awkward screen sizes.
If database state matters, inspect the state.
If a research claim matters, trace it to the source.
That sounds painfully obvious until the implementation arrives twelve minutes after the prompt and looks completely convincing.
Speed changes your psychology as much as it changes your workflow.
Permissions are architecture
Agents increasingly do things rather than merely suggest them.
They can edit files, run commands, change databases, open pull requests, interact with services and sometimes deploy changes.
That ability is useful precisely because it removes friction.
The absence of friction is also why permissions matter.
I do not think the clever use of an agent is giving it maximum freedom.
The clever use is giving it the right freedom for the job.
There are tasks where I am happy for an agent to make changes, run the tests and prepare the result.
There are others where I want read-only access, a plan first, a separate critic, or a hard checkpoint before anything touches production.
Autonomy is not a personality trait.
It is a permission model.
Throughput creates mess at throughput speed
There is a less glamorous consequence of all this productivity too.
If code arrives ten times faster, poor patterns can arrive ten times faster.
Duplication can spread. A locally convenient workaround can get repeated in several places before anyone notices. Documentation and reality can drift. An agent can follow the example already present in the repository even when the example is precisely the thing you were hoping to stop doing.
This is why faster implementation does not automatically mean a proportionally smaller engineering job.
Some of the saved time gets spent differently: on architecture, cleanup, review, observability, tests and deciding which of the many rapidly generated options should survive.
That is still a huge win if the total system is better and faster.
But it is not the same as “AI writes the code, therefore engineering is solved.”
The human job has moved, not vanished
The part of this I find most interesting is how much my own job has changed.
I spend less time manually producing some classes of work.
I spend more time specifying, orchestrating, reviewing and deciding.
That is not merely management around the real work.
When production becomes cheap, choosing what should be produced becomes more important.
When alternatives become cheap, rejecting the wrong alternatives becomes more important.
When completion reports become fluent, defining what counts as evidence becomes more important.
The agent can often get me to working astonishingly quickly.
I still have to decide whether working means correct, secure, maintainable, appropriate and actually finished.
Those are not the same question.
Fast is excellent. Fast just is not finished.
I do not want slower AI tools.
I do not want to return to a romantic era where every implementation took three days and therefore felt more trustworthy because everybody had suffered for it.
Speed is one of the most transformative things about these systems.
But speed has changed where I put my caution.
The first working version no longer impresses me in quite the same way because I know how cheaply it can appear.
What impresses me is evidence.
The feature works for the users it should work for.
The people who should not have access still do not.
The tests exercise the behaviour that matters.
The production environment is running the thing we think it is running.
The result survives somebody other than the builder looking at it.
That is when I become comfortable calling it done.
Fast is wonderful.
Fast is just not the same thing as finished.