Last week we sat down for an internal roundtable on AI with one of our client’s CTO and their VP, looking at it not just from a technology point of view but a societal one. No stage, no audience, just our engineers and their leadership around one table, asking each other the questions that don’t have easy answers. The conversation that followed was candid, occasionally heated, and full of the kind of honesty you rarely get in a formal setting.
Alongside the day-to-day rhythm of our collaboration, we wanted to carve out space for a different kind of conversation: open-ended, with no agenda beyond a list of hard questions. What follows is the result of smart people encouraged to say “I don’t know” out loud.
Here are some of the ideas that stayed with us.
Nobody wants to sign a long contract anymore
One of the first things that came up wasn’t about models at all. It was about commitment. Companies used to sign five- or seven-year software deals without blinking. One participant admitted that right now, even five to seven months feels uncomfortable. When the landscape shifts meaningfully every few weeks, locking yourself into anything starts to feel like a liability.
There was a concrete example behind that discomfort. The first prototype the team took to market took five or six months to build. In today’s terms, that timeline already feels like it belongs to a different era. When your own build cycles compress that dramatically, why would you commit to someone else’s roadmap for seven months
Interestingly, the companies that had always resisted long-term contracts, more out of culture than foresight, now find themselves accidentally well positioned. What used to look like commitment-phobia turned out to be exactly the right posture for a world where the ground moves every quarter.
That single observation says more about the pace of AI than any benchmark chart.
What companies get wrong: fear
When we asked what most companies get wrong about AI adoption, the answer around the table was surprisingly unanimous: they don’t take enough risks. The failure mode isn’t picking the wrong tool or the wrong vendor. It’s fear. Fear of experimenting, fear of letting AI touch anything that matters, fear of looking foolish if a pilot doesn’t work out.
The irony is that in an environment moving this fast, caution is not the safe option. The companies treating AI as something to study from a distance are quietly falling behind the ones willing to prototype, fail, and try again. The cost of a failed experiment has never been lower: a prototype that once demanded months of engineering time can now be stood up in days, evaluated honestly, and thrown away without regret if it doesn’t deliver. The cost of not experimenting, meanwhile, has never been higher, because your competitors are running those same cheap experiments right now.
None of this means being reckless with production systems or customer data. It means treating experimentation as a discipline rather than a gamble, and accepting that in this market, the biggest risk is refusing to take one.
A new way of building: watch the data, not the code
One participant described how his own workflow has already changed. He’s building prototypes constantly, but he’s honest about it: he’s not really looking at the code anymore. He looks at the data structures and the outputs. If the data flowing out is right, the code did its job. Are there other concerns, like scalability? Sure. But those can be dealt with later. What matters first is whether the inputs produce the right outputs.
That instinct turned into one of the central debates of the evening.
The end of code review as we know it
Last year’s best practice was simple: let AI write the code, have humans review it. Around our table, that idea was already declared obsolete. If a human has to review every line an AI produces, the human becomes the bottleneck, and you can’t move much faster than writing the code yourself. If you really want to be cautious, the group suggested, have another agent review the first agent’s code.
The alternative everyone kept circling back to was what one participant called the “black box” approach: judge the inputs and the outputs, not the middle. The analogy that landed: when you write code today, a compiler translates it into machine instructions. Nobody opens the binaries to review whether the translation went right, because that doesn’t scale. At some point, you trust the tool. And if the output is bad, the problem is probably your input, not the model.
It’s not a perfect analogy, and the room knew it. Compilers are deterministic; models are probabilistic. But the counterargument to the counterargument was just as sharp: models will soon be so good that you won’t be able to match them in programming skill anyway, at which point your input, the clarity of what you ask for, becomes where the value lives.
Testing still matters, of course. One participant described catching a query during acceptance testing that took three seconds when it should have taken thirty milliseconds. The AI came back and admitted it: you’re right, I found an error. But even those checks can increasingly be automated and handled by agents. The one thing that’s genuinely hard to test, someone pointed out, is long-term maintainability. Which raised an uncomfortable follow-up: if AI is doing the maintenance, does maintainability of code even matter anymore?
Do programming languages still matter?
This was where the conversation got provocative. Higher-level programming languages were invented so humans could talk to machines in something closer to English. Now you can actually talk to machines in plain English, and they produce working software. So do frameworks and languages still matter? One camp said no, and pointed out that this is exactly why traditional software companies are scared.
Someone pushed the idea further: today’s models learned the languages we already had, but there’s no rule saying it stays that way. Models could converge on languages that happen to suit them best, or in time develop representations of their own. Thirty years ago you were writing Turbo Pascal or C++ and thinking hard about things nobody thinks about today. The abstraction level has always moved up. This is just the biggest jump yet.
Not everyone agreed, and the disagreement was the best part. The pragmatic counterpoint came from a real story: one participant spent six frustrating iterations watching an AI fumble a drag-and-drop hierarchical UI, until he finally told it to just use a library. It immediately found a well-maintained open-source one with thousands of contributors, and the problem was solved. The lesson: even in an AI-first world, you don’t rebuild what the community has already perfected, and you don’t burn budget rebuilding solved problems from scratch. Knowing when to say “please just use a library” is still an engineering skill, and on that evening it saved the project a lot of iterations.
The nail gun problem
Does the engineer’s role go away? The room said no, and offered the best metaphor of the evening: buying a nail gun doesn’t mean you know how to build a house. AI can write code, but that doesn’t mean it knows how to write your application, or how to scale it. Maybe someday it will, but it’s not there yet. Engineers still have to be in the process. The work just shifts from writing lines to shaping systems, and the engineering fundamentals remain the same even as the coding itself fades away.
The race nobody can look away from
The group also marveled at how quickly the leaderboard itself keeps changing. New models arrive that aren’t incrementally better but categorically better, opening up things that simply didn’t work the month before. Labs that were trailing jump several places in a single release. And the competition isn’t only about raw capability anymore; it’s about cost and speed too, where the gains have been genuinely astonishing. As one participant joked, quoting a certain fictional race car driver: if you’re not first, you’re last.
Underneath the jokes was a serious point about data. Model quality is increasingly a question of creating the right data, which is why coding-focused models have leapt ahead so fast. And there’s a wilder theory on the table: as models get better, they start contributing to their own improvement, and we may not fully grasp the exponential that kicks in when that happens.
A detour through Europe, cookies, and privacy
Because this was a societal conversation and not just a technical one, it eventually drifted somewhere every AI discussion in Europe seems to end up: privacy. It started as a joke about cookies, the kind you click away on every website, the surest sign you’ve landed on the continent. But behind the joke sat a genuinely hard question about what privacy even means in a world where AI systems are trained on and reason over enormous amounts of data.
Nobody at the table had a good answer, and to their credit, nobody pretended to. It was one more entry on the growing list of questions where the honest response is “we don’t know yet,” and it was a useful reminder that the hardest AI questions in the coming years may not be technical at all.
The questions nobody could answer
We ended by asking: what’s a question about AI you don’t have a good answer to yet? Two stood out.
If nobody codes anymore, where do senior engineers come from? Everyone around the table agreed they became good engineers by going through trials of fire: production failures, bugs that wouldn’t compile, hard-won lessons learned at two in the morning. If AI removes those struggles, how does the next generation build judgment?
The debate here was genuinely unresolved. One view: you can’t become a good engineer without knowing how to code, full stop, because you need to understand how things work underneath. The optimistic counter: every generation has said this. Engineers who started decades ago understood layers of the machine that today’s excellent engineers never touch, and it didn’t stop them from being excellent. The next generation won’t have compilation errors and syntax bugs; they’ll have different problems, and they’ll learn from different kinds of failure. Both sides made their case, and the honest consensus at the end was simply: there is no answer here yet.
When does the curve flatten? Every other month there’s a model that isn’t marginally better but shockingly better. That rate of improvement has to plateau at some point and reach some steady state. But when? Nobody knows. And if the self-improvement theory is right, the curve might do something none of us can predict.
The takeaway
If there was one thread running through the evening, it was this: the fundamentals of good engineering, like understanding systems, judging trade-offs, and knowing what “good output” looks like, matter more than ever. What’s changing is everything around them.
For us, the session confirmed something we increasingly believe: the most valuable conversations about AI right now are not the confident ones. They’re the ones where experienced people compare notes on what they don’t know. And for companies watching from the sidelines, the message was even simpler: the biggest risk right now is being too afraid to take one.
