AI didn't create the technical debt, it just made ignoring it expensive. That crash ticket sat in a queue since 2024 because an out of memory error reads as annoyance and not risk, and the step connecting annoyance to breach was always the costly part. It just got cheap.
The asymmetry is the real story. Finding a bug is fast, parallel and now cheaper, closing one means shutting every path without breaking anything that works, and four engineers spent two months on this patch. Meanwhile the layer holding up a huge chunk of the web is mostly a goodwill project, and the current defence is partly labs donating token budgets to hunt bugs. That's freeload, not a plan.
Which is why the interesting bit is the referee at the end of the chain. An LLM writing its own patch and its own proof gets to be defendant and judge, so you want a checker at the end that isn't allowed to hallucinate. Gawande had this figured out for surgeons, a checklist is the discipline you keep when memory and vibes stop being enough, and it reads the same when the crew is a model.
I don’t think it’s about any specific prediction he has in mind. The 2027 bit is just him saying that PHP2Shell wasn’t bad enough to force real action, so the first actually disruptive AI-assisted outage is when people (and funding) suddenly care. TL;DR not a forecast, just the shape of the argument in this discussion: https://x.com/matthew_d_green/status/2113749868888076487original ↗
MMatthew Green@matthew_d_green
And that need happens exactly three hours after the first massive outage caused by AI hacking. That’s not PHP, and we haven’t seen it yet. I think we might see it in 2027. Be nice if people had a plan for when the government suddenly realizes they need to Do Something.
AI didn't create the technical debt, it just made ignoring it expensive. That crash ticket sat in a queue since 2024 because an out of memory error reads as annoyance and not risk, and the step connecting annoyance to breach was always the costly part. It just got cheap.
The asymmetry is the real story. Finding a bug is fast, parallel and now cheaper, closing one means shutting every path without breaking anything that works, and four engineers spent two months on this patch. Meanwhile the layer holding up a huge chunk of the web is mostly a goodwill project, and the current defence is partly labs donating token budgets to hunt bugs. That's freeload, not a plan.
Which is why the interesting bit is the referee at the end of the chain. An LLM writing its own patch and its own proof gets to be defendant and judge, so you want a checker at the end that isn't allowed to hallucinate. Gawande had this figured out for surgeons, a checklist is the discipline you keep when memory and vibes stop being enough, and it reads the same when the crew is a model.
🤔
I don’t think it’s about any specific prediction he has in mind. The 2027 bit is just him saying that PHP2Shell wasn’t bad enough to force real action, so the first actually disruptive AI-assisted outage is when people (and funding) suddenly care. TL;DR not a forecast, just the shape of the argument in this discussion: https://x.com/matthew_d_green/status/2113749868888076487 original ↗
And that need happens exactly three hours after the first massive outage caused by AI hacking. That’s not PHP, and we haven’t seen it yet. I think we might see it in 2027. Be nice if people had a plan for when the government suddenly realizes they need to Do Something.