Written June 2024. The original combined AGI, MLOps, blockchain, federated learning, synthetic data, neuromorphic computing and quantum computing into one post, which is a warning sign I did not read at the time. Rewritten in 2026 around the one real question underneath it.
There is a genuine question buried in the original: the operational practices we built for machine learning assume things that stopped being true. It is worth asking which ones broke, and the answer is more specific than “everything is different now.”
What classical MLOps assumed
The discipline was built around a particular object: a model you train, whose inputs and outputs are typed, whose quality is a number on a held-out set, and which you redeploy when that number moves. Every tool in the stack encodes those assumptions. I built one of these in the CIFAR-10 post, and the pipeline works precisely because the assumptions hold for image classification.
Four of them break for systems built on foundation models.
| Assumption | What replaced it |
|---|---|
| You trained the model | You are renting someone else’s, and it changes under you |
| Quality is a number on a held-out set | Quality is a judgment, often another model’s |
| Inputs are typed and bounded | The input is a string, and it may contain instructions |
| Retraining is the unit of change | A prompt edit ships instantly, with no review |
You do not control the model
The most consequential break. Your provider updates a model and your behaviour changes without a deploy on your side, without a version bump you initiated, and often without a changelog that mentions the thing that broke. Pinning a version buys time and eventually the pin expires.
The practice that follows is regression evaluation you own: a fixed set of cases with expected properties, run on a schedule against whatever the provider is serving today, so that the graph moves before a customer tells you. This is the single highest-value thing to build and it is conceptually just a test suite for a non-deterministic dependency.
Evaluation is the hard part now
Accuracy on CIFAR-10 is unambiguous. “Is this summary good” is not, and the field’s answer (use a model as the judge) works, with a caveat that gets skipped: your judge is a model with its own failure modes, and if you optimise against it you will find them. You need a human-labelled slice to keep the judge honest, which puts a small amount of expensive human labelling back into a pipeline that self-supervision was supposed to have removed it from.
The input is now an attack surface
A classifier takes a tensor. A language model takes a string, and the string can contain instructions, and if the system has tools then those instructions can cause actions. There is no equivalent of this in the MLOps of 2020, and it is not a security bolt-on; it changes what the deployment boundary has to be. That is the whole subject of the policy matrix post.
Prompts are code with no engineering around them
A prompt determines system behaviour, so it is code. In most organisations it lives in a database field, gets edited in a web form by someone who is not on the deploy rota, ships instantly, has no review, no version history and no rollback.
The fix is unglamorous and complete: put prompts in the repository, review them, version them, and gate them behind the same evaluation suite as everything else.
What I cut, and why
The original recommended blockchain for training-data provenance, predicted that quantum computing would revolutionise MLOps by solving optimisation problems “in seconds,” and offered neuromorphic computing as an emerging trend. None of these were connected to anything else in the post, and none had a mechanism attached.
That is what a post looks like when it is assembled from what sounds advanced rather than from what the author has run. The tell is that the buzzwords could be permuted without changing any argument, because there were no arguments, and I would rather leave that note here than quietly delete the paragraphs.