MLOps Simplified: Mastering CIFAR-10 on Your MacBook

Originally published December 2023 and rewritten in 2026. The first version described a pipeline without measuring it: it had a section called “Comparing Performance Improvements” that compared nothing. Every number below is from a run I did while rewriting it. Code: acebot712/mlops.

This is an end-to-end MLOps pipeline for a deliberately boring model: a small CNN on CIFAR-10, trained on a laptop. DVC and Google Cloud Storage for data and model versioning, MLflow for experiment tracking, GitHub Actions for CI, Docker for packaging, FastAPI for serving, ONNX Runtime for inference.

The model is boring on purpose. Nothing here is about squeezing accuracy out of CIFAR-10; that problem was solved a decade ago. It is about the distance between a notebook that produces a good number and a thing that answers requests, which is where most of the actual work in this field lives.

The model, so we can stop talking about it

Three convolutional blocks, batch norm, one linear head. 0.30M parameters. Eight epochs with one-cycle scheduling and standard augmentation, 254 seconds on an M4 Pro through the MPS backend.

def block(i, o):
    return nn.Sequential(
        nn.Conv2d(i, o, 3, padding=1), nn.BatchNorm2d(o), nn.ReLU(),
        nn.Conv2d(o, o, 3, padding=1), nn.BatchNorm2d(o), nn.ReLU(),
        nn.MaxPool2d(2),
    )

self.features = nn.Sequential(block(3, 32), block(32, 64), block(64, 128))
self.classifier = nn.Sequential(
    nn.Flatten(), nn.Dropout(0.2), nn.Linear(128 * 4 * 4, 10)
)

85.20% test accuracy. State of the art on CIFAR-10 is above 99%; a linear classifier on raw pixels gets about 40%. 85% in four minutes on a laptop is the right rung of that ladder for this purpose: good enough that serving it is a real problem, cheap enough that you can retrain it while reading this.

Note the dip at epoch 5 in the accuracy curve. That is the one-cycle schedule at its peak learning rate, and it is the reason to plot every epoch rather than only the last one: if you had stopped there you would have concluded something was broken.

The bit the original post skipped: does ONNX actually help?

The pipeline exports the trained model to ONNX and serves it with ONNX Runtime rather than PyTorch. The usual justification is “it is faster,” which I repeated for two years without checking.

So: single image, batch size 1, CPU, one thread for both runtimes so the comparison is like for like. 500 timed runs after 50 warmups.

	PyTorch	ONNX Runtime
p50 latency	0.63 ms	0.43 ms
p95 latency	0.65 ms	0.43 ms
Speedup (p50)	n/a	1.49×
Max |logit difference|	n/a	2.4e−06

1.49× at the median, and the tail is where it gets interesting: PyTorch’s p95 is 3% above its p50, ONNX Runtime’s p95 and p50 are the same to two decimal places. For a service with a latency SLO, the flat tail is worth more than the median.

The predictions are identical to within 2.4e−06, which is float32 noise. That check matters more than the speed number: an export that silently changes behaviour is not an optimisation, it is a bug you have shipped to production. Assert the agreement in CI.

The trap in the export

Here is what the export actually produced:

$ ls -la model.*
-rw-r--r--  1249211  model.pt
-rw-r--r--     5875  model.onnx
-rw-r--r--  1229184  model.onnx.data

model.onnx is 6 KB. It is tempting to read that as compression. It is not the model; it is the graph, and the weights are in the .data sidecar next to it. Copy only the .onnx into your container and it will build fine, push fine, and fail at session creation on the first request.

Totalled honestly: 1,235,059 bytes of ONNX against 1,249,211 bytes of PyTorch checkpoint. ONNX export buys you a runtime, portability and a flat latency tail. It does not buy you a smaller model. If you want that, you want quantisation, which is a different step with a different accuracy cost that you also have to measure.

And does the GPU help?

Same question as the ONNX one, a stage earlier. The training above runs through MPS because that is the reflexive advice, and I had not checked it either. This model is 0.31M parameters on 32×32 images, which is small enough that per-step dispatch overhead is a real fraction of the work.

Timed through the actual training loop (DataLoader, augmentation, host-to-device copy, then the step), because that is the throughput you experience rather than the one a microbenchmark reports.

So the advice holds, but the shape matters: at batch 4 the GPU is only 1.8× ahead, because each step finishes almost instantly and the loop spends its time waiting for the next four images. The device is not the bottleneck there; the pipeline is. Between batch 4 and batch 32 the ratio goes from 1.8× to 12× without the hardware changing at all.

The part that surprised me

The model in the repo before this rewrite was three plain conv layers and a 2048→512 linear head: no batch norm, and 1,147,466 parameters against this one’s 308,394. Running both through the same harness:

batch 128	parameters	CPU img/s	MPS img/s
old, no batch norm	1,147,466	2,035	18,956
new, batch norm	308,394	622	13,336

The smaller model is 3.3× slower on CPU. Batch norm is memory-bandwidth bound and does not vectorise the way a large matrix multiply does, so a network with a third of the parameters and six extra normalisation layers costs more per image on a CPU and less on a GPU. Parameter count is a fine proxy for memory and a poor one for time, and which direction it misleads you in depends on the device.

One honest note on method. The pure-compute version of this measurement (data preloaded, timer around the step only) gave MPS numbers that varied by up to 97% between repeats on this machine, so the figures above are from the full loop, where five repeats agreed to within a few percent. If you run this yourself and get wildly different GPU numbers, that is the reason, and it is why the code is in the repo.

The pipeline around it

Everything else is plumbing, and the plumbing is the point. In dependency order:

1. DVC + GCS. The dataset and the trained model are too big for git. DVC keeps a .dvc pointer file in the repo and the bytes in a bucket, so git checkout of an old commit plus dvc pull gets you the exact model that commit produced. This is the piece that makes “which model was serving on the 14th?” answerable.
2. MLflow. Logs parameters, metrics and artefacts per run. The value is not the UI, it is that six weeks later you can tell which of four nearly-identical runs is the one you deployed.
3. GitHub Actions. Lint, test, build the image, push to GHCR. The step worth adding beyond the obvious is the numerical agreement check between PyTorch and ONNX outputs: it is three lines and it catches the failure mode above.
4. Docker + FastAPI. The inference API in a container, so the thing that ran on my laptop is byte-identical to the thing that runs anywhere else.

End to end, from a clean machine:

docker pull ghcr.io/acebot712/mlops:latest
docker run -p 8000:8000 ghcr.io/acebot712/mlops:latest

curl -X POST 'http://localhost:8000/predict/' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F 'file=@cat.jpg'

What I would tell 2023 me

• Measure the thing you are claiming. I wrote a section titled “Comparing Performance Improvements” that contained no comparison, because I already believed the answer. The answer turned out to be right, which is worse; it means nothing would have corrected me.
• The latency tail is the number, not the median. Nobody pages you about a p50.
• Assert output agreement across runtimes in CI. Cheapest insurance in the whole pipeline.
• Most of this is version control for things git is bad at. DVC versions data, MLflow versions experiments, Docker versions environments, GHCR versions images. Once you see it that way the tool choices stop feeling arbitrary and start feeling like one idea applied four times.

Caveats

• Single machine, single run, batch size 1. Latency ratios shift with batch size, thread count and hardware: at batch 64 on a GPU this comparison would look completely different, and quite possibly reverse.
• CIFAR-10 at 32×32 is a small model. The ONNX Runtime advantage here is largely operator fusion and lower per-call overhead, which matters proportionally more when the model is small. Do not extrapolate the 1.49× to a transformer.
• The accuracy number is one seed. I did not tune anything; 85% is what the first sensible configuration produced.