Writing
-
Six million queries a day in Hinglish, where the entity is the whole query and word error rate is the wrong metric. What the ACL 2023 paper says about endpointing, why not RNN-T, and the transliteration trick that made code-switching stop being a special case.
-
Multiverse sold making other people's models smaller, and had a supercomputer allocation to train a 30B from nothing. If models are that overparameterised, why train a big one? What the layer sensitivity profile actually says, and the benchmark row that does not cooperate.
-
I implemented the decomposition from scratch and ran it on GPT-2. Trained weights turned out to be 1.5% more compressible than Gaussian noise, plain SVD beat the tensor-network split at every budget, and an 11% parameter saving cost a 22x increase in perplexity. The healing is the method.
-
Ninety lines, 0.54 microseconds per decision, and it never reads the payload. The follow-up I promised at the end of the permission matrix post, with a compromised model and 36 attempts that go nowhere.
-
Once an agent has tools, failures stop being “it said something weird” and start being “it did something expensive.” The permission matrix I use to stop that, on one page.
-
What actually ships for computer-controlling agents: OpenAI’s CUA, Claude’s computer use, the open-source options, and why designing for human-like control buys you reliability.
-
AI writes code well and designs systems badly. Where I think that line sits, and what it means for how engineering work gets divided up.
-
Most AGI talk assumes a better version of us. I argue the interesting target is an intelligence that thinks differently: self-directed, self-aligning, and generating knowledge we don’t have.
-
Training data is the bottleneck and curiosity is the way out. Six concrete steps toward systems that generate their own learning experiences instead of waiting to be fed.
-
I built an open-source system that turns plain English into structured, executable actions on a computer. An early run at Karpathy’s LLM OS idea, and what it taught me.