Essays on enterprise AI.

Two clocks run in AI and they tell different times. One says a frontier training run costs half a billion dollars. The other says the same capability will cost a tenth as much a few months later. This is an essay about the second clock.

The site is relaunched — new stack, new design, essays restored from the archive.

We've a problem in the tech world. Many teams get carried away and reach for LLMs when all they need is a straightforward classifier or regression model.

The AI future worth building is not Manhattan-sized data-center of GPUs, but shoebox-sized breakthroughs in physics, algorithms, and chips.

As artificial intelligence (AI) technologies advance rapidly, effective governance frameworks are essential to mitigate risks, ensure ethical deployment, and foster innovation.

In the fast-paced world of AI, governance often sounds like a lofty, bureaucratic hurdle, especially for companies just dipping their toes in.

Generative AI is everywhere in boardrooms today.

The hype is everywhere and the investments are massive — but what if we're expecting too much, too soon? We've been here before: flying cars, Mars colonies, limitless nuclear energy.

A curated collection of 50 quotes on AI from technologists, researchers, artists and critics — grouped by theme, for anyone who needs the right line to open a talk or start an argument.

We've quietly accepted AI for writing emails but raise eyebrows when a student uses it to draft a thesis. So where do we draw the line — and is the real question how much AI, or how we use it?

Meta shared model weights with researchers, the weights escaped within a week, and within a month people were running them on laptops and fine-tuning them for a few hundred dollars. Whatever anyone intended, the question of who can run a capable language model is now settled.

The fastest product launch anyone has seen has left every executive and every customer with a new idea of what software should be able to do. That expectation shock is the real event, and 'just add a chatbot' is about to become the most dangerous sentence in your roadmap.

Two months ago, image generation was a service you rented from one of two companies. Now the weights are on your hard drive and it runs on a gaming card. That's not a better product, it's a different market structure.

DeepMind trained a model four times smaller than its predecessor on four times more data, and it won on nearly everything. The whole field had been building models that were too large and too undertrained, and the correction is worth money to anyone serving them.

OpenAI's InstructGPT work shows a model a hundred times smaller being preferred by human judges over the original. The technique isn't a better architecture. It's asking people which answer they liked, and that changes what these systems are for.

Microsoft's LoRA paper freezes the entire pre-trained model and trains a tiny fraction of new parameters instead. Ten thousand times fewer trainable weights, comparable quality, and you can keep a hundred specialised versions on one machine.

Andrew Ng has spent the year arguing that the field has been optimising the wrong half of the problem. In enterprise settings I think he's straightforwardly right, and the evidence is in every project that stalled at eighty per cent accuracy.

The pandemic shut down the economy your forecasts, fraud checks, credit scores and staffing rotas were all quietly describing. They kept running, kept sounding confident, and kept being wrong — and almost nobody's monitoring said a word.

OpenAI's GPT-3 paper does something the previous ones didn't. You describe the task in plain English, show it two or three examples, and it does the job with no training at all. That changes who gets to build with this.

You hired good people, they built models that work, and nothing has reached production in eighteen months. This is the most common failure in the field and it is almost never about the modelling.

OpenAI announced a language model and then declined to release it, saying it was too dangerous. The debate has been about whether that was responsible. I think it's the wrong debate, and the right one is about what your business does when plausible text becomes free.

Google took the pre-training idea, let the model see the whole sentence instead of half of it, and posted results that embarrassed everything before them. Then they published the weights. That second part is the business story.

Three years ago I watched a team train a model and realised I'd been writing about a craft I'd never performed. I went and did the coursework. Here's what actually changed, and it wasn't the algorithms.

OpenAI's new paper reads seven thousand unpublished books, learns nothing in particular, and then beats the state of the art on nine tasks out of twelve. If labelled data has been your bottleneck, read this one.

A machine phoned a hair salon, said 'mm-hmm' at the right moments, and booked an appointment without the person on the other end knowing. The demo was extraordinary. The reaction to it was the more interesting result.

Most compliance work has focused on cookie banners and consent forms. The harder questions are about the models, and almost nobody has answered them: what happens to a right to erasure when the data is baked into weights?

It's a good line and it's doing real damage in boardrooms. Electricity was a utility with a standard socket. This is a bespoke, brittle, data-hungry capability that has to be rebuilt for every problem. Budget accordingly.

A Google paper in June threw out the recurrent network entirely and beat the state of the art in translation after three and a half days of training.

ProPublica's investigation into risk scoring in the American courts is the most important thing published about machine learning this year, and it has nothing to do with accuracy. Disparate impact does not care that software made the decision.

Microsoft launched a chatbot that learned from the people talking to it, and had to shut it down before the end of the day. It's the clearest governance lesson this field has produced, and almost nobody is drawing it.

DeepMind's program beat Lee Sedol 4-1 last week and the coverage has been breathless. The technique is genuinely important. The reasons it worked are also the reasons it won't transfer to your business, and that's the part worth understanding.

Microsoft Research just won ImageNet with a network eight times deeper than anything before it, and the trick was to let layers do nothing. It also crossed the human error rate, which changes the conversation about what you can buy off the shelf.

TensorFlow is free, and the reflex is to ask what the catch is. There isn't one, exactly. But the strategic logic behind giving away your machine learning infrastructure is worth understanding, because it tells you where the real asset sits.

I spent a day with the team Ajay Vishnu works on, watching them teach a computer to recognise hand-drawn doodles. I went in curious and came out with a reading list and a slightly embarrassing sense of urgency.

Adam is not a breakthrough in what machines can learn. It's a breakthrough in how little babysitting the learning needs. If you're paying for GPU hours and engineer hours, that's the paper that matters.

The vendors have a new word and it's doing a lot of work. Schema-on-read sounds like flexibility. In practice it means nobody agreed what anything meant, and you find out eighteen months later.

Ian Goodfellow's new paper sets two neural networks against each other, a forger and an inspector, and lets them train each other. The pictures it makes today are tiny and blurry. Put it on your risk register anyway.

I left Kondra Systems last month after nearly four years. The thing I keep thinking about is a project we never built: an assistant that manages your time. It was David Vogt's idea and I thought it was ten years early.

A London company with about fifty people, no product, and no revenue just sold for a reported four hundred million pounds. That price tells you something important about where capability in this field actually lives.

The summer's disclosures changed what a database is. It used to be an asset you protected. It's now also a set of obligations you inherited, and most companies have no idea how many they're carrying.

A team at Google worked out how to turn words into coordinates, and trained the whole thing on a billion and a half words in under a day. If your business runs on text, this is the cheapest useful thing you'll read about all year.

Every year there's a computer vision competition you've probably never heard of. This year a team from Toronto didn't just win it, they broke it. And the way they did it should change how you think about your image data.

He called all fifty states while television pundits called it a coin flip. The lesson for business isn't about polling. It's about the difference between a prediction and a probability, and why your company almost certainly makes the first kind.

A long look at how modern ad systems decide what to show you, and what happens in the tenth of a second between a page loading and an ad appearing. Written by someone who is still working out how half of it fits together.

The job title barely existed four years ago and now it's on every hiring plan. Most of those roles will fail, and they'll fail for organisational reasons rather than technical ones. Here's what breaks.

Everyone is standing up Hadoop clusters and calling it a data strategy. Storage got cheap. Asking good questions did not. Here's how to tell which one your company has actually invested in.

IBM's machine beat the two best Jeopardy players alive, and the coverage has been about as measured as you'd expect. The technique is real. The economics of putting it in your business are not, yet.

Customary first post! I've joined Kondra Systems in Gurgaon, I've been handed a copy of The Pragmatic Programmer, and I've started wondering what happens to everyone who drives for a living.