AI/Biz

The Model Isn't the Risk

In February OpenAI described a language model called GPT-2 and then declined to publish the full version, citing concerns about misuse. They released a smaller one instead, with a stated plan to release more over time.

The reaction split along predictable lines. Some called it responsible caution. Others called it a publicity exercise dressed as ethics, and pointed out that withholding weights while publishing the method delays reproduction by months at most.

I find that argument less interesting than the thing both sides are stepping around.

The Capability Is Coming Regardless

Here’s what I’d want a business audience to understand. The method is published. The architecture is a scaled-up version of the pre-training approach I wrote about in 2018. The compute required is substantial but not exotic, and it gets cheaper every year.

Whether OpenAI publishes this particular set of weights changes the timeline by a matter of months. It does not change the destination.

“The question isn’t whether one lab releases one model. It’s what your business looks like in a world where fluent, plausible, targeted text costs approximately nothing. That world is arriving either way.” — Sameer Gupta

So planning around the release decision is planning around the wrong variable. Plan around the capability.

What Actually Changes

I want to be concrete rather than gestural, because the discussion has been quite abstract.

Volume stops being a signal of legitimacy. A great many systems implicitly assume that producing a lot of coherent text takes human effort. Review platforms. Comment moderation. Consultation responses. Support queues. Complaint handling. Grant applications. That assumption is the load-bearing wall and it’s about to come out.

Personalisation at scale becomes trivial. The dangerous version of a phishing email isn’t a fluent one. It’s one that mentions your actual employer, your actual project, and your actual colleague’s name, written in a register appropriate to your industry. That currently takes a person a few minutes. It’s about to take a fraction of a penny.

Astroturfing gets cheap. Manufacturing the appearance of public opinion currently requires either a lot of people or obviously repetitive output. Neither constraint survives.

Content moderation economics invert. Defenders currently benefit from the fact that generating abuse takes effort. Remove that and the volume moderation systems face changes by orders of magnitude.

What I’d Actually Do

None of this is next quarter’s problem. All of it is worth an hour of thought now, while there’s no urgency and therefore time to think properly.

  • Find every process where text volume implies effort, and ask what happens when it doesn’t. This is a half-day exercise and most companies have never done it.
  • Stop treating “it was well written” as a credibility signal in any automated screening. It was always a weak signal. It’s about to be no signal.
  • Move verification to things that are expensive to fake. Payment history. Verified identity. Physical possession. Prior relationship. Anything that costs a real-world resource rather than tokens.
  • Warn your staff specifically. Generic phishing training is built around spotting bad grammar and odd phrasing. That advice is becoming actively harmful, because it teaches people that fluent means genuine.
  • Assume inbound text may be generated, and design processes that degrade gracefully rather than assuming a human wrote it.

“Every piece of security advice that amounts to ‘look for the typos’ has an expiry date, and it’s sooner than the people giving that advice think.” — Sameer Gupta

On the Release Debate Itself

I’ll offer a view since everyone else has.

I think staged release is a reasonable thing to try and I think the criticism that it’s unfalsifiable is fair. There’s no way to know what would have happened otherwise, which means there’s no way to evaluate the decision, which means it can be defended indefinitely regardless of outcome.

What I’d rather see, and what I think would genuinely help, is the labs publishing what they learn about detection alongside what they learn about generation. If you’ve built the most capable text generator in existence, you’re also the party best placed to characterise how its output can be identified. That’s a contribution that helps defenders and doesn’t require anyone to trust a release process.

I’d also note something slightly uncomfortable. A norm where capable labs withhold work on safety grounds sounds prudent, and it also happens to concentrate capability in the hands of a small number of well-funded organisations who decide what the rest of us are ready for. I don’t think that’s the intent. I do think it’s an effect worth naming.

Final Thoughts

The honest summary is that a research lab made a judgement call under genuine uncertainty, took criticism from all directions, and the substantive question went largely undiscussed.

That question is not “should they have published.” It’s “what does your organisation do when the cost of producing convincing text falls to zero,” and it has an answer you can start working on today without waiting for anyone’s release decision.

Most companies have a handful of processes that quietly depend on writing being effortful. Find yours now. It’s a cheap exercise and the alternative is discovering them one at a time, under pressure, when it matters.