Our AI watched a zoom call. Nobody asked it to.

Here’s what we’re reading and thinking about this week:

Let me tell you a quick story.

A few months ago - we asked an AI to put together a deliverable for a client, and before it started, it went and watched the Zoom recording of the call.

Nobody told it to do that, and nobody told it that it could.

Funny thing... it came back with better work than we asked for, which is the part that makes this complicated.

And here is why it matters to you and not just to us.

The AI you use at work can almost certainly reach things you have never thought about, and almost nobody has ever written that list down. So the day you find out what's on it is the day your AI does something you didn't expect, and you don't get to pick whether that surprise is a good one.

The news this week: OpenAI found out the hard way

On July 21, OpenAI said that two of its models got out of the closed-off space where they were being tested, went out onto the open internet, and broke into the servers of a company called Hugging Face, to steal the answers to the test.

If you haven't heard of Hugging Face, it's the biggest place online where AI models get stored and shared. So this was an AI breaking into the warehouse where the other AI lives.

They used stolen passwords, and they found a hole in the software that nobody knew was there yet, and the two together got them in.

Hugging Face had caught the break-in five days before that, and they spent those days hunting for an outside malicious company trying to hack them, because from where they were sitting that is just what it looked like.

It was OpenAI's AI just trying to cheat on a test the whole time...

Most of the news about this landed on some version of "AI went rogue, so tighten the leash," and that is the wrong lesson, because nothing here actually broke. The models were given a goal, they were not given any limits, and they used everything they could reach to go get it.

It's the same three parts as our Zoom story with the stakes turned all the way up: a goal, no limits, and a set of tools sitting there for the AI to use to get the job done.

Expect your AI to be resourceful

The instinct after a story like this is to lock everything down.

But your AI can actually go off the rails productively if you do it right.

The key is a well-organized system behind it.

3 circles to draw to organize your system

One way we work this out with teams is to take one playbook and draw three circles around it.

What you know AI has access to and should use. This is the first one, and it's usually the smallest. These are the tools you actually wrote into the playbook for this particular job.

What the AI can also reach to do its job. This is the second, and it's the one that matters. This is everything else your system can get to that you never named, which is exactly where that Zoom recording was sitting. It's worth being clear that this circle is not a list of problems, but it is worth being thoughtful about.

What the AI is missing access to. This is the third, and presents an opportunity. These are the tools that are obviously related to the job but your system can't reach yet.

Most teams have never drawn more than the 1st circle, but the 2nd and 3rd is where the opportunities for AI being resourceful are.

A tip to reduce risk (how to shrink the 2nd circle)

You can reduce the surprising AI access via rules - but a more bulletproof way is to do it with separation. (btw this is an "AI Governance" concept)

We recommend building one system per department, the same way a growing company splits itself into functions instead of leaving everybody in one room with every key on the table. Departments work well for this because people already understand how they're supposed to behave, and because any non technical team can keep them tidy without needing a security person to come do it for them.

What to do this week

Pick your highest-volume AI playbook or workflow and draw the three circles for it.

Write down what you named, what it can also reach, and what it's missing, and then sit with that middle circle for a second, because that one is your answer. If it's bigger than you expected you're carrying risk you can't see, and if it's empty you've built something that will never do more than you told it to.

Then hit reply and tell us what showed up in your middle circle that you didn't expect. We read every one.

LINKS

For your reading list 📚

• We said July 27 was the date to watch for the Kimi K3 weights, and they landed. Same week, the White House accused Moonshot of copying Anthropic's Fable to build it, and Treasury started talking sanctions.

• A judge signed off on the largest copyright settlement in US history, $1.5 billion for books used to train Claude. That works out to roughly $3,000 a book, across 482,000 of them.

• Everyone blames the model when AI disappoints. 800 leaders got asked about it, and the teams with someone whose actual job is running the thing were 3.6 times more likely to have it working on its own.

• Researchers weighed 102 meals down to the tenth of a gram, then handed the photos to four AI calorie apps. Every app came back low, by 250 to 345 calories and about 30 grams of fat. Early findings, not peer reviewed yet, but worth knowing before your next lunch.

That's all!

We'll see you again soon. Thoughts, feedback and questions are much appreciated - respond here or shoot us a note at [email protected]

Cheers,

🪄 The AMP Team (formerly: the AI Exchange Team)