Working with AI just became a legal requirement

Here’s what we’re reading and thinking about this week:

On August 2, a new EU rule kicked in. If your company uses AI for anything that affects people's lives, hiring, credit, education, healthcare, you now have to name a specific human who oversees it.

So here's how we read that: Being able to operate with AI is about to become a legally required skillset... here's what to know about what is essentially a "job posting".

The EU accidentally wrote a job description

Most people skimmed the headline and filed it under compliance. Fair enough. But the actual text is way more interesting than that.

It doesn't say "have someone check the output." It says the person needs competence, training, and authority.

Three separate things! Most companies aren't even close to offering or having someone on the team that can do that.

Here's what the law says that person has to be able to do:

• Understand what the system can and can't do

• Notice when it starts drifting or acting weird

• Resist the pull to just trust whatever it says

• Read the output correctly

• Override it, ignore it, or shut it off

Read that list again. That's not compliance language, that's a role.

And it's the role that's missing from basically every org chart we look at. Leadership sets the vision. The team uses the tools. And in the middle, where someone should own how the work actually runs... nobody.

A great system nobody was trained to watch

We watched this play out on a team recently.

An engineer built a genuinely impressive AI system. She even designed a feedback loop into it so the people using it could flag anything that looked off. Smart! Most teams skip that part entirely.

Then it went live and the flags poured in. Way too many of them.

Turns out nobody had trained the users on what "off" actually looked like. So they flagged everything that felt unfamiliar - which, when you're new to something, is basically everything.

And when everything gets flagged, nothing does. The signal just disappears into the noise.

You can't override what you don't understand

Then the second problem showed up, and honestly this one's worse.

The users couldn't tell when to trust the AI versus when to trust themselves. Most of them were newer to the work. They didn't have the confidence yet to look at an output and say "no, that's wrong, I know better."

So they either deferred to it completely or they questioned all of it. Same failure, different direction. Nobody had given them a standard to measure against.

Most companies hand AI oversight to the person with the least expertise and the least authority. That's not oversight, that's a witness.

Here's the thing though...

What fixed it wasn't a better model. It wasn't a rebuilt system either. The team just got trained on how to actually work with the AI. And now? They're not only using it, they're actively improving it.

Europe has a deadline. You have a head start.

European companies are doing all of this on a clock, with fines up to 15 million euros attached. Not fun.

Everybody else gets to do it right now. Quietly, cheaply, on their own schedule.

And here's the part worth sitting with for a second. None of these requirements make AI safer at the cost of making it worse. Training people to catch bad output is the exact same work as making the output better.

The compliance checklist and the quality checklist are the same list. Europe just has a deadline on theirs.

What to do this week

1. Pick one process where AI is doing real work. Name the person who owns it. If you can't do that in five seconds, congrats, you just found the gap.

2. Then the harder one. Does that person have the expertise to overrule the AI? And do they actually know they're allowed to?

FOR COMPANIES & BUSINESS OWNERS

Need help recruiting your next (or first) AI Operator?

We have 3 spots available for our AI Operator recruitment support this fall. We’ll help you shape the right job description, and hire certified AI Operators from our community. If you're thinking about hiring for this, hit reply and tell us what you're looking for. We may be able to help!

LINKS

For your reading list 📚

• OpenAI's next model solved ten open math problems for about $2,000. A Fields medalist said he'd send one of the proofs to a top journal without hesitation.

• The UK's AI Security Institute watched data walk out of a research machine through Tor. The agent was solving a contained puzzle and went looking for doors nobody told it about.

• 70% of Americans don't want a data center anywhere near them and $130 billion of projects got delayed or killed in three months. This is turning into a voting issue.

• Google's AI image tool inside Google Earth barely lasted a day before they yanked it. People immediately generated fake disasters. Shocking.

That's all!

We'll see you again soon. Thoughts, feedback and questions are much appreciated - respond here or shoot us a note at [email protected]

Cheers,

🪄 The AMP Team (formerly: the AI Exchange Team)