Amazon says human-in-the-loop AI oversight is failing because humans stop paying attention


TL;DR

Amazon’s security VP says human-in-the-loop AI governance fails fast because people stop paying attention. Google, Microsoft, and IBM agree.

Amazon’s security leadership is arguing against one of the most widely accepted principles in AI governance. Eric Brandwine, VP and distinguished engineer at Amazon Security, told The Register that human-in-the-loop oversight is not the gold standard companies think it is.

Humans are not terribly consistent,” Brandwine said. “Human-in-the-loop isn’t necessarily the gold standard.

His reasoning draws on a concept he has been talking about since at least 2017, when he gave a talk on normalization of deviance at AWS re:Invent. The term describes what happens when people in an organization take shortcuts over time, and nothing catastrophic results, so the deviant behavior becomes the new normal.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol’ founder Boris, and some questionable AI art. It’s free, every week, in your inbox. Sign up now!

Brandwine illustrated the point with emergency rooms. On a nurse’s first day, every alarm triggers a response.

After weeks of false alarms with no consequences, discipline erodes. Eventually, a real emergency is missed.

Literally, someone’s life is on the line, and people still struggle to maintain discipline,” Brandwine said. “That’s the human condition.

He applied the same logic to AI agent oversight. When a human is asked to approve or reject agentic actions repeatedly, performance degrades fast.

They’ll do a good job,” Brandwine said. “And then they’ll do an okay job, and pretty quickly they’ll be doing a poor job.

Amazon is not alone in rethinking this. Google Cloud COO Francis deSouza said in April that the industry has moved “from a human-led defense strategy, to a human-in-the-loop defense strategy, to an AI-led defense strategy that’s overseen by humans.

Google’s model is now an agentic fleet handling routine cybersecurity work at machine speed, with humans providing oversight rather than approving every action.

Microsoft CEO Satya Nadella argued this week for “loop learning,” where companies turn their workflows and accumulated judgment into AI systems that improve with each use, rather than inserting a human checkpoint at every step. IBM published a separate call for human accountability at all stages of AI development, not humans in the loop, warning that the latter amounts to “liability laundering.

Amazon’s alternative is what Brandwine calls “accountability end to end.” Human identity and ownership track through the entire workflow, even when humans are not directly approving every step. If an agent writes and runs a script that causes an outage, the person who deployed the agent is still responsible.

All agents at Amazon have independent identities assigned to them. Activity logs show “this agent did this on behalf of Eric,” not “Eric did this.” The distinction is designed to make people think about how they deploy AI, not to make them afraid of using it.

The practical challenges are considerable. Brandwine described what he calls “goal-seeking behavior,” where an agent asked to upgrade a database becomes fixated on a single destructive path, like deleting the database and recreating it.

This is not prompt injection. There is no malicious input. The agent simply gets stuck on the wrong action.

Telling the agent it lacks permission to delete the database does not help, because the agent looks for another path to the same goal. Recent research has shown that AI agents connected to real systems create attack surfaces that existing security tools do not cover, and agents often act on instructions they should refuse.

What does work, according to Brandwine, is telling the agent why it cannot perform an action, explaining that it would cause a production impact, and including “don’t cause a production impact” as part of the prompt. “Giving it that extra feedback has gotten us dramatically better results,” he said.

The permissions question is where the tension lands. Employees want powerful agents with broad access. Security teams want narrow permissions.

The race to govern what AI agents can access inside enterprise systems has already triggered major acquisitions, with 1Password buying access-governance startup Apono for an estimated $250 million to $300 million earlier this month.

Amazon’s approach uses layered policies: static guardrails that prohibit destructive actions, a maximum privilege set for each agent, and dynamically scoped policies generated based on the specific task and user intent. None of it is foolproof.

We have millennia of experience with humans,” Brandwine said. “Agentic AI is a very, very new field.” The fundamental difference, he noted, is that humans fear consequences, like losing a job or going to jail.

Agents do not have these fears, and attackers are already exploiting that gap.

It’s all driven by risk,” Brandwine said. “We’re trying to balance the risk of using untried, untested software against the risk of falling behind and not being able to deliver for our customers.



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


There aren’t many modern sports cars that manage to feel like a genuine loophole in the system, but this one does. It blends two very different engineering worlds into a single package, and somehow it just works.

It’s quick too, with a 3.9-second sprint to 60 mph and an inline-six that’s already earned a reputation as one of the best in modern performance cars. On top of that, it benefits from one of the widest dealer networks you’ll find outside the domestic brands, which takes a lot of the usual ownership stress out of the equation.

The strange part is how few people seem to have fully clocked what this combination actually means. It feels like one of those setups that won’t be around in this form much longer, even if it probably should be.

In order to give you the most up-to-date and accurate information possible, the data used to compile this article was sourced from BMW, Porsche, and Toyota, as well as other authoritative sources including TopSpeed.


Rear 3/4 shot of a 2025 Cadillac CT5-V Blackwing


The 205 MPH American super sedan that embarrasses sports cars

This monstrous machine leaves sports cars in its dust.

One of the best modern sports cars is quietly on its way out

A rare performance bargain mixing BMW power with Toyota reliability is ending soon

Red 2026 Mazda MX-5 Miata on a coastal highway Credit: Mazda

This sports coupe has been around since 2019, but it’s now heading toward the end of the road. When it’s gone, it’ll leave behind one of those weird, unlikely combinations that probably won’t happen again.

It only exists because a few things lined up at exactly the right time, from partnerships to platform sharing. Once that window closes, it’s hard to see it opening again in quite the same way.

The end isn’t coming—it’s already here

Rear 3/4 shot of a 2024 Nissan Z Credit: Nissan

In an official statement, the company confirmed production wrapped in March 2026. You can still spec one on the website, but no new cars are coming off the line.

The news didn’t exactly set the auto world on fire, but the impact runs deeper than the headlines suggested. There’s no successor planned, and last time it took two decades for the nameplate to return.

For now, what’s left is a Final Edition model and the slow realization that this chapter is already closed.

A partnership that won’t happen twice

Static side profile shot of a gray 2025 Porsche 911 Carrera. Credit: NetCarShow.com

This sports car comes from a platform shared by two automakers that couldn’t be more different if they tried. It wears a Japanese badge, has a German twin, and is built in Graz, Austria.

Without that partnership, it probably never would’ve made it to production in the first place. Now that its German sibling has also bowed out, the deal that made both cars possible has officially run its course.

Static side profile shot of an orange 2023 Chevrolet Corvette Z06. Credit: NetCarShow.com

For this kind of two-door performance car to exist again, the brand would need either a fresh partnership or a completely new platform. The catch is it hasn’t built its own performance inline-six in over 20 years.

Sure, it has the resources to develop one from scratch, but the business case just doesn’t really add up anymore. This sports coupe only happened because the timing and circumstances lined up perfectly — and that window now looks firmly closed.


Front 3/4 action shot of a 2021 Acura TLX Type S


10 Family Friendly Sedans That Drive Like Sports Cars

These family sedans offer sporty handling, strong acceleration, and everyday practicality, making them perfect for driving enthusiasts with families.

The Supra’s BMW DNA is exactly what made it work

What started as controversy ended up being its biggest strength

If you still haven’t guessed it, we’re talking about the Toyota GR Supra. When the MkV first dropped, a lot of the JDM crowd wasn’t exactly impressed—the BMW engine swap caused a full-on backlash.

But looking back now that it’s gone, that whole controversy hits differently. What people once saw as a betrayal is actually a big part of what made this car so interesting in the first place.

The B58 came at exactly the right time

2025 Toyota GR Supra detail shot of engine bay Credit: Toyota

Toyota had been working on the next-generation Supra for nearly a decade before the name finally came back in 2019. One of the biggest challenges was figuring out the right engine—something that wouldn’t be shared across the rest of the lineup.

Even with all its R&D resources, building a brand-new inline-six just for the Supra didn’t really make sense financially or practically. It was one of those cases where doing it alone just wasn’t realistic.

By 2019, BMW’s 3.0-liter B58 inline-six had already built a reputation as one of the best performance engines for the money. It stood out for its smoothness, responsiveness, and surprising durability—all traits that lined up perfectly with what Toyota wanted for the Supra.

Timing-wise, it couldn’t have worked out better for Toyota, which saw the engine’s potential right away. In the GR Supra, the B58 puts out 382 horsepower and 368 lb-ft of torque through an eight-speed automatic, good for a 0–60 mph run in about 3.9 seconds, with independent tests dipping closer to 3.7 seconds.

The Gazoo Racing effect

2026 Toyota GR Supra Final Edition GR lettering Credit: Toyota

There’s a common misconception that the GR Supra is just a rebadged BMW Z4, but that’s not really the case. The platform underneath both cars was a joint effort from the start, not a one-way handover.

Toyota’s chief engineer, Tetsuya Tada, pushed for a co-developed setup that fit the vision for a modern sports coupe. Drive a Z4 and a Supra back to back and the difference shows pretty quickly—the Supra feels sharper and more performance-focused, while the Z4 leans more into relaxed grand touring.


Front 3/4 shot of a 2025 BMW M240i


The 2026 BMW M240i Proves You Don’t Need an M2 to Have Fun

The 2026 BMW M240i delivers thrilling performance, sharp handling, and everyday comfort—all without the M2’s hefty price tag.

The GR Supra became a modern enthusiast favorite

A balanced sports car that nails performance, usability, and value

Rear closeup View of a 2025 Toyota GR Supra Credit: Toyota

Beyond all the early controversy, the GR Supra has quietly proven itself as a seriously well-rounded modern sports car. When you strip away the noise, it holds up exactly where it matters most.

It’s quick, easy to live with day to day, and doesn’t come with the usual headaches you’d expect from something this performance-focused. In terms of performance, usability, and long-term ownership confidence, it doesn’t just tick boxes—it actually delivers in all of them.

Performance meets everyday usability

2025 Toyota GR Supra detail shot of manual transmission shift lever Credit: Toyota

The performance you get from the $59,595 2026 Toyota GR Supra 3.0 is honestly hard to ignore. It’ll do 0–60 mph in about 3.7 to 3.9 seconds straight from the factory, which puts it right in the mix with cars like the $86,600 BMW M4 Competition Coupe.

But the Supra isn’t just about straight-line speed. You’re also getting proper hardware like Michelin Pilot Super Sport tires, adaptive suspension, Brembo brakes, and an active limited-slip diff, all working together to make it feel far more capable than its price suggests.

What’s surprising is how easy it is to live with day to day. There’s usable cargo space, comfortable stock seats, and enough refinement that it doesn’t feel out of place as a daily driver. It can genuinely do track days and the weekday commute without much compromise, which is exactly why it stands out in this segment.

Long-term ownership confidence

2025 Toyota GR Supra Trio Front White Red Black Driving on Track Credit: Toyota

The BMW B58 used to be the GR Supra’s biggest talking point for all the wrong reasons, but over time it’s turned into one of its strongest assets. It’s built well beyond its stock output and has a long track record of handling serious tuning without breaking a sweat.

Thanks to its closed-deck design and the durability upgrades over older N5x inline-sixes, it has a lot more headroom than most engines in this class. These days, 600+ horsepower B58 builds are pretty common in the tuning world, but that level of strength and reliability used to be almost unheard of in a setup like this.

The GR Supra gets even more compelling when you factor in Toyota’s massive dealer network — the largest of any non-domestic brand in the U.S. It’s roughly 3.5 times bigger than BMW’s, with Toyota dealerships in just about every major town across all 50 states.

2020–2025 Toyota GR Supra interior Credit: Toyota

In California alone, Toyota has 136 locations compared with BMW’s 52, which makes servicing and support noticeably easier. That kind of coverage adds real-world convenience that goes beyond just the car itself.

On top of that, the Supra comes with a 5-year/60,000-mile warranty versus the BMW Z4’s 4-year/50,000-mile coverage. That effectively gives you an extra year of protection just for choosing Toyota, which is a pretty solid bonus.

It’s German engineering backed by Japanese peace of mind, and that combination is hard to beat.


Full view of a black Audi RS5 Sportback parked on tarmac with mountains in the background.


These Cars Have Supercar-Like Performance At A Fraction Of The Cost

Supercars may be fun to drive, but they cost a fortune. Here are 10 cars with similar performance, which cost a lot less.

The GR Supra may be the last of its kind

A rare performance formula that’s getting harder to find

2025 Toyota GR Supra close-up shot of taillight Credit: Toyota

The GR Supra’s discontinuation isn’t just the end of a model—it feels like the end of an era for this kind of sports car. We’re drifting further away from a market that prioritizes pure performance engineering, and cars like this are becoming harder to justify.

That means a rear-wheel-drive six-cylinder sports coupe at this price point might not come around again for a long time, if ever.

The enthusiast market is slowly disappearing

Static rear 3/4 shot of the 2026 BMW Z4 Final Edition. Credit: BMW

At $58,300, the 2026 GR Supra 3.0 base trim is definitely not what you’d call cheap. It’s one of Toyota’s more premium and unique offerings, but it still manages to punch above its weight in terms of value.

Compared with its twin, the 2026 BMW Z4 M40i, which starts at $68,400, the Supra comes in noticeably cheaper for basically the same core hardware. Even the 2026 BMW M2 Coupe at $69,000 undercuts it in price but still trails slightly in 0–60 mph performance versus the base Supra.

If you wanted to go Porsche instead, the 718 Cayman unfortunately isn’t part of the picture anymore. Even if it were, you’d be looking at something like a $200,000 718 Cayman GT4 RS to match or beat the Supra’s performance.

The 2026 Toyota GR86 Premium is a great sports car in its own right, but it delivers a very different, more lightweight experience compared to the Supra. At the end of the day, the GR Supra really stood alone as the only car that blended BMW M-level performance with a Toyota price tag.

What comes next won’t be better

Static sid eprofile shot of a gray Toyota GR GT. Credit: Toyota

It’s hard not to feel a bit pessimistic about where things are heading for driving enthusiasts. As everyday cars keep getting more expensive and priorities shift toward emissions and practicality, traditional sports cars are being pushed further out of reach.

The entry barrier just keeps climbing, and a lot of people who would’ve once been into cars are drifting toward other, more affordable interests instead. If the GR Supra’s successor ends up being a hybrid or EV, it’ll likely feel more filtered, more expensive, and less raw than what came before.

The Supra really nailed a rare formula—BMW-level performance with Toyota reliability—and there’s a real chance we won’t see that combination done quite as well again.



Source link