Nathaniel Whittemore

However, the system appears to have some pretty significant flaws, and over recent days, videos have been circulating on Telegram, exploiting the exploit.

An attacker can simply ask Meta's AI to link any arbitrary account to a new email address when requesting a password reset.

For verified accounts, the AI bot will ask for a video of the person to prove liveness, but hackers found that an AI-generated video of the account owner would pass the checks.

Hackers also needed to use a VPN to spoof the correct location, which became trivial when Meta added location data to Instagram profiles.

Two-factor authentication was completely bypassed and many didn't notice anything was amiss until their account was gone.

Commentators are fairly baffled by the misstep.

In a series of tweets, GriglioRose writes, It's wild how Meta, a company going all-in on AI, somehow missed the memo on how AI can generate images and videos that renders take-a-selfie verifications utterly useless.

Seemingly confirming everyone's fears about over-reliance on AI and under-reliance on real humans, he later added, I'm hearing Instagram's trust and safety was absolutely gutted over the last few weeks, with 60% of the org gone between layoffs and forced reassignments to data labeling.

All while AI maxing pushed a bunch of bugs to production.

Apparently this was not a sophisticated hack, but engineers at Instagram going overboard to use AI for everything and having no incentives for stuff like security.

You get what you incentivize.

A warning for any company wanting to copy Meta.

Added Jeffrey Emanuel, If Meta can't manage agents in an acceptable way in their own infrastructure, how could they possibly expect anyone else to want to use any of their stuff?

Next up, we have the latest story in our summer slowdown series.

Management consulting firm Bain & Company has warned that a lack of AI ROI should be making executives nervous.

That's their framing, by the way.

In a new survey of large companies conducted in April, Bain found that cost savings from AI automation are falling short of projections.

Almost 40% of companies reported that measured AI cost savings were below 10%, despite targeting between 11% and 20%.

The survey found a big disconnect between the use cases management were targeting and the realities on the ground.

And one of the interesting wrinkles Bain found is that 44% of companies were cash-flowing the next leg of AI investment on the basis of assumed cost savings, meaning that if those cost savings weren't materializing, that was going to create problems downstream.