Nathaniel Whittemore

Meta AI leader Alexander Wang argued that MuseSpark is just the beginning, posting, This is step one.

Bigger models are already in development with infrastructure scaling to match.

Private API preview open to select partners today with plans to open source future versions.

One strand of the response that's been fairly consistent was basically, welcome back to the party, guys.

To some, even though this model is clearly behind the other leaders, the fact that the Meta Superintelligence Lab was able to get it out in less than a year since that lab was formed was a feat in and of itself.

Others were just less impressed.

Ethan Malek writes, After playing with it a bit, Meta's Muse Spark thinking is fine so far, but really doesn't match the current Big 3 models.

It is also a bit weird.

Like some strange language and tone, a little loose with facts, etc.

After giving a few examples, he concludes, anyhow, it's not bad, just not the vibe level that the benchmarks might indicate.

And for a first re-entry into the frontier model space, given the engineering efficiencies they achieved, it feels like a solid attempt.

I'm sure we will see better from Meta in the future.

ArcPrize founder Francois Chalet was less forgiving.

He wrote, the new model from Meta is already looking like a disappointment, over-optimized for public benchmark numbers at the detriment of everything else.

Knowing how to evaluate models in a way that correlates with actual usefulness is a core competency for AI labs, and any new lab is unlikely to be successful without first figuring that out.

Wang actually decided to respond to that one, saying, We're always open to feedback and welcome any perspective on weaknesses you've noticed in the model from using it.

We're quite upfront that our model does not perform well on ArcAGI2, for example, and publish those results for the community to understand.

That might reflect some areas of improvement of the model that we could focus on in the future.

In general, though, Wang reports, we have been pleasantly surprised by users' feedback on the model in areas like visual coding, writing style, and reasoning queries.

Voss on Twitter, who previously did work on Meta AI, said, Meta's latest model, MuseSpark, is actually much better than I had expected.