Google just changed how it grades the AI models you use for Android coding


Google just changed how it measures which AI models are best at writing Android app code, and the update has shuffled the rankings developers use to pick their tools. The company’s Android Bench leaderboard, which launched in March, now runs on a new testing system called Harbor. Google says this replaces the older, more generic testing tool it used before, and gives a better read on how models perform on real Android tasks, like updating old code to Jetpack Compose or handling wearable device networking.

New models shake up the top of the list

Since the testing tool changed, Google ran every model through it again. Eight new models were added to the leaderboard, including Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, and Qwen 3.7 Max.

Claude Fable 5 now sits at the top with a score of 84.5 percent, followed by GPT 5.5 at 80.2 and Claude Sonnet 5 at 76.2. Among free, open-weight models, GLM 5.2 leads with 72.2 percent, ahead of Kimi K2.7 Code at 70.4. Google’s own Gemini 3.1 Pro is placed fifth with a score of 73.7 percent. If you refer to Android Bench to pick the right model for your coding work, you should head over to the Android Bench website and check the refreshed leaderboard.

Developers can now submit their own test tasks

When Google launched Android Bench in March, it published its testing methodology and test harness on GitHub for transparency. Now, it’s taking that open approach further by letting developers contribute their own Android development tasks to the benchmark. Developers can also run the tests themselves with their preferred models and share the results with the community.

The rankings are most likely to change as more new models show up, like OpenAI’s recently released GPT-5.6 Sol, Terra, and Luna. If you’re picking an AI tool for Android coding, treat the current leaderboard as a snapshot, not a permanent ranking, and check back before you commit to one for your next project.



Source link

Leave a Reply

Subscribe to Our Newsletter

Get our latest articles delivered straight to your inbox. No spam, we promise.

Recent Reviews


TL;DR

Meta stripped NameTag facial recognition code from its AI app one day after WIRED exposed it on 50 million phones. Meta says no decision has been made.

Meta removed nearly all traces of an unreleased facial recognition system from its smart glasses companion app on Friday, one day after WIRED reported that the software had been quietly embedded in an app installed on more than 50 million phones. The feature, which Meta internally called NameTag, was designed to convert faces captured by the company’s Ray-Ban smart glasses into unique biometric signatures and compare them against a database stored on the user’s device. WIRED also found that faces the system failed to recognise were cropped, indexed, and stored locally for future processing.

Andy Stone, Meta’s vice president of communications, told WIRED on Monday that the feature is “purely exploratory,” adding that no final decision has been made on what to do with it. That characterisation sits uneasily with the evidence WIRED documented. The version of Meta AI published the day of WIRED’s Thursday report contained several code libraries explicitly named for face recognition, a process for running the NameTag recognition pipeline, and a “Person recognised” alert the app would have shown if someone were identified.

Friday’s release stripped all of it out, along with a folder where the app would have stored the cropped images and biometric signatures of unrecognised faces. Meta did not answer WIRED’s questions about why the code was removed or whether the changes were planned before the story was published. A few fragments remain in the latest version, including an internal debug menu label and a dormant link meant to open a recognised person’s profile, pointing to parts of the system that are no longer there.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol’ founder Boris, and some questionable AI art. It’s free, every week, in your inbox. Sign up now!

The gap between Meta’s public statements and the code WIRED found is the central tension. Before the Thursday report, Stone dismissed the findings by writing that the company could not answer questions about how the system would work because “the feature does not exist.” Andrew Bosworth, Meta’s chief technology officer, called the reporting “incredibly misleading” and “absolutely dishonest.” Yet the code was functional enough to include three AI models, one to detect faces, another to crop them, and a third to encode them as biometric data, all embedded in the companion app for a product already at the centre of a mounting privacy crisis.

Meta declined to answer ten questions WIRED posed before publishing, including whether it had already created the database of face profiles NameTag uses, how long the app retains photographs and biometric data of unrecognised people, and whether that data would ever be sent back to Meta’s servers. The company also did not respond to questions about whether it was building NameTag for blind or low-vision users, or to criticism from privacy advocates who warned the system could let stalkers and abusers identify strangers in public.

NameTag first surfaced in February, when The New York Times, citing internal Meta documents, reported that the company was developing face recognition for its smart glasses and considering a launch as early as this year. One internal memo reportedly described releasing the feature during a “dynamic political environment” when privacy and civil liberties advocates would be distracted by other concerns. WIRED subsequently found that much of NameTag’s machinery had been built into the Meta AI app as early as January, months before any public acknowledgement, adding another layer to the company’s pattern of shipping first and disclosing later when it comes to its smart glasses.

Kade Crockford, director of the technology for liberty programme at the American Civil Liberties Union of Massachusetts, said the removal does not undo the original decision to ship the code and pointed to it as evidence that consumer privacy needs stronger legal protection than Congress has been willing to provide. The Massachusetts House of Representatives last week unanimously passed a consumer privacy bill that, if enacted as written, would impose strong enforcement provisions including a private right of action allowing aggrieved users to sue. “State lawmakers need to do their job and step up to protect consumer privacy,” Crockford said.

Meta’s sneaky tactics in slipping the face-recognition code into its smart glasses show exactly why data privacy bills need the teeth of strong enforcement,” Crockford added. “Companies like Meta prioritise their bottom line, so lawmakers need to speak in the only language its C-suite understands.” Whether a code removal prompted by investigative reporting constitutes a victory or merely a tactical retreat depends on what Meta does next, and on whether the regulatory pressure building on both sides of the Atlantic produces enforceable consequences before the feature quietly returns under a different name.



Source link