How 2 Students Used The Meta Ray-Bans To Access Personal Information

 

An audience member watches the Meta Connect Developer Conference keynote while wearing Ray-Ban Meta 2nd generation smart glasses at Meta's headquarters in Menlo Park, California on September 27, 2023.

An investigation by two Harvard students went viral Friday as they demonstrated how they used Meta’s Ray-Ban smart glasses to access the personal information of people on Harvard’s campus—including name, age, home address and phone number—raising significant security concerns about facial recognition and artificial intelligence technologies.

Key Facts

In a demo posted to X, formerly known as Twitter, students AnhPhu Nguyen and Caine Ardayfio can be seen walking around the Cambridge, Massachusetts, campus and subway stations and asking people they have never met if the information they have is correct: “Are you Betsy?” Ardayfio asks, before a woman confirms she is.

Nguyen and Ardayfio, who run the Augmented Reality Club at Harvard, told Forbes they happened to have access to a pair of Meta Ray-Ban smart glasses and realized that when they combined the glasses with various software, there were privacy implications.

By livestreaming to Instagram and having a program monitor the stream, the students engineered a way for a facial recognition software to start a search by scraping data from reverse image search engine PimEyes.

When a face was detected on the livestream, online links surfaced in which the person appeared, and by using a large language model, the students said they could access personal information like home addresses, phone numbers and even relatives’ names.

“This could be done with any regular phone camera and still do the same amount of damage,” Nguyen told Forbes, adding the smart glasses were just a tool they used for the project.

They built the project, dubbed I-XRAY, as a public service announcement to raise awareness that large language models have gotten to the point where they can scrape data and get insight at a completely automatic scale, Nguyen said.

Chief Critic

A Meta spokesman told The New York Times the company takes privacy seriously and designed safety measures, including a tamper-detection technology, to prevent users from covering up the LED light indicator with tape. The LED light is outward-facing on the glasses, and glows to indicate the camera is recording.

Key Background

Meta has faced backlash for the discreet camera on its smart glasses, which critics have said could enable bad actors to do more harm. The smart glasses have already reportedly led to a college football spy scandal, and reviews describe the LED light indicating the camera is on as a gentle white light that’s easily overlooked in broad daylight. Meta also confirmed that it may train its AI on images taken by the smart glasses if the images are uploaded into Meta AI.

How Do You Reduce Your Digital Footprint?

Nguyen and Ardayfio recommend people remove themselves from a list of databases and search engines that people may not realize their information is on, including the search engine PimEyes and major people search engine FastPeopleSearch, which typically offer free opt-out choices.