Meta unveils plan to build “universal,” real-time translation for metaverse

It will rely on a handful of new AI models, as well as a technique called self-supervised learning.
article cover


· less than 3 min read

Stay up to date on tech

Drones, automation, AI, and more. The technologies that will shape the future of business, all in one newsletter.

If the metaverse is going to work as many tech execs envision, it needs to offer not just real-time communication, but also real-time translation between hundreds of languages.

To get there, Meta plans to go all-in on language AI. During a live announcement Wednesday, Mark Zuckerberg and others laid out the plans for the research projects, which center on new AI models and a new AI teaching strategy.

No more middleman: Right now, according to Zuckerberg, the translation process for many languages—especially ones with less internet data—looks something like this: Language A → English → Language B. Meta wants to change that with a new open-source AI model that removes the English step, translating directly between 100 languages.

  • In the future, Zuckerberg said, Meta hopes to apply that same technology to “content and experiences in the metaverse.”

Underrepresented languages: Meta also outlined new ambition for a single AI model that “can learn every language even if there isn’t a lot of text available to learn from.” Three years ago, it could translate 30 languages, Zuckerberg said. This year, the company’s goal is hundreds.

Instant translation: The third language AI tool Meta announced is also the toughest to imagine: a real-time speech translation tool for all languages, dubbed a “universal speech translator” (think: an all-knowing Rosetta Stone).

  • “The ability to communicate with anyone in any language—now, that’s a superpower that people have dreamed of forever, and AI is going to deliver that within our lifetimes,” Zuckerberg said.

Secret sauce

To underpin all these goals, Meta plans to lean on a pivot to self-supervised learning.

  • In the past, the company taught its AI systems using a supervised learning method (e.g., using large data sets of human-generated examples).
  • Now, Meta will allow these AI systems to learn “without any human supervision,” Jérôme Pesenti, Meta’s VP of AI, said—for example, removing words from language text and guessing the blanks by inferring patterns in surrounding words.

But, but, but: There’s a reason why these technologies don’t exist yet. As your friends who downloaded Duolingo over the pandemic will tell you, language is complex and requires a lot of nuance and common sense—two things that AI is notoriously bad at. Without humans in the loop, things can get mistranslated in confusing or problematic ways. ​​

Stay up to date on tech

Drones, automation, AI, and more. The technologies that will shape the future of business, all in one newsletter.