Power up. From supercharging employee productivity and streamlining processes to inspiring innovation, Microsoft’s AI is designed to help you build the next big thing. No matter where you're starting, push what's possible and build your way with Azure's industry-leading AI. Check it out.
“Roger has five tennis balls. He buys two more cans of tennis balls. Each can has three tennis balls. How many tennis balls does he have now?”
For you, this may have triggered stress-flashbacks to grade-school math quizzes. But for Google’s brand-new large language model, it’s a key part of training.
This week, Google introduced the Pathways Language Model (PaLM), its new AI tool designed to answer questions, reason through arithmetic questions, and even explain jokes. According to Google’s performance report, it may be the most advanced model of its kind on a number of benchmarks, including tasks in reasoning and logical inference.
Quick recap: Large language models (LLMs) are an increasingly popular—and controversial—AI tool used for all things natural language processing (think: summarizing text, participating in dialogue, writing articles, and more).
- Generally speaking, the more parameters a model is trained on, the higher its performance—and the more capable it is of reflecting biases learned from training data.
- These types of models have also become less expensive and faster to train in recent years.
With 540 billion parameters, Pathways isn’t Google’s largest language model—the company’s 1.6-trillion-parameter model, announced last year, owns that spot. But PaLM is a headliner in other ways, performing better than comparable LLMs (think: GPT-3 and LaMDA) in reasoning tasks, multi-step arithmetic, and multilingual tasks like translation.
- And according to Google, its training efficiency is the “highest yet achieved” for language models of its size.
- PaLM’s training data includes “books, Wikipedia, web documents, conversations, and GitHub code.” About 22% of the data is non-English.
Abilities and implications
One of PaLM’s notable advancements is its ability to use ideas that build upon each other to arrive at a conclusion. For the arithmetic question at the beginning of this article, the model’s example output may look similar to how a human would think through it: “Roger started with five balls. Two cans of three tennis balls each is six tennis balls. 5 + 6 = 11. The answer is 11.”
- PaLM solves 58% of grade-school-level math questions in a popular 8,500-question dataset. That beats GPT-3’s record of 55%—and comes close to the 60% average for 9- to 12-year-olds.
And when given a string of emojis, the model can also guess the movie title from a multiple-choice list (🤖🪳🌱🌎 = WALL-E) and explain a niche tech joke (the punch line: a group of TPU computer chips and a group of whales are both known as a pod).
Big picture: With great model size comes great responsibility. It’s important to note that LLMs tend to propagate human bias, which can harm vulnerable populations. For its part, Google provides some risk analysis for PaLM, including a data sheet, model card, and responsible AI benchmark results.
Last year, the company fired two top AI ethics researchers after they wrote a paper about the dangers of large language models, though Google disputes this account. That very same paper is cited by Google in its research on PaLM.—HF