semiconductors

'It's going to run a lot of the backbone of the internet': Lightmatter CEO Nick Harris on its energy-efficient AI chips

The four year-old startup has raised $113 million so far, and is building out its product and go-to-market teams
article cover

Lightmatter

· 8 min read

What if we took the tech powering quantum computing, and retrofitted it to AI processing?

Many years ago, Nick Harris, cofounder and CEO of Lightmatter, asked himself that question. He was a doctoral student at MIT, working on quantum computers, and realized he could repurpose the way they process light to solve a problem with another technology he was deeply familiar with: semiconductors.

As an R&D engineer fresh out of college, Harris had worked on the physics of transistors, or the nano-sized “switches” for electrical signals on computer chips. He saw not only how much energy they ate up, but also how much that energy usage was projected to climb over the next two decades. So Harris founded Lightmatter to marry his quantum experience with his semiconductor knowledge and try to make AI chips faster and more energy efficient.

Technically speaking, the startup deals in photonic computing, a new method of using light in AI processing. Its chips are built for use in data centers. The company says its AI chips are 10x faster than similar offerings from leading chipmakers like Nvidia, and that if its chips replaced NVIDIA DGX-A100 processors in 100 data centers, it would save the annual equivalent of the electricity used by 61.6 million homes.

In May, Lightmatter announced an $80 million funding round from investors including HP Enterprise and Lockheed Martin, and it’s using some of that money to build out its go-to-market teams and strategy. But at this point, the company is still largely research-driven, meaning the road to replacing incumbent datacenter chips is just beginning.

We chatted with Harris about the startup’s biggest challenge, its market strategy, and how its chips could affect everything from Siri responses, to Google Search results, to your next retail shopping experience.

Tell us the major challenge Lightmatter aims to tackle, with a new spin on an existing technology.

We’re building AI acceleration chips that power neural networks. There's a whole challenge around the roadmap for building next-generation computers, and people talk about that normally in terms of Moore’s Law. But that’s not really the major challenge—it’s really about energy density. Essentially, computer chips today are way too hot.

I saw Intel has a new 600-watt computer chip. Those are pretty wild numbers; you start to get to a point where you're getting close to fundamental limits on your ability to pull heat out of silicon. The problem is with the fundamental physics of the transistor; Nvidia’s CEO, Jensen Huang, talks about the challenge with heat in computer chips. So heat is the major challenge in energy consumption, and when you pair that with the growth in AI—and the kind of compute you need to power algorithms like GPT-3—it's clear that a new kind of technology is needed.

I think with current technologies, you’re going to see the opposite of democratization. The only people who will be able to run their neural nets will be the big software companies, places like Google, Amazon, and Facebook. I don’t want it to be a future where compute is inaccessible. There are only a few players that can really do this, and those machines using tons of energy is not something that I'm excited about.

Let’s get into the company’s namesake: light. How is Lightmatter’s approach different, compared to how most transistors work today?

The main fundamental challenge that we're dealing with is called Dennard scaling. And that main challenge is coming from transistors—the device that everyone uses and tiles out to build computers. They're so small now that they're kind of on the scale of the electron, and so they're just leaking all the time—they don’t really turn off. And that's what gives rise to this energy problem.

So we've developed a new type of compute element that is based on light—processing light, from lasers. Using that element, we're able to get around this fundamental technology challenge, this energy problem. And since light defines the speed limit in our universe, it’s the fastest thing out there.

So what we're able to do is to leverage light in a silicon platform, which is built in a standard complementary metal–oxide–semiconductor (CMOS) foundry to build these processors for AI.

Moving data around on the computer chip is a big energy suck. So let’s say you’re nano-sized and sitting on the chip: What are you going to experience, as far as sending data and photonics?

Stay up to date on emerging tech

Drones, automation, AI, and more. The technologies that will shape the future of business, all in one newsletter.

For sure. So Google Ventures is an investor in Lightmatter. And Google came up with the tensor processing unit (TPU). So our architecture looks a lot like Google's TPU chip. The data passing through it can be images, voice, or any kind of data format. What we’re doing is all of that calculation in the optical domain, rather than using electrical signals, so it happens at very high clock speeds. And it uses very little energy because when you move data around a computer chip, that's where a lot of the power dissipation happens. In our case, that’s all happening optically, which means big energy savings.

To put it another way: On an electronic chip, when you want to send a signal between points, you have to turn a wire off and on. And turning a wire off and on costs energy—you have to dissipate energy through resistance, inductance, and capacitance.

But with an optical wire, you don't have those properties. So turning it off and on takes very, very little energy, and you don't have time delays associated with that, which allows you to run the processor at very high clock speeds. So if you’re sitting on the chip, sending data is a lot less work. You’re spending less energy.

When do you realistically anticipate bringing the tech to market? What do the next five years look like as far as widespread adoption of Lightmatter’s chip?

This year, we just finished three rounds—we raised $80 million, bringing the total to $113 million. We're growing the team to support all the engineering work that goes into the software stack that runs on top of this processor, so that's a really big effort. We're also building out the go-to-market team. We recently announced hires like our VP of sales, who we hired from Graphcore, and our head of product, out of Uber. Essentially, we’re growing teams to support customers who will actually receive the hardware and may need support using it.

We're targeting data centers and on-premises computing. So we're looking at companies like Google Cloud, Microsoft Azure, Amazon AWS, and we have relationships there. We’re also looking at big retail companies and financial institutions, like trading.

So I think that our hardware is going to be deployed in these hyperscale data centers and supporting services like—maybe as an example, but not specifically—Siri, Alexa, products like that, and also Image Search. And generally, a lot of the search algorithms these companies are developing and using internally. So I think it's going to run a lot of the backbone for the internet.

Then, we hope to get into a spot where we're deploying it for applications of AI that are closer to the edge—so not like a cell phone-type product, but something that's receiving video feeds from hundreds of thousands of cameras running big AI algorithms on that.

Obviously photonics is faster, but would a consumer conducting a photonics-powered Google Search see an infinitesimally quicker result turnaround? Or would the speed lead to more accurate search results in the same amount of time—being able to parse through more colloquial terms to get the thing you’re actually looking for?

Yeah, it would speed it up, but I think the proper way to use it would be to do a much more advanced neural network in the same amount of time—to give you a lot more information that’s relevant. We're allowing them to operate cheaper, so it costs them less money to do each inference, which means they can run bigger models.

Another reality that people are dealing with right now is that AI processors are really expensive, and so for engineers at Google, Amazon, Facebook, and places like that, there are a lot of AI features that the product teams of these companies would like to roll out, but when they look at the return on investment, it costs too much money to deploy AI-specific hardware.

So if we can have more compute-per-unit-area and use less energy, then we can help make the case to their bosses that it's time to deploy the model that they were thinking about.

This interview has been edited for length and clarity.

Stay up to date on emerging tech

Drones, automation, AI, and more. The technologies that will shape the future of business, all in one newsletter.