Power up. From supercharging employee productivity and streamlining processes to inspiring innovation, Microsoft’s AI is designed to help you build the next big thing. No matter where you're starting, push what's possible and build your way with Azure's industry-leading AI. Check it out.
To go open or closed?
It’s one of the key questions businesses face when building new LLM-based applications. Whether enterprise developers choose to build around open models or plug into proprietary APIs can result in trade-offs around customization, cost, security, and performance.
At New York Tech Week in IBM’s Manhattan offices last week, a panel of experts talked about some of the considerations for businesses building on open models like Meta’s Llama family and Google’s Gemma. The AI Alliance, an IBM- and Meta-backed group that promotes open-source AI, hosted the event.
The panel began with an exchange of definitions; this is important because there’s no fully agreed-upon standard for what is or isn’t an open-source model, though the Open Source Initiative has offered one up. It depends on which of the three broad components—model weights, source code, or training datasets—developers make available.
When asked what type of systems they worked with, a show of hands failed to result in an obvious front-runner—sizable portions of the audience indicated support for open, closed, and hybrid models.
Tailored models: Customization was a big theme for why businesses might choose open-source or some sort of hybrid model stack. Anthony Annunziata, director of AI open innovation at IBM, said organizations may turn to open-source models as a way to customize trade-offs as they scale an AI application from proof of concept to full deployment.
“I think open-source has a great opportunity here because you can engineer the system the way you want and to have a certain cost profile,” Annunziata said on stage. “You can optimize the size of the model and the optimizations that the model itself has to balance accuracy and cost. All that’s possible with open-source tech, especially with open-weight models; it’s really not possible with the closed approach.”
Evaluation needed: There’s no one-size-fits-all answer for businesses, though, according to Rebecca Qian, co-founder and CTO of AI evaluation platform Patronus AI. Qian said hybrid approaches are on the rise among Patronus’ clients, and she works with customers to decide which model best serves their purposes.
One such customer is Volkswagen, which Qian said is building automotive AI applications. Volkswagen didn’t care as much about standard benchmarks as whether a given model could meet its needs, and Qian said Patronus assembled a custom automotive industry-specific benchmark.
“Volkswagen came to us and said, ‘I don’t really care about my AI doing well on middle school math, but I care about it answering questions about my car models in different major European and global languages,’” Qian said in the panel.
While out-of-the-box closed models typically perform better than usually smaller open models, Qian said she believes small language models (SLMs) fine-tuned on task-specific data can always outperform generalized models on those tasks.
“Of course, the caveat is, like, the data that you curate to do that fine-tuning or the tasks being constrained and well-defined,” Qian said. “So with some caveats, but that is what we believe, and that is what we’ve seen as well.”
Compliance concerns: One other major factor when choosing between open and closed models is vetting proprietary vendors for regulation compliance. Open-source can require more investment in compliance—a business must check compliance for itself rather than relying on vendor certifications—but can be worth it in the long term, especially for companies in strictly regulated sectors, according to Yacine Jernite, head of machine learning and society at Hugging Face.
“In many, many cases, we found that—[we are] obviously biased [toward open-source]—that it is worth it if you’re going to…have your business life depend on that compliance, to do it for yourself,” Jernite said.