How startups are turning generative AI into workplace tools

Ever since ChatGPT first catalyzed a frenzy around language-generating AI last fall, prognosticators have imagined the technology’s place in a hyper-efficient, busywork-free office of the future.

By one recent account, AI could automate the tasks that take up 60% to 70% of a typical office worker’s day. But how does that vision square with the tech’s tendency to make things up? And where might generative AI actually be useful in the course of a daily routine?

In continuation of our series on workplace AI (part one focused on venture capitalists), we’re looking at how some enterprise-focused startups are attempting to mold large language models (LLMs) into workplace tools. Companies see the technology overhauling office communication, organizing internal documents, and augmenting coding teams, among other use cases.

Grammarly, which has been developing language-based AI since well before the current wave took hold, frames its new generative text tools as a way to use a customized AI to draft emails, rewrite a message in a different tone, or manage other pieces of office communication. The startup advertises its GrammarlyGo LLM-powered writing assistant as being able to tailor to a business’s or worker’s voice, style, and role.

Founded in 2009, Grammarly has raised $400 million to date, according to Crunchbase.

Grammarly CEO Rahul Roy-Chowdhury said at an event earlier this year that as written communication has proliferated during the pandemic, it has also grown less effective, according to a survey the company conducted with the Harris Poll.

Roy-Chowdhury said he believes that Grammarly’s experience iterating from rule-based communication software to present-day generative tech gives it the data required to make LLMs more useful in context-specific settings.

“With this new AI-enabled workplace, Grammarly is going to help you by understanding all the knowledge you need to know to be able to communicate effectively,” Roy-Chowdhury said.

The goal is to save time on misunderstandings and wasted messages by giving Grammarly's AI directives like “shorten it,” “improve it,” or “make it sound more professional.”

“You need a really good retrieval system”

Other startups also see opportunity in using LLMs to help navigate specific situations. The startup Glean aims to provide a centralized repository that workers can sift through via conversational chatbot.

In a marketplace swimming in AI-powered productivity tools, Glean CEO Arvind Jain said part of his company’s advantage is its ability to work across common enterprise programs from vendors like Microsoft, Salesforce, Google, and Atlassian to summon and synthesize internal information.

“The core foundation of how you actually bring the power of LLMs into the enterprise is you need a really good retrieval system,” Jain told Tech Brew.

“The state-of-the-art technical architecture for leveraging enterprise LLMs is to build a system where you complement the reasoning powers of the LLMs with a knowledge engine, which is what you use to actually…answer questions that people have. And building that knowledge engine is what Glean’s strength is.”

Mrinal Mohit, a Glean software engineer, said that by directing the LLM to ground its answers only in information the software has on hand, the AI is better equipped to avoid the problem of fabricated information, or hallucinations, that often plagues LLMs.

With the addition of AI to its existing workplace search engine, the company claims clients can ask the tool to do things like summarize the status of a project or gather documents to help onboard a specific role, with the AI drawing from different internal resources to formulate an answer. The startup’s clients have included Databricks, Duolingo, and Grammarly, and it’s raised $155 million to date, according to Crunchbase.

Building a “more precise” LLM

Productboard, a product management platform, recently rolled out a generative AI tool  designed to suggest new product feature ideas, extract important snippets of customer feedback, and otherwise navigate the data on the platform.

Hubert Palan, co-founder and CEO of Productboard, said he sees the AI as being more of a guide to point product managers in the right direction within the platform’s visual interface, rather than a solely text-based format. Productboard has raised $262 million to date, according to Crunchbase, and has counted Autodesk, Salesforce, Kroeger, and Volkswagen as clients.

Like Glean, Palan said structuring the system so that it only relies on information within the program can help improve the accuracy.

“I’m excited to see how the LLMs are going to get more precise and better,” Palan said. “In a structured system like ours, if you get an answer—I mean, the product manager should critically evaluate what the AI is saying—but if you get links to the underlying structured data, and you can validate it, then you can go and read the insights, and you can kind of make sure that the suggestion is right, then, that’s a little mitigated.”

Companies like these have taken this tack as valuations and acquisitions in generative AI continue to be a bright spot in an otherwise flailing startup funding market. But Constellation Research VP and Principal Analyst Andy Thurai said it’s only a matter of time before market activity calms down again.

“This craziness of valuation and M&A will continue as the pendulum swings hard before it settles down,” Thurai said in an email. “Eventually, it might settle down, but right now it is red-hot.”