5 Top Product Applications of LLMs

Jessica Yao
Jul 26, 2023

Natural language is familiar and flexible — it’s always been the interface between people and people. Now, with the recent developments in large language models (LLMs), we are finally entering a world where it can be the interface between people and machines.

When LLMs were first launched, their primary UI was a chatbot. Though many 3rd party apps continue to provide their own customized version of this, developer-friendly LLM APIs have resulted in new interaction patterns. You may have seen many of your go-to apps displaying a new “Generate with AI ” button (sparkle emoji prefix and all).

UIs aside, when a lot of companies say they’re using generative AI — what are they actually using it for? In this post, we highlight the five main ways we’ve seen LLMs integrated into products, including ten real-world examples.

A Brief Overview of NLP Tasks

Many products we’ve seen have integrated a variation/combination of natural language processing (NLP) tasks. After being trained on so much data, LLMs capture many of the patterns inherent in language — and consequently excel at these tasks.

When it comes to NLP, all tasks can be boiled down to some sort of “transformation”, where information is changed from one form to another. Each of these transformations lies somewhere on the spectrum of “reducing the input” to “enriching the input”:

Most approaches we’ve seen fall into the following five categories, which we’ll cover in this post:

1. Information Retrieval/Search

If you’ve used Google (or any other search engine), you’re likely familiar with information retrieval and search. It can refer to:

  1. Finding information in a single given document
  2. Finding the correct document (or excerpt) across a larger corpus of documents

LLMs change the picture like so:


Dropbox is a file storage product — it’s in a perfect strategic position to allow users to search/query their own private docs (PDFs, powerpoints, text files, etc).

The AI-enhanced version of their Dropbox Dash product provides a natural-language search functionality, and also specifically cites which files the answer is derived from. For instance, if your company is storing all of its artifacts in Dropbox, it becomes very easy to ask company-specific questions:

CommandBar (HelpHub)

CommandBar’s HelpHub product allows companies with public-facing websites to easily create a chatbot interface to answer their customers’ questions.

To set up this workflow, you first specify which webpages should be used as input sources for your particular HelpHub bot:


You can then either (1) issue a natural language query to get a list of relevant search results or (2) interact with a chatbot that automatically synthesizes the source into an answer.

(See the publicly available demo of a Strava HelpHub integration)

2. Summarization

Summarization is another widespread application of LLMs. You can think of “summarization” as a task where a larger chunk of text is condensed into shorter form, while preserving the most important information.

The following applications are all some variation of this:

  • Outlining a document
  • Chunking a meeting transcript into notes
  • Splitting a detailed tech spec into tickets
  • Classification – bucketing text snippets (e.g. customer reviews) as “positive” or “negative” or otherwise applying some relevant tag


Notion is a productivity/note-taking app that has integrated AI capabilities into its editor. Among other capabilities that it’s wrapped with shortcut buttons, it allows you to “summarize” a chunk of text

For example, here is some source text:

When summarizing, NotionAI provides the following:

When generating an outline, NotionAI provides the following:


Superpowered is an AI notetaker for meetings — in addition to using a speech-to-text model to transcribe what was said during a meeting, it also passes this text through an LLM to extract (1) a summary and (2) any potential action items

An example of possible meeting output

3. Structured Translation (for Machine APIs)

One particularly effective application of LLMs is being able to easily transform natural language into structured form. In the past, user-machine interactions were limited by the fact that machine interfaces expect structured input like JSON or SQL. LLMs give your app’s users the flexibility to ask for what they want in plain English (or any other natural language).

This significantly reduces the learning curve! Your users are much more likely to get up and running with your app’s core functionality, rather than getting stuck reading docs. You can now provide:

  • More intuitive query bars. Without having to change their existing structured query interface, any given app can put a layer of “natural language” on top. Users can now type a single question instead of filling out a series of input fields, applying filters, or having to learn app-specific query syntax
  • Cross-API orchestration. There are also quite a few new apps where the LLM model serves as the core “workflow glue” between various 3rd party APIs. As an analogy – a secretary might receive a high-level goal from their boss (“organize a client dinner”), and accomplish this by breaking it down into discrete steps, some of which involve reaching out to other folks (e.g. messaging clients, calling a caterer, etc). Similarly, LLMs can chunk out a natural-language directive into smaller parts, calling out to APIs to either fetch information it needs or execute an action in some external system.


Booking.com is a travel search engine that allows people to find the best deals on flights, hotels, and rental cars. Normally, users have to enter their itinerary into a series of input fields:

With an LLM-integrated chatbot, users can ask about their itinerary, which is likely handled as follows:

  1. Raw user input is wrapped in a more sophisticated prompt, where ChatGPT’s native language abilities can restructure it into a known format (e.g. JSON with fields `origin`, `destination`, etc).
  2. This postprocessed, structured input is then issued to existing Booking.com API
  3. A prompt wrapper also instructs ChatGPT how to handle the resulting response (which follows some known format).


Linear is a work-management app that allows users to create and track tasks. To find specific subsets of tasks, users often apply a series of filters. However, these filters are not always obvious to new users, and can become tedious to reapply.

One way around this is to have saved filters – but why not improve the querying experience from the get go? Linear has implemented AI filters that translate an English query like “open bugs with SLAs” into the filters that their original API expects

What the user types in:

“open bugs with SLAs”

Which the LLM will restructure into the following:

4. Unstructured Translation (for Human Users)

Translation is typically thought of as going from one language to another (e.g. English to Spanish) – and LLMs are indeed capable of doing this.

However, LLMs are also capable to “translating” in the following sense:

  • Proofreading
  • Rewriting a passage with a different tone, or for a different target audience
  • Explaining code as English (or vice versa)
  • Refactoring code


Mutiny is a no-code app that enables non-technical users to update their company marketing websites (and to observe the effects via A/B testing).

They’ve integrated AI into their website editor, so that users can select specific text, and request that it be rewritten “friendlier” or “more aimed towards clients in financial services”.


Replit is a collaborative browser-based IDE, with a recent AI feature that allows users to ask questions about their code, refactor it, debug it, and more. It’s based on OpenAI’s Codex model, which was specifically trained on code (as opposed to ChatGPT, which was trained on natural language).

Explaining code:


Refactoring code:

5. Text Generation

Pre-LLM algorithms could still achieve the previous NLP tasks (“reduction” and “translation”) to some extent, even if the results weren’t as high quality.

LLMs really unlocked the ability for text generation — adding large amounts of text where there was none before. You can give the LLM a high-level seed, and ask it to brainstorm on your behalf (e.g. “Give me five different titles for this blog post”).

The possibility space is quite wide, especially if you take into account that:

  • “Output text” can represent more than just natural language, but also HTML, UML diagram syntax, etc — meaning that the output can be visual
  • Generation doesn’t have to be one-shot — it can happen in a interactive, recursive way (e.g. “Give me five options”, “Now for option X, break it down into main sections”, “Now for the each of these sections, write a short paragraph”)


Jasper AI is a copywriting tool targeted towards marketers, which uses AI to easily generate  blog posts, product descriptions, social media captions, etc.

Giving the user an entirely blank text box is overwhelming, and more likely to require several iterations of trial/error to get what the user desires. Instead, Jasper’s UI “binds” the space of what the LLM can generate by having the user specify things (e.g. intended use case, product name, tone, etc) — wrapping this all up in a separate prompt to the LLM behind the scenes. This allows Jasper a lot more control, quality, and consistency when it comes to the auto-generated text.


Tome is an app that allows you to use AI to generate and refine full presentation decks from a single input (e.g. “create a presentation about X”). Their straightforward prompt likely abstracts out a bunch of different calls from the user (“Make an outline about X”, “Write a paragraph for each section detailed in the outline”) as well as some potential DALL-E (text to image) integration.

For instance, you can input:

Which results in the following outline:

Each section of which gets a slide:

The user can tweak this initial scaffolding in subsequent iteration loops, using other AI features like “rewrite”.


Hopefully, these examples have given you some sense of how to use LLMs in your own product. We encourage you to experiment with integrating LLMs into your product if you haven’t already— we’re excited to see what you build!