Blog Tidepool

Our continued partnership with Snowflake makes it even easier for analytics teams to derive valuable insights from production scale text datasets that they have stored in Snowflake.

Excited to make it easier for teams on Snowflake to unlock insights in their text data!

Get started with analysis of your text within minutes and automatically sync Tidepool’s outputs back to your data warehouse, all without storing sensitive data outside of your cloud environment.

Providing examples can greatly improve your LLM prompts, but did you know that the specific choice of examples can dramatically impact performance? 

LLM-powered applications, particularly those with chat interfaces, appear to have significant problems with user churn. For a multitude of reasons, including not understanding how to use the app, getting bad or unreliable results, or simply finding using the app too tedious to use, users often abandon these tools nearly as quickly as they came. Knowledge workers are tired of juggling multiple apps, and are looking for ways to pare down their tool stack to the bare minimum needed to get their work done.

Something like 90% of company data is unstructured data. Unstructured text data can include emails, messaging conversations, user generated content (like surveys, feedback, reviews, etc.), or user prompts to LLM apps.

At Tidepool, one way we help users interpret their text datasets is by surfacing interesting patterns in their data. There are many techniques for doing this, such as clustering or network analysis, but the easiest way to get started is just asking an LLM to do it. While you lose some explainability and scalability, you can get something working in no time, with easy zero-shot steering in case you have specific ways you’d like to interpret the data. Compared to other methods for topic modeling, using a generative LLM approach is also better at producing topics that are semantically meaningful to your data and other explicit domain constraints.

Anyone in software engineering knows by now that being able to learn and get up to speed on new topics, libraries, or languages is a critical skill, both for onboarding onto a new team, as well as for continuously providing value on an existing team.

In discussions around improving the performance of Retrieval Augmented Generation (RAG) systems, a lot of focus is given to the retrieval step. Embedding model selection, vector databases, chunking strategies, distance functions, reranking, etc., all factor into retrieving the most relevant documents.

Say you’re Walmart, trying to build an LLM-powered customer support tool. You want the LLM’s answers to take into account things like past orders, product descriptions, corporate brand guides, and Walmart’s documentation. To do this, you need RAG.

You can’t scroll through Twitter these days without coming across a new open-source LLM evaluation framework, guide, or startup; it’s clearly a topic on people’s minds. Now that the first flush of LLM hype has faded, people who are deploying LLMs in production are struggling to answer basic practical questions.

As large language models (LLMs) like GPT-4 become more widely used, companies are integrating them into customer-facing products and relying on them for business-critical applications. This presents a challenge: how can you systematically evaluate where your LLM is performing well or poorly?

One of the most intriguing LLM use cases is querying databases in natural language. Imagine that you’re a non-technical person, say an ops manager at Citibike, and you want some quick answers to questions like, “Where is there congestion in our bike network at 3 p.m. on a Tuesday?” In the past, you would have to spend two hours finding the right tables in the data warehouse and another half an hour painstakingly constructing a complex SQL query. But in the near future, with LLMs, you’ll be able to type in that question in your data warehouse and get a correct, natural language answer back.

This past month, I’ve been hitting up the AI meetup circuit as we look to market our new ML product, Tidepool. Tidepool is a product analytics tool for LLM text interfaces, and it’s really most useful for companies building LLM products that already have users. (If you don’t have users submitting queries, there’s nothing for us to analyze).

Charlene Chambliss is a senior software engineer at Aquarium Learning, where she’s working on tooling to help ML teams improve their model performance by improving their data. In addition to being an incredible engineer with an inspiring backstory, Charlene previously worked on NLP applications at Primer.AI. In this blog post, we interview Charlene about her experiences working with older models like BERT, and the perspective this gives her on the more recent wave of generative, RLHF-based LLMs (e.g. GPT-4 and LLaMA).

This post is targeted towards folks focused on building LLM (Large Language Model) applications (as opposed to research).

If you’re working on a software product (consumer-facing or SaaS), you’ve probably witnessed the recent explosion of AI-powered features. Now that AI has become an API, a lot is being built very quickly, and there’s an ambient sense of possibility in the space. (Seriously, try keeping up with ML Twitter!)

Natural language is familiar and flexible — it’s always been the interface between people and people. Now, with the recent developments in large language models (LLMs), we are finally entering a world where it can be the interface between people and machines.

The Aquarium team is thrilled to launch our new product, Tidepool! Tidepool does product analytics for AI text interfaces. With Tidepool, product teams can find patterns in their user text interactions to help make better product decisions.