Announcing Lamini Classifier Agent Toolkit

There’s so much data hitting your organization every day. However, only a portion of that data is highly urgent, important, or actionable which makes extracting signal from noise challenging for any large organization. Large Language Models (LLMs) have transformed this process which makes your data even more valuable and useful than ever. For developers, LLMs reduce the need for extensive coding or manual labeling of messy unstructured data. With LLMs, you can simply make a call to the model and let it do the heavy lifting.

Today, we’re excited to announce our new Classifier Agent Toolkit (CAT), to help organizations accurately classify data at scale and make better decisions faster. With the Classifier Agent Toolkit, you can quickly categorize a large volume of content based on intent, sentiment, severity, and more, and finally replace manual and error-prone human labeling tasks with high-accuracy automation. 

We’ve worked closely with our customers to help them achieve 9s of accuracy in just a matter of days on their classification tasks. Copy.AI had a particularly gnarly task involving a whopping 1,000 distinct classes. We helped them cut manual classification time by 75% (1,200 hours of work annually) and achieve 99.9% accuracy. 

Beyond text classification, there are many other high impact ways to use this toolkit:

• Determining severity of a support ticket so support teams can prioritize urgent issues
• Analyzing sentiment of product reviews, social media posts, customer surveys, earnings calls, and more
• Inferring user intent to return the correct response
• Triaging legacy application code based on importance

What would the impact to your business be if you could quickly and accurately triage incoming data, detect and prioritize urgent issues, and reduce inefficient manual work?

Current approaches to classification and their challenges

Categorizing and prioritizing a large volume of real-time data is a persistent challenge. Traditionally, businesses have relied on human labeling, for example, relying on customer support representatives to manually tag each ticket. The limitations of this approach are: 

• You can only scale linearly
• Prone to errors and inconsistency, particularly with large taxonomies
• Inaccuracy creates distrust in the system so urgent issues may be ignored
• In some tasks, you need experts to do this labeling

Traditional ML classifiers promised automation but came with their own challenges:

• Months of development time 
• Extensive, manually labeled datasets
• High upfront investment
• Inflexible to changing data distributions, as you mix complex data sources together

Enter LLMs. LLM-based classifiers have significant advantages over other approaches: they are faster to develop and require less data preparation so companies are able to quickly build prototypes using models like GPT. While LLMs are great for general classification tasks, they lack deep domain expertise to make them truly useful in most enterprise applications. Limitations include:

• Only can handle 20-30 categories, but not 100s or 1000s of them in a complex taxonomy
• High latency and low throughput – takes several seconds for one page
• Can be prohibitively expensive for some applications, when you move to production
• Inconsistent, unreliable outputs (hallucinations on your real, proprietary data) 
• Low accuracy, often below 90% or even close to 0% on more categories – this inaccuracy can create business risk

We’ve worked closely with organizations across industries to develop a toolkit that addresses the critical challenges of large-scale data classification using open LLMs.

Our solution delivers high accuracy, at a fraction of the time and cost of other approaches — and is an important, easy plugin to any agentic workflow. Whether you have two classes or 1,000 you can expect the same high accuracy and throughput with exceptionally low latency as illustrated below.

We’re now making this workflow accessible to every developer and we’re excited to hear your feedback. 

Classifier Agent Toolkit isn’t like regular LLM finetuning: it makes tuning for this agentic use case straightforward and it works for software devs and AI engineers alike. We understand that not every team is quite ready for fine-tuning. That's why we've developed a solution that bridges the gap between general purpose LLMs and custom SLMs—a faster path to intelligent, accurate, and cost-effective classification. If you want to learn more about fine-tuning LLMs to become domain experts, check out our Enterprise Guide to Fine-Tuning whitepaper.

How the Classifier Agent Toolkit works

Consider a common enterprise challenge: scaling customer service teams that are overburdened with repetitive support requests. AI chatbots have become ubiquitous, but offer only a partial solution—gathering customer info, creating a ticket, and resolving the easiest requests, but leaving a long tail of requests that end up getting escalated to a human. In the worst case scenario, chatbots have been known to hallucinate answers eroding user trust in the system. To unlock the highest ROI from these systems, a higher degree of accuracy is needed. 

Getting started is simple. All you need is:

• A list of categories
• A few examples for each category

Then, just follow the streamlined workflow:

1. Create a new project. 

2. Select your base model, add your categories, and create a few examples for each class. When you click "Create Project", the model will be trained on the categories and examples you entered.

3. Choose “Create New Eval Data”.

4. Create an evaluation data set and specify the inputs and targets. 

5. Run the evaluation and get visibility into overall results as well as individual results. You can download these results to a CSV file. 

6. Continue iterating by adding additional examples until you reach your desired level of accuracy.

7. Now you can easily compare results between datasets. As your model approaches 100%, feel free to create a new evaluation set to understand how the model is doing in tougher cases.

And that’s it! You’ve created your first Classifier Agent!

Get started

Watch our short demo. If you’d like to try the Classifier Agent Toolkit yourself, check out our recipe repo for examples. We’re offering $300 in free credit for Lamini On-Demand so you can try it out at no risk. If you’re interested in a demo, please contact us here. 

Additional Resources

• Watch the demo
• Read the Classifier Agent Toolkit Docs
• Check out the Classifier Agent Toolkit Repo for examples
• Get started with $300 free credit

‍