What You Can Do With a Private Business LLM

Posted on 2nd April 2024.

By now it is obvious that LLMs are very powerful tools that bring plenty of value through tasks such as summarization, knowledge retrieval, creative content creation, and certain reasoning tasks. In our previous blog posts, we outlined popular commercial LLMs and open LLMs, yet none of those LLMs are private business LLMs tailored specifically to the needs of your business.

There are often cases where you have a requirement to host an LLM on your own infrastructure, for instance because you cannot upload private business data to third-party commercial systems, or becuase you the LLM needs access to your own knowledge bases. These are some of the most common cases where you may need a private business LLM.

In this post we discuss what you can expect from an LLM that is tailored to your needs, namely a private business LLM. Specifically we outline some of the possible use cases of such a system. To better understand the landscape, especialy if you haven't thought much about the LLM space yet, we also recommended reading our post on where to start with LLMs.

What is a private business LLM?

In a nutshell we define a private business LLM as a system where proprietary documents and data of your business are securely exposed to an internally available LLM chatbot. This LLM chatbot can be used by your stakeholders to securely ask questions and carry out language based tasks, all informed by this business data. The stakeholders can for instance be your employees, directors, existing customers, or prospective customers.

A private business LLM consists of three main parts.

1. An LLM that serves as the engine for reasoning and language processing. This LLM already has world knowledge, high language ability, and an ability to execute summarization and basic reasoning tasks. The LLM runs on your infrastructure and unless you want it to, it does not communicate with the outside world.
2. Your proprietary or private business data. This can include customer data, internal documents, or public documents that the LLM was not trained on.
3. A system that ties 1 and 2 together with a user interface, including integrations to platforms used by stakeholders.

Technologically, the most common way to implement a private business LLM is based on Retrieval Augmented Generation (RAG). Yet combining prompting, fine-tuning and other approaches can sometimes yield sufficient results. Our focus in this post is not on the technological details per-se, but rather on the business use cases. Future blog posts will deal with specific technological aspects.

The most basic user interface for a private business LLM might feel just like a simple chat engine. That is, it runs in a web browser or another application and you type in text and upload documents, and you get text and documents back. Other more specific interfaces can also be created, and you may integrate the private business LLM in your own applications that are either customer facing or internal. This is especially important for the generative applications where you often would like your private business LLM to generate MS Word documents or similar.

The Use Cases of a Private Business LLM

There are hundreds of specific possible use cases for private business LLMs. To make things simple we categorize them according to three main categories: resource query engines, generative engines, and advanced chatbots. In each category some of the use cases are internal facing (i.e., employees of your business use them) while other use cases are customer facing.

It is however important to keep in mind that LLMs are not yet quite there to replace humans for most tasks. This however, does not mean that they cannot yield incredible productivity boosts. Most currently feasible applications (except some chatbot uses) work by having a human in the loop, selecting and tailoring output from a private business LLM. This might invovle the LLM producing multiple possible outputs and a human expert selecting the most appropriate (or in a rare case writing one themselves). This can make the human expert orders of magnitude more efficient, while simultaneously generating valuable training data that can be further used to fine-tune such systems.

We now outline three categories and subtypes of possible private business LLMs.

Resource Query Engines

We can think of resource query engines as LLMs that pull from your business's private data, open data, or both. These products provide your stakeholders with an ability to query specialized, proprietary, and private resources.

Here are a few example variants of resource query engines. Note that often one private business LLM can provide a few of these variants in one product:

1. Query docs (customer or internal facing): With this private business LLM you get a conversational agent that allows you to ask specific questions about your (external or internal) documentation. This is useful if your internal or external products have extensive reference documentation.
2. Query regulatory documents (customer or internal facing): With this private business LLM you get a conversational interface that allows querying regulatory documents. It is useful for following rules and regulations that are complex, detailed, an vary by domain jurisdiction.
3. Query internal contracts: Similar to the above, this private business LLM allows you to query contracts, especially those involving your own business.
4. Query technical specification sheets: With this private business LLM, you get a conversational agent that allows you to ask detailed questions about product specification sheets. For example if your business uses thousands of sub-products, each can have its detailed specifications and you can query across the sub-product collection or about specific sub-products.
5. Query sensitive company data: With this private business LLM, managers can get a bird's eye data-scientific view of company data, directly by asking the LLM. The private business LLM has access to internal databases and documents and is able to generate reports directly based on managerial queries.
6. Query market data and business intelligence: With this private business LLM, the LLM seeks to collect information about competitors, clients, and the general business space where you are operating. You can then query information about your competitors and clients (based on publicly available data or internal intelligence generated by your company).

Generative Engines

The generative engine form of a private business LLM allows to automate and improve some writing tasks in your business. This is especially important if your employees spend much time and resources on writing. Here are some variants of such a product:

1. Generative tender submission creation system: This private business LLM knows the details of your business, your strengths, your market edge, and other details. With this information, based on basic input, the LLM helps you draft and refine tender proposals.
2. Generative statement of work creation system: Similar to the above, this private business LLM generates statement of work proposals for your customers.
3. Generative work summary creation system: This variant of a private business LLM generates work summary statements for your customers. This is useful in cases where your business requires generation of large documents as part of a work summary. For example, inspection reports, or other conclusions.
4. Generative customer specific marketing system: This variant of a private business LLM generates marketing information for your customers in a specific manner based on sales person intervention, or company internal databases.

In almost all cases you would still require a person in the loop, operating such generative engines. Nevertheless, the productivity of this person can increase considerably and as a consequence you can cut costs.

Advanced Chatbots

Chatbots have been around for a while and many customers are aware of their abilities and limitations. In older settings, a chatbot often begins the conversation before handing it off to a human operator. Now in the age of LLMs, advanced chatbots are able to deliver much more value by taking conversations all the way to completion, only requiring costly human intervention in a small number of the cases. Here are some variants:

1. Customer sales chatbot: A customer sales chatbot may answer queries on your business website and engage with customers towards a sale, answering frequently asked questions and guiding your potential customers to more information. In contrast to classic chatbots that just answer questions, such advanced chatbots learn about your potential new customer during the engagement and offer compelling help that may drive monetization.
2. Customer account support chatbot: A customer account support chatbot knows information about the (existing) customer that is already engaging with the chatbot. It provides the customer private information about their account (after verification) and yields a seamless friendly experience for customers.
3. Customer technical support chatbot: Similar to a customer account support chatbot, a technical support chatbot provides customers with technical support, specifically tailored to their configuration, setup, and needs.
4. Chatbot as part of a product: Some products may need a chatbot as part of the product. Such a chatbot is aware of the state of the system and the customer, and integrates within the product. User facing consumer products may benefit from very friendly and customizable chatbots. Other products may need a more technical chatbot.
5. Supporting chatbot for customer support: This is a system that helps the people providing customer or technical support in rendering this support. In contrast to the customer account/technical support chatbots, which are used by the customer directly, this is a tool that helps your customer service agents be more productive. This kind of system can also be used to suggest improvements for the customer represntative based on conversational (either voice or text) patterns. Another possibility is for the tool to suggest a few different AI generated responses, allowing the user to pick the most appropriate or write their own.

Combining aspects of the three

A private business LLM is not restricted to one category or use case. For instance, if your firm writes a lot of tender submissions or other proposals, you might want a private business LLM that is able to query technical documents about the entity you are writing the tender/proposal for, while simultaneously being tailored to your company's strengths and specialization to generate submission text, all while pulling in information from your databases about figures and numbers related to the task. Such systems still require a person to operate the engine, but can significantly speed up and enhance such a common yet complicated task, freeing up more of your employees' time to do more important things.

Where to from here?

We have presented a multiple variants of private business LLMs which we categorized as resource query engines, generative engines, and advanced chatbots. If you think that any of these variants may add value to your business then your next step would be to discuss with your IT team or LLM specialists for understanding the costs, potential value, and challenges with incorporating such solutions in your business.

Most importantly, you may want to engage staff from your business to understand how they think such solutions could be helpful or not. Interestingly, it is becoming a common phenomenon for employees to use commercial LLMs in an ad-hoc manner. This may cause a data security problem when you do not control what your employees paste in a commercial LLM's chat box. If LLMs provide enough value and a big enough productivity boost for your employees, it will be hard to stop them from utilizing such services. To understand more about this phenomenon, we recommend this Harvard business review article which summarizes a study of how some people are currently using LLMs.

© 2018 – 2024 Accumulation Point Pty Ltd. All rights reserved.