LLMs in your business - where to start?

Posted on 7th March 2024.

LLM Cloud Image

If you are reading this then you might already be thinking that LLMs can help increase the productivity of your business. You have most probably used ChatGPT from OpenAI, and you may know that there are similar commercial products such as Google's Gemini, or Claude from Anthropic. In fact, you might know already that each of these commercial products has multiple flavours, for example GPT-3.5 vs. GPT-4 in the case of OpenAI, and similarly with Google and Anthropic that each have several levels of LLMs. Still, even if you have used such off the shelf commercial LLMs for ad-hoc tasks, you might be unsure how to integrate LLMs in your business in an effective and safe manner. You certainly might be wondering about the productivity increase and cost savings that your teams can gain.

This field is advancing at a very rapid pace, where literally every week there are new LLMs made available, new tools for combining and integrating LLMs, and new applications and success stories published. So if you are now trying to use it for your business for the first time, where should you start? How do you make sense of all the possibilities out there?

One way to make sense of it all is to first understand where you stand in terms of your business, the industry you are operating in, and the parameters that matter most to you. You probably already have a hunch that with LLMs you can achieve effective document summarization, effective data retrieval, text creation, or even build a chatbot for customer facing activities. You might also have some handle about the risks associated with data privacy and LLMs making mistakes. So at this point, you probably want to refine what it is that you are really trying to do, how important certain attributes are to you, what kinds of costs savings and productivity boosts you envision, and what the nature of the application that you have in mind is.

Here are a few dimensions that you may want to consider.

Cost - How much are you willing to pay? Costs can include subscription costs, development costs, infrastructure costs, and others.
Data privacy - In general, are you happy for your data to be uploaded to a provider, or is it important for you to have your data private on your infrastructure, or safe cloud services? Is it important that your data is not used to train commercial LLMs?
Customer facing vs. internal - Is your primary application an internal application where the staff in your business will use the LLM, or are you looking for a customer facing solution?
Hallucination toleration level - As you may know, LLMs can occasionally hallucinate and provide content or answers that are far fetched from reality. In your application(s), how often are you willing to tolerate hallucinations? Some applications can be mission critical and hallucinations need to be avoided at all costs. In other cases inaccuracies in responses can be tolerated much more.
In-house data - As you may know, without additional methods such as RAG, LLMs cannot provide answers based on data that they have not seen. For example, if your application requires access to a set of internal documents from your business, you may need to tailor a solution for that purpose. Does your application require use of such in-house data, or do you think that general knowledge on which the LLM is trained is enough?
Novelty of your application - LLMs are already employed for hundreds if not thousands of use cases. Do you think that elsewhere in the world, the application that you have in mind has already received treatment via an LLM based solution? If so, then finding the most cost-effective similar solution, or following paradigms employed elsewhere, is probably best. If not, then you may need to spend more time refining your requirements and design.
Modality - At the moment LLMs thrive on text (including computer code and multiple natural languages), yet support for other modalities such as audio, images, video, equations, tables, and charts is possible but is not as mature. Does your application mostly involve text or does it involve other modalities as well?

As you think about each of the above dimensions, you are probably thinking about the tasks that staff in your business currently handle without LLMs and then you might think about how LLMs can help. One approach which you may take is to engage with the workflow of various employees and brainstorm with them about how they think LLMs can help with what they do. You may obviously have multiple applications in mind, and multiple goals. For each case, consider the above 7 dimensions and determine where you stand.

In planning applications, starting with simple free online test access to ChatGPT, Gemini, or Claude can be a starting point where you and your staff can experiment with what is possible. Then you can further refine your needs, define your application(s), and answer some of the questions posed above.

LLM Cloud Image

As a final point, you should already be aware there are plenty of open source LLMs that you can run locally on your own machine or managed cloud infrastructure. These include the LLAMA2 models from Meta, the incredible Mistral models, and several others. Using such models with the right setup can help you gain the goals that you need, often in a cost effective and safe manner. A slight challenge is that to do so, you need to learn how to deploy them, and how to manage them within your business domain. After reasoning about 1--7 above, investigating if to go with an a custom open sourced based solution, or using a proprietary off the shelf solution, might be your next step.