With the rising popularity of Large Language Models for use in daily tasks, discovering and testing applications for use in Legal functions, like contract negotiations, becomes increasingly important.

What are Large Language Models?

Large Language Models (LLMs), like OpenAI’s GPT series or Google’s PaLM 2, are AI models that work with language and have been trained on expansive datasets that include a large variety of texts from the internet. 

These models leverage machine learning, specifically deep learning, to understand and generate language. They operate by internalizing the syntactic, semantic, and pragmatic aspects of language, essentially learning from the context of the words and sentences they have been exposed to during training.

As a result, LLMs can generate logical and contextually relevant text based on a given input, making them adept at a variety of language-based tasks.

With every new iteration, these models have demonstrated remarkable improvements. As LLMs continue to advance, potential applications for more complex and contextually accurate results will likely increase across various domains, including the legal field. Utilizing LLMs to assist with legal documents—for tasks such as provision mutualization—is one potential application.

Provision Mutualization in Legal Documents

Provision mutualization is a concept in legal drafting where two or more parties share similar obligations or benefits under a contract. Mutual provisions are common in contracts and aim to ensure a fair and balanced agreement between the parties.

For instance, consider a non-disclosure agreement (NDA) where both parties agree to keep certain information confidential. Instead of having separate confidentiality clauses for each party, the agreement can include a mutual confidentiality clause, where both parties are equally obligated to maintain confidentiality.

Another example of a term that might be made mutual:

Unilateral provision

The Company may terminate the agreement at any time.

Mutual provision

Either party may terminate the agreement at any time.

Benefits of Using LLMs for Mutualization

Provision mutualization is an excellent example of how the application of LLMs can provide value in legal work. There are several benefits from utilizing a Large Language Model for editing legal documents in this way.

Scalability of Automating Legal Processes

LLMs can analyze and modify large quantities of legal language much faster than humans, reducing the time taken to add mutualized language to contracts. These models are well-suited to large projects and recurring requests, and they are not subject to burnout. In turn, LLMs are equipped to handle large volumes quickly and are highly scalable.

Whether you give the model one task or hundreds, an LLM can quickly take an input containing unilateral language about obligations or benefits that need to be shared between the parties and return revisions in mutualized language. This is a strength that LLMs hold over other forms of automation, which may struggle to complete such specific language-based tasks. Automating this step with an LLM can speed up the legal process, allowing deals to flow faster and businesses to grow through accelerated contracting cycles.

Enhancing Human Power

Having been trained on large volumes of text, LLMs can help maintain the consistency of language and terminology throughout a legal document and can check for errors or omissions that a human reviewer may have missed. Automating provision mutualization and contract reviews not only helps reduce human error, it can assist in onboarding team members to company policies and legal positions much quicker, ultimately reducing onboarding time and cost.

By taking repetitive work off of legal professionals’ plates, teams have more capacity to focus on high-value strategic work while ensuring that their company’s preferred positions are applied consistently.

Mitigating Legal Risks

When adapting unilateral provisions to mutual language, it is important to ensure the language accurately reflects the desired outcome and that every necessary provision is captured and updated.

Pairing an LLM that generates mutualized provisions from unilateral inputs with other AI tools, like rules-based automated redlines and issue spotting, reduces legal risk—because not even the most experienced lawyers can easily catch these risk-filled positions with 100% accuracy.

By leveraging AI for these tasks, the work required from legal professionals becomes less about producing redlines and correcting provisions and more about confirming the accuracy of the model’s results.

Challenges of Using LLMs for Mutualization

Despite the significant potential of LLMs in legal document mutualization, several challenges persist.

Hallucinations

An AI hallucination refers to instances where an artificial intelligence system generates an inaccurate or unrealistic output not based on its training data. These hallucinations occur because AI systems (including LLMs) recognize and extrapolate patterns from their training data, but can sometimes misinterpret these patterns. This is particularly true when handling inputs from complex fields, like law, that require nuanced understanding.

As a result of these uncertainties, careful human review of all generative AI outputs is vital before publication to avoid the potentially serious consequences of acting on incorrect or misleading information.

Inability to Understand Legal Context

While LLMs have become quite sophisticated, they are still not capable of fully understanding the legal context or the specific nuances of a matter. This can lead to generation of text that may be contextually incorrect or not fully aligned with the intent of the provision.

For instance, given the initial provision “The customer shall indemnify the company,” the LLM may simply swap the parties and generate “The company shall indemnify the customer.” This doesn’t truly mutualize the provision, which should instead be something like “Each party shall indemnify the other.”

Additionally, because the accuracy and effectiveness of LLMs largely depend on the data on which they are trained, there may be gaps in the value of outputs. If the training data lacks a broad range of mutual provisions or includes incorrect or biased provisions, the generated text will likely reflect these limitations.

Legal and Ethical Considerations

The use of LLMs in legal drafting also raises important legal and ethical considerations. There are concerns about accountability and liability if an AI-generated provision leads to a legal dispute. Moreover, since legal work frequently involves sensitive and confidential information, AI tools that process such data are subject to compromise, leading to breaches of privacy. Using AI tools also creates concern regarding consent and how and where data is stored and used.

How LexCheck is Overcoming and Using LLMs for Mutualization

LexCheck has been developing its proprietary rule-based Natural Language Processing (NLP) system—an approach that differs from LLMs in that the system draws conclusions from predefined rules—since 2015. This form of rules-based AI is not prone to the same concerns as those of an LLM because there is not a generative component; for instance, rule-based NLP systems are implemented with a specific project scope and do not generate new text outside of it.

As a contract review and acceleration platform that provides redlines according to your negotiation guidelines, there are many benefits to integrating a generative AI tool with our proprietary software—including the ability to mutualize provisions with greater success. Although the benefits of these integrations largely outweigh the challenges, addressing concerns about the use of LLMs in the legal field has been a priority for our team.

Approaching Hallucinations

There are two main approaches to reducing the effect of hallucinations when prompting an LLM. First, we have relied upon a human-in-the-loop evaluation. Leveraging domain experts—subject-matter experts in law—to validate the outputs helps to ensure that the model is not hallucinating during the development phase. These experts are able to quickly evaluate and identify if unilateral inputs produce accurate mutualized outputs.

In addition to a human review, the models themselves can be prompted to evaluate the provisions. The main goal of this approach is to ensure that the mutualized provision produced by the model is, in fact, mutualized—reducing the risk of hallucinations in the ultimate result.

By working to reduce the amount of hallucinations before implementation, we aim to provide legal professionals with a solution that is highly effective, allowing for seamless integration of the tools and their outputs.

Addressing Context and Training Data

When it comes to the model’s inability to understand legal context, domain experts again come into play as a first method of improvement. By working with subject-matter experts on domain-specific prompt engineering, prompts are able to be designed effectively for the custom task. They are able to inject their legal knowledge into the prompt description, such as by providing definitions of legal concepts, which helps the model better understand the legal context and terminology.

Dependence on training data (which may be biased or too limited) can be mitigated by giving the model examples of inputs and their corresponding outputs within the prompt. Providing examples, known as few-shot prompting, helps the LLM to better understand the data set. Although its working mechanism is still being studied, few-shot prompting helps alleviate the dependence on the initial training data, helping the model to produce more accurate results over time.

Breaking free of these limitations is necessary to provide long term value for users. By customizing prompts to develop clarity and expanding context using examples, consistency of results can be achieved faster and with more accuracy. The more the model is able to understand what is being asked, the better mutualized provisions provided by it will become.

Ethical Implementation

Ethical and legal considerations surrounding the implementation of an LLM for legal support are vast. In the case of utilizing an LLM for provision mutualization, caution is taken to remove all Personal Identifiable Information (PII) before input into the model. By removing sensitive, confidential data in advance, concerns are eliminated regarding data leaks from the model or the inappropriate storage or use of private information.

Ensuring that your data is protected and remains confidential is key to maintaining legality and providing quality services. As we continue developing these tools, data protection and security remains of the utmost importance.

Critical Questions Before Implementing an LLM Tool

As the use of AI continues to expand in the legal field, it is vital to thoroughly evaluate solutions before implementation, especially regarding how they handle the challenges mentioned above. Through the product exploration and consideration phases, important questions to ask potential vendors include:

Functionality: What functionality does this AI tool offer? Is it for drafting, review and editing, or for negotiating contracts with the other party?

Data Privacy: Do you send the whole document including PII to the AI tool for it to generate predictions? If yes, then is that data retained for further training? Is there a way to disable that feature so that it does not retain my data for further training purposes?

Technical approach: Does the AI tool leverage commercially available LLMs like GPT? Does it also generate content using generative AI? If yes, then how do you make sure that the content is not the result of hallucinations? Are these tools generating predictions and outputs in real-time based on most current data or do they work on snapshot?

Security: Where is the AI tool hosted? Is it in your application hosting environment?

Feedback: Can we review/override the AI tool’s predictions? Do we have the ability to configure the AI tools outputs to fine tune to our needs?

In conclusion, while LLMs offer significant potential for provision mutualization in legal documents, it is crucial to carefully navigate the associated challenges. As the capabilities of these models continue to advance, ongoing research and thoughtful regulations will play a vital role in ensuring their effective and ethical use.