Artificial intelligence in law firms - insights from an AI experiment

AI solves legal examination tasks - what is already possible today

Expert opinion style from the machine: prioritisation and subsumption

When AI reaches its limits: Logical errors and lack of intuition

A better result with the right prompt

AI as an assistant in the law firm - relieving everyday workloads

Increased efficiency without loss of judgement

Intelligent agent systems for law firms - AIMAX® as an example

The expert interviews in full length

Conclusion

A recent experiment impressively demonstrated what modern AI is capable of: an advanced language model ("Gemini 2.5 Pro Experimental") was able to pass a complex exam in civil law with 8-9 points (i.e. in the satisfactory range).

Even GPT-4 (ChatGPT 4.1) still achieved a "sufficient" in this test run and delivered a solution that was roughly equivalent to what many human candidates would deliver.

The experts were surprised that an AI could already master such a demanding legal examination task. After all, it was previously assumed that legal case solutions require a high degree of human understanding, experience and intuition.

However, the results show that with the right approach, AI systems can solve complex legal problems in a structured manner.

The decisive factor for success was how the AI was introduced to the task. In the experiment, the researchers gave the AI a detailed prompt that instructed it, among other things, to work like a lawyer in the style of an expert opinion.

In other words, the model was supposed to construct the solution as a fully formulated expert opinion: systematically check all relevant claim requirements, recognise the important key points and deal with them in detail, and keep less central points brief.

In fact, the AI was able to produce a 14-page solution with a plausible structure and sensible focal points. It subsumed - i.e. it assigned the facts of the case to the legal requirements - in principle like a human editor.

Even typical formulations such as upper clauses ("Who wants what from whom and why?") were taken into account, giving the solution an authentic examination style.

This structured approach also facilitates quality control: a thoroughly composed expert opinion text immediately shows whether a thought step is missing somewhere or whether there is a logical break. The AI solution was therefore able to make it clear how it arrived at its results - an important aspect for creating trust in AI work.

As impressive as the AI performance is, there are clear limitations. Dr Jessica Flint, for example, observed that the models struggled to remain truly consistent in their clean review style. In the middle of the text, the AI suddenly inserted keywords and plus/minus lists - deadly sins for human proofreaders in exams that indicate a lack of care. Such slip-ups show that the AI can follow the exam style formally, but has no real understanding of why this strict form is important.

In terms of content, AI answers can also sometimes contain faulty reasoning. Although AI often arrives at plausible results, the mental path it takes to get there is not always stringent. Dr Sophie Garling points out that an AI solution can arrive at the same result as a human being, but with an unusual focus. In other words, the AI's reasoning can have gaps or inconsistencies that are not obvious at first glance. This shows that the model lacks the in-depth legal understanding to distinguish reliably between equivalent solutions.

Another limiting factor is the lack of legal intuition. AI systems ultimately work statistically - they calculate the most likely continuation word for word - and have no gut feeling or real legal awareness. Dr Sophie Garling emphasises that current models such as "Gemini" do not really understand law, nor do they "think correctly" in a human sense. Dr Flint recalls that it was traditionally assumed that legal problem solving required human intuition, experience and understanding - qualities that a machine lacks. So despite all its impressive achievements, AI remains a tool without its own legal judgement.

The experiences from the interviews make it clear that the key to reliable AI results lies in prompt engineering - in other words, in the art of explaining the task to the AI in the best possible way. A naive approach ("Here is the problem - solve the exam!") does not lead to the goal. Without precise instructions and contextual knowledge, the system will not provide a useful exam solution.

Instead, the prompt must contain important information: for example, which legal role the AI should take on (e.g. "You are an exam candidate"), which style is expected and what to pay particular attention to. You need at least a certain amount of legal methodological knowledge for this - either you bring it with you or you "borrow" it by using a prompt prepared by a lawyer.

In other words, you don't necessarily have to be a fully qualified lawyer to use AI, but without legally sound instructions, AI will not realise its full potential.

Prompt engineering has thus crystallised as a new key qualification. It has been shown that finding the perfect prompt is by no means trivial - in the aforementioned research project, experts spent a very long time working on the optimal input methodology.

One successful approach is to provide the AI with a clear path of thought. For example, a chain-of-thought process can be simulated in the prompt by providing step-by-step instructions: first identify the basis of the claim, then check each fact, then merge the results, and so on.

In effect, the prompt developed resembled a collection of if-then rules: e.g. "If you are a lawyer, here's what you would do..." or "As an exam candidate, you adhere to the expert opinion style...". This structured guidance enabled the AI to argue more logically and comprehensibly.

In addition to the prompt itself, the training of the AI model also plays a role. Large language models (LLMs) such as GPT-4 are pre-trained on huge amounts of text; however, it is often not transparent whether this included specific legal data and to what depth. In the case of "Gemini", for example, it is unclear whether it was also trained with legal texts or decisions.

One thing is certain: the more high-quality legal information a model receives in its training or via upstream knowledge databases (keyword: Retrieval Augmented Generation - RAG), the better it can respond to legal questions. In the future, specialised legal LLMs and hybrid approaches (combining AI with rule-based expert systems) are likely to become increasingly important in order to achieve even more precise and reliable results.

Away from exams, the question arises: How can such AI capabilities help law firms in practice? In fact, AI systems offer enormous potential for reducing the workload of time-consuming routine tasks. Some key areas of application are

• Research: AI agents such as AIMAX® can search through extensive legal databases and deliver relevant information in seconds. Tedious research work - such as searching for judgements on a specific problem - can thus be automated. By skilfully combining it with databases, the AI can even extract facts from long documents and present them in a clear and concise manner.
• Document analysis and pre-structuring: Intelligent systems are able to analyse contracts, pleadings or mountains of files. They filter out important information from the documents and recognise connections. Lawyers thus receive a pre-structuring of complex issues - e.g. a list of disputed points or a summary of the key details of a case. This makes further processing much easier.
• Drafting of pleadings: Generative AI such as the AI agent AIMAX® can formulate initial draft pleadings - be they contractual clauses, statements of claim or expert opinions. In a matter of seconds, the system creates texts that can serve as a basis, saving lawyers a lot of time when drafting documents. Of course, a human lawyer has to check and refine the draft, but most of the routine formulation can be done by the AI.
• Argumentation support: AI systems can help to compile arguments in favour and against a legal problem. For example, an AI can be asked to look through the relevant case law or literature on an issue and summarise the most important lines of argument. This provides input for your own argumentation or allows you to check whether you have overlooked an aspect.
• Automating repetitive tasks: In law firms, there are many recurring routine tasks. Whether filling out standardised forms, checking contract clauses that are always the same or checking the receipt of new cases - AI and robotic process automation (RPA) can perform such repetitive tasks reliably and quickly. It reduces errors (e.g. due to fatigue caused by monotonous work) and frees up lawyers for more demanding tasks.

All of these fields of application boil down to the fact that AI and RPA systems such as EMMA provide qualitative support and free up capacity. Routine tasks that would otherwise take hours are completed by AI in seconds or minutes. Complex analyses that used to require poring over files are prepared at the touch of a button. Lawyers can invest the time gained in strategy, client care or creative legal research.

However, it is important to note that AI does not replace human judgement and decision-making. The best results are achieved when humans and machines work together. Generative AI cannot replace human lawyers, but it can play an important role if used correctly. Its output must always be checked by a lawyer before it is used in pleadings or client advice.

As long as AI results are not accepted unquestioningly, but rather subjected to critical appraisal, the technology can serve as a "second pair of eyes" or internal partner.

In practice, this means that AI designs and researches, while humans validate and decide. Incorrect suggestions from the AI must be recognised and corrected - ultimately, the responsibility remains entirely with the lawyers.

If this is taken to heart, AI tools offer an enormous increase in efficiency and quality. They make it possible to process cases faster, meet deadlines better and still work thoroughly.

For busy law firms in particular, tested AI support can lead to a new level of productivity. However, the core of legal work - thinking and making judgements - remains the domain of humans.

Suitable tools are needed to fully utilise the advantages of AI in practice. In addition to general services such as ChatGPT, special AI agent systems are being developed that can be trained for legal applications. AIMAX® is one such example: a specialised AI platform that supports workflows in law firms as an intelligent agent system.

AIMAX® can be integrated into existing law firm processes and provide a wide range of AI functions - from research assistance to automated document creation. By combining several AI models and incorporating in-house knowledge, AIMAX® works directly in the day-to-day work of lawyers in a contextualised and data protection-compliant manner.

In addition, AIMAX® can also utilise the cognitive AI EMMA - an RPA solution from Germany - and thus be integrated into any process in order to automate it or allow it to benefit from the advantages of artificial intelligence.

Solutions like these show that AI can be utilised in practice without lawyers having to completely change their established processes. The technology adapts to the legal workflow and not the other way round.

AI systems already offer law firms considerable opportunities to work more efficiently and effectively. Even if successfully solving legal audit tasks is more about gaining knowledge than increasing efficiency in day-to-day business, there are many possible applications.

It is important to consciously deal with the limits of AI: it is a powerful assistant, but not a substitute for human legal expertise. Law firms that find this balance and integrate suitable AI tools such as AIMAX® and the cognitive AI EMMA RPA in a meaningful way can improve the quality of their services and secure competitive advantages - without sacrificing the indispensable power of human judgement.