The Risks of Paid Legal AI Chatbots

Oh Jeong-ik, Head of AI & Tech Team, Law Firm One (Attorney)

News|
|
By Sedaily IN
||
An AI-generated image depicting the risks of paid legal AI chatbot services. - Seoul Economic Daily Opinion News from South Korea
An AI-generated image depicting the risks of paid legal AI chatbot services.

In Korea, providing paid legal AI chatbot services (excluding here discussions about introducing, brokering, or soliciting legal cases) is in principle prohibited under the Attorney-at-Law Act.

Prohibited legal AI chatbot services do not include simple automation services—for example, services that provide a fixed legal format where users directly fill in the required content, or services that mechanically categorize content written by the user and input it into the corresponding form. Additionally, among legal AI chatbot services, those that explain a legal principle or issue and find precedents dealing with similar cases can be considered not to fall under legal AI chatbot services prohibited by the Attorney-at-Law Act. This is because such services do not constitute the handling of legal affairs concerning legal cases by non-attorneys, which the Attorney-at-Law Act prohibits.

Article 109 of the Attorney-at-Law Act prohibits any person who is not an attorney from handling legal affairs for profit, including appraisal, representation, arbitration, settlement, solicitation, legal consultation, or the preparation of documents concerning legal relations in relation to legal cases. According to case law, "legal affairs" means "the handling of matters that create, modify, or extinguish legal effects, or that preserve or clarify them." Therefore, if a legal AI chatbot directly presents necessary future legal measures or response plans for a user's specific case or issue, this goes beyond mere information provision and constitutes the performance of legal affairs concerning a specific legal case, which cannot be permitted under current law.

Accordingly, the automation services mentioned above, which provide mechanical services, cannot be considered as handling legal affairs. And finding precedents, which merely presents the precedent deemed most consistent with what the user entered, also cannot be considered as handling legal affairs, and is therefore permissible.

Some argue that legal AI chatbot services, like the large language models (LLMs) on which they are based, merely combine and generate data that AI probabilistically judges to be most appropriate or consistent with the user's question based on learned data. Therefore, they contend, viewing the provision of legal AI chatbot services as handling legal affairs is a misunderstanding, and such services should be permitted to the general public. However, this logic could lead to unreasonable results—since autonomous driving AI operates in a similar manner, the strange conclusion would follow that its operation is not "driving" (and by this logic, it would also follow that non-attorneys should be permitted to provide legal consultations through similar probabilistic reasoning). However, the reason paid legal AI chatbot services are prohibited is that, regardless of the process, the provision of such output functions as substantive legal judgment and assistance, such as legal consultation or legal advice, to the user.

There may be a fundamental question as to why the Attorney-at-Law Act prohibits non-attorneys from handling legal affairs. This is because legal affairs directly and significantly affect important matters such as the rights, obligations, and property of individuals. Therefore, only those who can guarantee expertise, ethics, and accountability are permitted to handle such work, in order to protect these rights and obligations. This principle is the foundation of the judicial system commonly maintained not only in Korea but also in most countries including the United States and Europe, with some differences in details. It aligns with the purpose of other licensing systems, such as limiting medical practice to doctors and pharmaceutical sales to pharmacists.

In addition, there may be another question. According to news reports, some legal AI chatbots have passed the bar exam with excellent scores. Since the performance or results of legal AI chatbots are guaranteed, shouldn't non-attorneys be allowed to handle legal affairs? However, this cannot be viewed simply either.

First, the Attorney-at-Law Act requires attorneys to have not only legal knowledge but also a high level of ethics (attorneys also bear various obligations such as pro bono activities and ethics training), as well as a duty of confidentiality. It is questionable whether legal AI chatbots meet the same level of ethical standards as attorneys. This is even more so given that those providing and operating legal AI chatbot services are for-profit companies (case law has consistently denied the commercial nature of attorneys and law firms).

Furthermore, if legal AI chatbots were permitted, there could be concerns that this discriminates between humans and AI, and gives preferential treatment to AI based only on results. For example, consider someone who has not passed the bar exam but has more practical experience in registration work than attorneys and scores as high as attorneys in registration-related exam fields. However, this person cannot handle registration work because they do not have an attorney license. There are social reasons why licensing systems have been established and operated over a long period of time. This could also violate the constitutional principle of equality by discriminating between non-attorneys and non-attorneys who operate legal AI chatbots or the companies that do so.

Meanwhile, if legal AI chatbot services were permitted on a paid basis to the general public without restriction, there would be a very high risk that some companies with massive capital and technological capabilities would monopolize the market (indeed, look at how chatbot services are dominated by a few giant companies). The domestic market could also be eroded by foreign companies.

In terms of effectiveness, it cannot be said that AI chatbots are unconditionally better either. For example, one of the most important judgments in legal disputes is the issue of fact-finding. Legal AI chatbots cannot determine facts and are often used in a manner that applies legal principles based on the facts entered by the user. Legal AI chatbots do not go through the process of pointing out and correcting problems—such as judging whether the facts recognized by the user are correct, what materials support them, and which materials cannot be considered supportive. That is, legal AI chatbots cannot sufficiently go through procedures to actively verify or correct errors inherent in the user's input. However, non-attorney users firmly believe the results provided by legal AI chatbots and act accordingly (recently, although not legal AI chatbots, clients often bring results provided by AI chatbots and ask for them to be handled as stated, and when the attorney's opinion differs, they often do not trust the attorney and instead question them. However, most are wrong, and in many cases, they are not persuaded despite the attorney's opinion).

And the decisive point is that legal AI chatbots or the companies providing such services will not be held responsible like attorneys when the results are wrong (look at the terms of service of platforms providing AI chatbot services. Most of them state that they are not responsible for the results provided by the AI chatbot. And the terms of service for legal AI chatbot services used by attorneys also state that they are not responsible for the accuracy or appropriateness of the results, and that the final user—the attorney—should review, judge, and use them). This is similar to the structure in which Tesla, which charges separate service fees for autonomous driving functions, shifts the final driving responsibility to the user when an accident occurs while using the autonomous driving function.

Ultimately, the argument that legal AI chatbots should be open to the general public because they have passed the bar exam and provide good legal advice at a lower cost than attorneys may seem correct at first glance, but it is not such a simple issue. Rather, such statements are, in some respects, irresponsible and dangerous arguments. Whether paid legal AI chatbot services should be open to the general public is a matter that must be approached with great difficulty and caution, considering the institutional characteristics of the relevant business area, the actual impact, and the position and direction of domestic AI technology in the global market. It is not a matter to be approached solely with the logic of cost and performance (although the performance aspect is also controversial).

Oh Jeong-ik's AI Law Insight - Seoul Economic Daily Opinion News from South Korea
Oh Jeong-ik's AI Law Insight

Original reporting by Sedaily IN for Seoul Economic Daily.

AI-translated from Korean. Quotes from foreign sources are based on Korean-language reports and may not reflect exact original wording.

00:0006:16

AI KEY

Preview
Korean Corporate Intelligence HubKOSPI · KOSDAQ · 12 sectors

A live, cap-weighted view of every KOSPI and KOSDAQ sector, with same-day Korean reporting distilled by company — built for foreign investors, correspondents and analysts who need to scan Korea before the next session.

Korea Chaebol Tree

Preview
Families Behind the GroupsKFTC May 2026 · DART filings

An English-first interactive map of Samsung, SK, Hyundai, LG and Lotte — built for foreign investors, correspondents and analysts. Korea translates companies into English. We translate the families behind them.