\n<\/p>\n
Artificial intelligence (AI) tools have got significantly better at answering legal questions but still can not replicate the competence of even a junior lawyer, new research suggests.<\/p>\n
The major British law firm, Linklaters, put chatbots to the test<\/a> by setting them 50 “relatively hard” questions about English law.<\/p>\n It concluded OpenAI’s GPT 2, released in 2019, was “hopeless” but its o1 model, which came out in December 2024, did considerably better.<\/p>\n Linklaters said it showed the tools were “getting to the stage where they could be useful” for real world legal work – but only with expert human supervision.<\/p>\n Law – like many other professions – is wrestling with what impact the rapid recent advances in AI will have, and whether it should be regarded as a threat or opportunity.<\/p>\n The international law firm Hill Dickinson recently blocked general access<\/a> to several AI tools after it found a “significant increase in usage” by its staff.<\/p>\n There is also a fierce international debate about how risky AI is and how tightly regulated it needs to be.<\/p>\n Last week, the US and UK refused to sign an international agreement on AI<\/a>, with US Vice President JD Vance criticising European countries for prioritising safety over innovation. <\/p>\n<\/div>\n This was the second time Linklaters had run its LinksAI benchmark tests, with the original exercise taking place in October 2023.<\/p>\n In the first run, OpenAI’s GPT 2, 3 and 4 were tested alongside Google’s Bard.<\/p>\n The exam has now been expanded to include o1, from OpenAI, and Google’s Gemini 2.0, which was also released at the end of 2024.<\/p>\n