The Rundown AI: AI tutors edge out of law faculty

Study: AI tutors edge out law facultyImage source: Images 2.0 / The Rundown

The Rundown: A new study led by Stanford ran a blind test of AI legal tutoring, asking 16 contract professors to judge anonymized answers from their peers and from two of Google’s AI systems — with the faculty siding with the AI outputs 75% of the time.

The details: The study tested contract-law office-hours questions, a setting where strong answers need judgment and critical thinking instead of just one correct answer. Sixteen professors from 14 schools blindly judged 2,918 matchups between their own answers and those of Google’s Gemini 2.5 Pro and NotebookLM.Faculty chose Gemini 2.5 Pro and NotebookLM responses 75% of the time, with only a single top professor staying level with the models in evaluations. Extending the testing with an AI stand-in judge, the team ranked nine more systems, with Claude Opus 4.7 on top and all models beating the professors.

Why it matters: Early models like GPT-4 were already passing the bar exam, but this study puts AI in tougher, more subjective judgment situations on office hours contract law questions. AI’s rollout into the education world is still controversial and jagged, but areas like on-demand tutoring can help change the learning process for the better.
QUICK HITS
🛠️ Trending AI Tools⚡ CData Connect AI – Give Claude, ChatGPT, Copilot or any AI tool live, governed, read & write access to your business data in one unified layer.*🎆 Ideogram 4.0 – Ideogram’s new top-rated open-weight image model👋 Meta Enterprise Agent – Meta’s AI agent for customer sales and support💻 Personal Computer – Perplexity’s local orchestrator, now on Windows*Sponsored Listing
📰 Everything else in AI todaySuno raised over $400M at a $5.4B valuation, with the AI music startup rolling out its first model built in partnership with the music industry “in the coming months.”Google released Gemma 4 12B, a new multimodal model able to run on a 16GB laptop, and the first Gemma variant of this size built for native audio.xAI rolled out Grok Imagine 1.5 Preview, the company’s latest image-to-video update, which brings upgraded realism, audio syncing, and prompt following.Microsoft and the Mayo Clinic are building a frontier healthcare AI, trained on anonymized patient data and owned by the clinic, with Azure handling distribution.Google Labs launched a new experiment called Dreambeans, connecting to users’ Gmail, Calendar, Photos, and Search to generate a feed of personalized daily stories.

Unknown's avatar

About michelleclarke2015

Life event that changes all: Horse riding accident in Zimbabwe in 1993, a fractured skull et al including bipolar anxiety, chronic fatigue …. co-morbidities (Nietzche 'He who has the reason why can deal with any how' details my health history from 1993 to date). 17th 2017 August operation for breast cancer (no indications just an appointment came from BreastCheck through the Post). Trinity College Dublin Business Economics and Social Studies (but no degree) 1997-2003; UCD 1997/1998 night classes) essays, projects, writings. Trinity Horizon Programme 1997/98 (Centre for Women Studies Trinity College Dublin/St. Patrick's Foundation (Professor McKeon) EU Horizon funded: research study of 15 women (I was one of this group and it became the cornerstone of my journey to now 2017) over 9 mth period diagnosed with depression and their reintegration into society, with special emphasis on work, arts, further education; Notes from time at Trinity Horizon Project 1997/98; Articles written for Irishhealth.com 2003/2004; St Patricks Foundation monthly lecture notes for a specific period in time; Selection of Poetry including poems written by people I know; Quotations 1998-2017; other writings mainly with theme of social justice under the heading Citizen Journalism Ireland. Letters written to friends about life in Zimbabwe; Family history including Michael Comyn KC, my grandfather, my grandmother's family, the O'Donnellan ffrench Blake-Forsters; Moral wrong: An acrimonious divorce but the real injustice was the Catholic Church granting an annulment – you can read it and make your own judgment, I have mine. Topics I have written about include annual Brain Awareness week, Mashonaland Irish Associataion in Zimbabwe, Suicide (a life sentence to those left behind); Nostalgia: Tara Hill, Co. Meath.
This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

Leave a comment