Colombian court rejects appeal over alleged AI writing and is flagged by its own detector

The Supreme Court of Colombia dismissed a cassation appeal after AI-detection software flagged the filing as largely AI-generated, and subsequent tests using the same and other detectors returned high AI scores for the court's own ruling and for older documents.

The Supreme Court of Colombia dismissed a cassation appeal after concluding the attorney's brief was largely produced with generative AI. The court said it used the Winston AI tool and that analysis indicated the filing contained only 7% human content. Lawyers and legal experts then ran the court's ruling and other older texts through the same and other detectors and obtained high AI scores for the court document and for pre-AI-era filings. The sequence has prompted renewed attention to Colombia's December 2024 guidelines on judicial use of artificial intelligence. Key facts: - The court reported submitting the contested brief to the Winston AI tool and said the analysis indicated roughly 7% human content, which the court cited when declaring the filing inadmissible. - Attorney Emmanuel Alessio Velasquez and others tested the court's ruling (Auto AP760/2026) and reported that Winston and other detectors returned high AI percentages for the court text, including a reported 93% AI result from Winston. - Independent tests showed inconsistent results across detectors: for example, GPTZero reportedly flagged the opening words of the ruling as 100% AI but returned 100% human when a longer version was scanned. - Several Colombian lawyers ran historical documents through detectors and said filings from 2019–2020 (before large language models of current scale) were flagged as largely AI-generated, raising concerns about false positives and commercial incentives in some detection services. - Academic and institutional studies and actions have documented detector limitations: a 2023 Patterns study and systematic reviews cited high false-positive rates for non-native writing, Turnitin acknowledged elevated false positives in 2023, OpenAI withdrew its detector, and some universities later disabled or removed detection features. - Colombia's judicial guidelines (PCSJA24-12243, December 2024) permit administrative AI uses but prohibit relying on AI to evaluate evidence, interpret the law, or make judicial decisions; the Supreme Court has not issued an additional public statement about the detector-based ruling. Summary: The episode has highlighted limits of current AI-detection tools and the potential for inconsistent or false-positive results, particularly with formal legal prose and texts by non-native writers. Colombia's December 2024 judicial guidelines set boundaries for AI use in courts, but the Supreme Court has not provided further public clarification about this specific decision. Undetermined at this time.