SpotIt+：基于验证和数据库约束的文本到SQL评估工具 / SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints
1️⃣ 一句话总结
这篇论文提出了一个名为SpotIt+的开源工具，它通过自动寻找能区分AI生成的SQL查询和标准答案的数据库实例来评估文本转SQL系统，并利用挖掘出的数据库约束来确保找到的差异具有实际意义，从而比传统测试方法更高效、更准确地发现潜在问题。
arXiv 提交日期: 2026-03-04
Abstract - SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints
We present SpotIt+, an open-source tool for evaluating Text-to-SQL systems via bounded equivalence verification. Given a generated SQL query and the ground truth, SpotIt+ actively searches for database instances that differentiate the two queries. To ensure that the generated counterexamples reflect practically relevant discrepancies, we introduce a constraint-mining pipeline that combines rule-based specification mining over example databases with LLM-based validation. Experimental results on the BIRD dataset show that the mined constraints enable SpotIt+ to generate more realistic differentiating databases, while preserving its ability to efficiently uncover numerous discrepancies between generated and gold SQL queries that are missed by standard test-based evaluation.
源自 arXiv: 2603.04334