菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-04-07
📄 Abstract - JTON: A Token-Efficient JSON Superset with Zen Grid Tabular Encoding for Large Language Models

When LLMs process structured data, the serialization format directly affects cost and context utilization. Standard JSON wastes tokens repeating key names in every row of a tabular array--overhead that scales linearly with row count. This paper presents JTON (JSON Tabular Object Notation), a strict JSON superset whose main idea, Zen Grid, factors column headers into a single row and encodes values with semicolons, preserving JSON's type system while cutting redundancy. Across seven real-world domains, Zen Grid reduces token counts by 15-60% versus JSON compact (28.5% average; 32% with bare_strings). Comprehension tests on 10 LLMs show a net +0.3 pp accuracy gain over JSON: four models improve, three hold steady, and three dip slightly. Generation tests on 12 LLMs yield 100% syntactic validity in both few-shot and zero-shot settings. A Rust/PyO3 reference implementation adds SIMD-accelerated parsing at 1.4x the speed of Python's json module. Code, a 683-vector test suite, and all experimental data are publicly available.

顶级标签: llm systems data
详细标签: json serialization token efficiency structured data context window parsing optimization 或 搜索:

JTON:一种面向大语言模型的、具有Zen Grid表格编码的令牌高效JSON超集 / JTON: A Token-Efficient JSON Superset with Zen Grid Tabular Encoding for Large Language Models


1️⃣ 一句话总结

这篇论文提出了一种名为JTON的新数据格式,它通过一种叫Zen Grid的表格编码方法,在完全兼容JSON的同时,大幅减少了LLM处理表格数据时所需的令牌数量,从而降低了使用成本并提升了部分模型的性能。

源自 arXiv: 2604.05865