OpenAI says GPT-6 Astra lifts coding scores in 20 less-used languages

![OpenAI illustration accompanying its benchmark](https://openai.com/_next/static/immutable/media/lab.44a_dymau51y3.webp) OpenAI’s illustration for the benchmark; the scores below are the company’s own evaluations. OpenAI says GPT-6 Astra solved 87.4% of its tests across 20 less commonly used programming languages, compared with 26.8% for GPT-5.6 Sol. The company published the benchmark on Oct. 19 alongside an open test suite; the scores are OpenAI’s own evaluations, not an independent comparison or a measure of production reliability. The benchmark, called MultiLangBench, asks models to build a SQLite-compatible database engine in each language. OpenAI says it adapted SQLite’s Tcl tests and ran the models at their highest reasoning setting. It reports GPT-6 Luna at 40.3% and estimates a cost of $0.77 per sample, versus $3.48 for GPT-5.6 Sol. On a separate COBOL migration test, OpenAI reports GPT-6.1 Sol at 86.6% with low reasoning effort, against 5.4% for GPT-5.6 Sol at the same setting. The figures point to a possible improvement for work involving older or less widely represented languages, where training data is thinner. They do not show that generated code is secure, correct in a real deployment, or ready to replace expert review. OpenAI’s [benchmark write-up](https://openai.com/index/low-resource-programming-languages/) explains its test setup, and the [public MultiLangBench repository](https://github.com/openai/MultiLangBench) contains the SQLite test suite so others can inspect and run it. Independent replication and performance on real-world maintenance projects remain open questions.

0
Discuss 0 3
0 replies
Be the first to join this thread.