6.1 Sol的mid比xhigh档要好一点

作者：#zzhi 板块：#搞七捻三 github.com/humanlayer/advanced-context-engineering-for-coding-agents benchmarking-new-models-on-slop-code-bench.md main # SlopCodeBench update 10/2026 - Opus 5.5 update, Astra improves by 1.5% over gpt-5.5, xhigh considered harmful October 15, 2026 - [Dex Horthy](https://x.com/dexhorthy) w/ contributions from [Gabe Orlanski](https://x.com/gorlanski) > **NOTE:** This is Part I of our October 2026 writeup: the Q4 state of the slop. Gabe and the [SlopCodeBench team](https://arxiv.org/abs/2603.24755) will share their analysis and a more qualitative deep dive into slop patterns in Part II. > For my previous writeups on SlopcodeBench, see [benchmarking-opus-5-on-slop-code-bench.md](./benchmarking-opus-5-on-slop-code-bench.md) and [benchmarking-sol-fable-kimi-on-slop-code-bench.md](./benchmarking-sol-fable-kimi-on-slop-code-bench.md). Both include deeper dives on the core structure of SCB and how each of its challenges is broken into checkpoints. ### Overview In May, the last time SCB was published, gpt-5.5 got a **14.8% strict pass rate on slopcodebench**. This month, I ran the frontier models through the same campaigns, and the highest scorer was gpt-6 astra, hitting **16.3%**. While the benchmark reports several different pass rates, the one we focus on is **strict pass rate** - in how many checkpoints did the model pass every test case while still passing all regression tests the frontier models we tested here had strict pass rates in the **12.2-16.3%** range: - Opus 5.5 - Fable 5.1 - SOL 6 and 6.1 - Astra - GLM 5.3 此文件已被截断。 显示原始文件 看起来只有Sol的xhigh会有这个问题，且该结论可能只适用于coding场景