翻訳言語

Cursorbench: Grok 4.5がGPT-5.5を上回り、コストは約半分

Cursorが発表したベンチマーク「Cursorbench」において、xAIのGrok 4.5がOpenAIのGPT-5.5を総合性能で上回る結果となった。さらに、推論コストはGPT-5.5の約半分であり、コストパフォーマンスの面でも優位性を示している。

背景メモ

Cursor（AIコードエディタ）が独自のコード生成ベンチマーク「Cursorbench」を発表。GPT、Claude、Grokなど主要な大規模言語モデル（LLM）をソフトウェアエンジニアリングの実タスクで評価し、xAIの「Grok 4.5」がOpenAIの「GPT-5.5」をスコアで上回り、かつAPIコストは約半分と主張。ただし「GPT-5.5」はOpenAIの公式モデル名ではなく、Cursorが独自に定義した内部コードネームである点に注意。このベンチマークはAIコーディング市場での競争激化を背景に、従来の定型的なテスト（例：HumanEval）ではなく、より実践的なコード編集・デバッグ能力を測定しようとする試み。CursorはAnysphere社が開発するVS Codeベースのエディタで、AI補完機能で知られ、同分野でGitHub Copilotと競合する主要プレイヤー。

RT Max Weinbach: I had early access to this model and I'll post more Thursday, but I think the most impressive part is this model NEVER GIVES UP If yo...
9.0
Max Weinbach says he had early access to OpenAI's new model GPT-5.6 Sol, calling it his favorite model by far. He highlights that it never gives up and will keep reasoning until it's done. OpenAI announced that GPT-5.6 Sol, along with Terra and Luna, will launch publicly on Thursday, with preview access expanding globally now.
Better Models: Worse Tools
7.5
Newer Claude models sometimes invent extra keys in tool call arguments, breaking validation in Pi's edit tool. The author suspects post-training for Claude Code's forgiving harness makes alternative schemas fail. This suggests closed RL training can degrade general tool-use reliability.
Writing an LLM from scratch, part 34b -- from bigrams to GPT-2, one component at a time (in JAX)
3.0
The author builds and trains a GPT-2 small model from scratch in JAX, starting from a basic bigram-style model and incrementally adding components like LayerNorm and Transformer blocks. Achieved a final loss of 3.418, beating their PyTorch version (3.538) and original GPT-2 small (3.499) on the same test dataset.
Flax debugging: making a hash of things
2.0
The author debugged a Flax NNX training loop where the loss was stuck at 10.82, indicating random guessing. By hashing the model parameters and comparing hashes across steps, they discovered the parameters weren't changing. The root cause was using @jax.jit instead of @nnx.jit, which is needed for proper in-place state propagation of parameter updates in NNX.
Writing an LLM from scratch, part 34a -- building a JAX training loop for an LLM training run
1.0
Building a JAX training loop for an LLM from scratch using Flax NNX and Optax. First validated the harness with a minimal "A-to-A" model (embedding then projection) before adding Transformer layers. Key challenges included fixing JAX's GPU memory default and slow data iteration by committing data to CPU memory.

Cursorbench: Grok 4.5がGPT-5.5を上回り、コストは約半分

背景メモ

関連記事

RT Max Weinbach: I had early access to this model and I'll post more Thursday, but I think the most impressive part is this model NEVER GIVES UP If yo...

Better Models: Worse Tools

Writing an LLM from scratch, part 34b -- from bigrams to GPT-2, one component at a time (in JAX)

Flax debugging: making a hash of things

Writing an LLM from scratch, part 34a -- building a JAX training loop for an LLM training run