No JIT
10/10
- Wall
- 207.6s
- Turns
- 69
- Sec / turn
- 3.01
- Tokens
- 57,989
- Discovery
- 40
- Live JEV
- 0/10 · fb 0
Full tool trace in the prompt. No capsule.
SAME SESSION · 2026-09-23 · GEMINI 3.8 FLASH
Three modes. Ten fixtures. One log.
No JIT, JIT with heuristic ranking, and JIT with a live JEV call, interleaved on the same ten fixtures. JIT plus JEV finished in 132.0s against 207.6s with no capsule. That is 1.57x on wall clock, not 2.44x. Tokens went up. This is not official SWE-bench.
JIT, no JEV
9/10
- Wall
- 138.5s
- Turns
- 52
- Sec / turn
- 2.66
- Tokens
- 63,202
- Discovery
- 26
- Live JEV
- 0/10 · fb 0
Heuristic file ranking. The miss is a model JSON parse error, not a red pytest.
JIT + live JEV
10/10
- Wall
- 132.0s
- Turns
- 49
- Sec / turn
- 2.69
- Tokens
- 64,797
- Discovery
- 23
- Live JEV
- 10/10 · fb 0
HTTP 200 on every task. Fallback count 0. Same session as the other two modes.
What moved, and what did not
Bars share a scale inside each group. Lower is better for wall, turns, and discovery. Tokens are the exception.
No JITJIT, no JEVJIT + live JEV
- Wall clock, JIT+JEV vs no JIT: 132.0 / 207.6 = 1.57x. 63.6% of the raw duration.
- Turns: 49 vs 69, 29.0% fewer. Discovery: 23 vs 40, 42.5% fewer.
- Tokens: 64,797 vs 57,989, 11.7% more. The capsule adds prompt. It does not shrink the bill.
- JIT+JEV vs heuristic JIT: 6.5s and 3 turns. Not a gulf. JEV was slower on django-11848 and flask-4045.
Per task, wall clock
Scale is the slowest single run, requests-2148 at 52.99s. PARSE means the model reply was not valid JSON. Pytest never ran.
django-11099
django-11848
django-14382
django-15400
astropy-14995
flask-4045
requests-2148
django-11583
pytest-5221
sympy-18057
Scope, so the chart cannot be misread
- Isolated multi-file fixtures plus decoy files. Not full repository checkouts. Not official SWE-bench.
- Agent model gemini-3.8-flash. JEV model typesafe/jev-1.13-20260917. Live call on 10/10 JIT+JEV tasks, HTTP 200, fallback 0. JEV usage cost $0.000261.
- Gemini dollar cost is not in the API response, so it is not printed. Token totals are
totalTokenCount. - The 9/10 on heuristic JIT is
pytest-5221:Expecting value: line 1 column 1. Do not read it as a ranking failure or a JEV rescue. - An older log (150.9s / 133.9s) is a different session. It is not this comparison.
- The 2.44x figure belongs to the Synthapse 45-module run. It is not this fixture battle.