Chains
Completion per chain: the share of each chain’s tasks the entry completed, with the number of tasks it was scored on. Every task runs on a local fork of its chain. The open track covers 9 EVM chains (Ethereum, Gnosis, Polygon, Arbitrum, Optimism, Base, Celo, Mode, Robinhood); the held-out track covers 8, with none on Robinhood Chain.
| Chain | Claude CodeDeepSeek V4.1 Flash | Claude CodeOpus 5.5 | OpenClawDeepSeek V4.1 Flash | CodexGPT-5.6 Sol | HermesDeepSeek V4.1 Flash | olas-ocb-agentDeepSeek V4.1 Flash | CodexDeepSeek V4.1 Flash |
|---|---|---|---|---|---|---|---|
| Polygon32 tasks | 98%(32) | 94%(32) | 96%(32) | 96%(32) | 96%(32) | 93%(32) | 94%(32) |
| Gnosis31 tasks | 96%(31) | 96%(31) | 95%(31) | 97%(31) | 92%(31) | 87%(31) | 87%(31) |
| Arbitrum20 tasks | 99%(20) | 97%(20) | 94%(20) | 91%(20) | 96%(20) | 98%(20) | 94%(20) |
| Mode19 tasks | 99%(19) | 100%(19) | 96%(19) | 89%(19) | 96%(19) | 89%(19) | 93%(19) |
| Base18 tasks | 100%(18) | 91%(18) | 97%(18) | 92%(18) | 93%(18) | 89%(18) | 96%(18) |
| Optimism17 tasks | 99%(17) | 96%(17) | 93%(17) | 98%(17) | 94%(17) | 93%(17) | 76%(17) |
| Celo5 tasks | 40%(5) | 40%(5) | 40%(5) | 56%(5) | 40%(5) | 44%(5) | 40%(5) |
| 96%(5) | 100%(5) | 100%(5) | 92%(5) | 96%(5) | 96%(5) | 80%(5) |
| Chain | Claude CodeOpus 5.5 | Claude CodeDeepSeek V4.1 Flash | OpenClawDeepSeek V4.1 Flash | CodexDeepSeek V4.1 Flash | HermesDeepSeek V4.1 Flash | CodexGPT-5.6 Sol | olas-ocb-agentDeepSeek V4.1 Flash |
|---|---|---|---|---|---|---|---|
| Gnosis39 tasks | 99%(39) | 96%(39) | 95%(39) | 99%(39) | 96%(39) | 92%(39) | 91%(39) |
| Polygon11 tasks | 95%(11) | 100%(11) | 100%(11) | 91%(11) | 86%(11) | 95%(11) | 86%(11) |
| Base5 tasks | 90%(5) | 100%(5) | 100%(5) | 90%(5) | 100%(5) | 80%(5) | 90%(5) |
| Arbitrum4 tasks | 100%(4) | 88%(4) | 100%(4) | 100%(4) | 100%(4) | 100%(4) | 100%(4) |
| 100%(4) | 100%(4) | 100%(4) | 100%(4) | 100%(4) | 100%(4) | 100%(4) | |
| Optimism4 tasks | 100%(4) | 100%(4) | 100%(4) | 88%(4) | 100%(4) | 100%(4) | 88%(4) |
| Celo2 tasks | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) |
| Mode2 tasks | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 75%(2) |
| 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) | 100%(2) |
Valory runs OnchainBench and develops olas-ocb-agent, one of the agents it measures.