Protocols
Completion per protocol, from the tasks’ protocol tags: the share of each protocol’s tasks the entry completed, with the number of tasks it was scored on. A task that uses several protocols counts under each. Results describe how agents use a protocol, never the protocol’s quality.
| Protocol | Claude CodeDeepSeek V4.1 Flash | Claude CodeOpus 5.5 | OpenClawDeepSeek V4.1 Flash | CodexGPT-5.6 Sol | HermesDeepSeek V4.1 Flash | olas-ocb-agentDeepSeek V4.1 Flash | CodexDeepSeek V4.1 Flash |
|---|---|---|---|---|---|---|---|
| ERC-2086 tasks | 94%(86) | 90%(86) | 91%(86) | 92%(86) | 90%(86) | 89%(86) | 86%(86) |
| Wrapped native asset38 tasks | 97%(38) | 94%(38) | 94%(38) | 97%(38) | 91%(38) | 88%(38) | 91%(38) |
| Aave v333 tasks | 90%(33) | 86%(33) | 85%(33) | 94%(33) | 83%(33) | 83%(33) | 78%(33) |
| Long-tail contracts29 tasks | 97%(29) | 97%(29) | 94%(29) | 92%(29) | 94%(29) | 92%(29) | 92%(29) |
| Uniswap v325 tasks | 98%(25) | 98%(25) | 96%(25) | 94%(25) | 95%(25) | 98%(25) | 96%(25) |
| Omen15 tasks | 100%(15) | 99%(15) | 99%(15) | 100%(15) | 97%(15) | 87%(15) | 89%(15) |
| Web research15 tasks | 100%(15) | 100%(15) | 100%(15) | 100%(15) | 99%(15) | 97%(15) | 99%(15) |
| Native asset11 tasks | 100%(11) | 100%(11) | 100%(11) | 84%(11) | 96%(11) | 91%(11) | 95%(11) |
| Olas mech marketplace10 tasks | 100%(10) | 98%(10) | 98%(10) | 100%(10) | 96%(10) | 86%(10) | 90%(10) |
| 90%(2) | 100%(2) | 70%(2) | 50%(2) | 90%(2) | 80%(2) | 40%(2) | |
| 90%(2) | 100%(2) | 70%(2) | 50%(2) | 90%(2) | 80%(2) | 40%(2) | |
| 100%(2) | 100%(2) | 100%(2) | 100%(2) | 70%(2) | 90%(2) | 80%(2) | |
| Honeyswap1 task | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) |
| Sushiswap v21 task | 100%(1) | 80%(1) | 80%(1) | 80%(1) | 100%(1) | 80%(1) | 80%(1) |
| Protocol | Claude CodeOpus 5.5 | Claude CodeDeepSeek V4.1 Flash | OpenClawDeepSeek V4.1 Flash | CodexDeepSeek V4.1 Flash | HermesDeepSeek V4.1 Flash | CodexGPT-5.6 Sol | olas-ocb-agentDeepSeek V4.1 Flash |
|---|---|---|---|---|---|---|---|
| ERC-2035 tasks | 99%(35) | 99%(35) | 99%(35) | 96%(35) | 97%(35) | 93%(35) | 91%(35) |
| Wrapped native asset31 tasks | 100%(31) | 95%(31) | 97%(31) | 100%(31) | 92%(31) | 100%(31) | 92%(31) |
| Aave v310 tasks | 100%(10) | 95%(10) | 100%(10) | 95%(10) | 95%(10) | 100%(10) | 90%(10) |
| Long-tail contracts8 tasks | 94%(8) | 94%(8) | 94%(8) | 100%(8) | 94%(8) | 88%(8) | 94%(8) |
| Uniswap v38 tasks | 100%(8) | 100%(8) | 100%(8) | 100%(8) | 88%(8) | 100%(8) | 81%(8) |
| Web research5 tasks | 100%(5) | 100%(5) | 100%(5) | 100%(5) | 100%(5) | 90%(5) | 100%(5) |
| Olas mech marketplace4 tasks | 100%(4) | 75%(4) | 88%(4) | 100%(4) | 75%(4) | 100%(4) | 88%(4) |
| Omen4 tasks | 100%(4) | 75%(4) | 88%(4) | 100%(4) | 88%(4) | 88%(4) | 100%(4) |
| 75%(4) | 100%(4) | 88%(4) | 50%(4) | 88%(4) | 63%(4) | 75%(4) | |
| 100%(4) | 100%(4) | 100%(4) | 100%(4) | 100%(4) | 88%(4) | 100%(4) | |
| Honeyswap1 task | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) |
| Native asset1 task | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) |
| 50%(1) | 100%(1) | 100%(1) | 50%(1) | 100%(1) | 0%(1) | 50%(1) | |
| Sushiswap v21 task | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) | 100%(1) |
Valory runs OnchainBench and develops olas-ocb-agent, one of the agents it measures.