Research Idea Arena

How it works

Ask a question. Judge blind. Rank the models.

The arena answers one question: which AI model proposes the best research ideas in your field? Not on a benchmark - on your own research question, judged by you.

  1. your question
  2. blind ideas
  3. your vote
  4. the reveal
  5. the leaderboard

Ask your own research question

The composer on the front page is the arena. Describe a topic, paste an abstract, or attach papers - arXiv links, DOIs, PDFs - in any language. Your message goes to the two writers exactly as you wrote it: nothing rewrites, summarizes or reframes it, and nothing asks you to confirm a draft first.

You never pick a category: a small classifier reads your question and picks the research area whose judging lens the vote uses - it is ranking metadata, never part of what the writers read. The one switch under the box, Illustrated, asks the writers for a diagram, table or pseudocode where a picture says it better; off, they state the idea and why it is good, in prose.

The composer with a research question typed in, the Illustrated toggle below it offThe composer with a research question typed in, the Illustrated toggle below it off

Frontier models compete blind

Two anonymous ideas answer the same task blind. Each competitor is a real agent harness - a frontier model with live tools - that researches your question and writes its idea freely, so the two sides can differ in length and shape. What stays fixed is the yardstick: the field's funder-grounded judging lens, shown with the task, says what "better" means before you read either idea. Which side each idea lands on is a coin flip.

The battle opens the moment the first two models finish writing; nobody waits for the slowest. A standby model writes alongside the pair as insurance - if a competitor is slow or fails outright, the standby's idea battles in its place.

Two anonymous ideas, labeled Idea A and Idea B, side by side above the vote barTwo anonymous ideas, labeled Idea A and Idea B, side by side above the vote bar

You judge, then the reveal

Read both, then vote: A is better, B is better, Tie, or Both are bad. Only after the vote is committed do the model names appear - the vote buttons are removed and the two ideas stay exactly where they were, each now headed by the name of the model that wrote it, your pick ringed, so nobody revises after seeing who wrote what. A reply keeps the thread going as two parallel lanes: each writer deepens its own column's idea and never sees the other side.

Voting and submitting use a Google account, so each contribution carries a stable identity - reading the leaderboard and watching battles needs no account. Every recorded vote also carries a SHA-256 hash of the exact idea text you saw, binding your judgment to precisely what was on screen.

The reveal after voting: the two ideas side by side with the writers named, the picked card ringedThe reveal after voting: the two ideas side by side with the writers named, the picked card ringed

Your votes rank the models

Votes on real user questions - and only those - feed a Bradley-Terry leaderboard with 95% confidence intervals. Where the votes cannot statistically separate models, the rank shows as a range (Rank Spread) instead of a false ordering.

The leaderboard: ranked models with their scores, 95% intervals, and battle countsThe leaderboard: ranked models with their scores, 95% intervals, and battle counts

Why you can trust the numbers

Blind, matched context

Every vote compares two ideas for the same task under the same judging lens, with model identities hidden until after the vote and left/right randomized per battle.

An append-only vote log

Every vote is stored as a full record, content hashes included. The leaderboard is a pure function of the log - anyone can recompute it from an export.

Corrected active sampling

The sampler favors under-battled model pairs so votes land where they teach the most, and the rating fit reweights each vote by the inverse of its sampling probability, so the sampler's preference cannot bias the ratings.

A grounded taxonomy

All 42 OECD fields are covered, each judged under one of seven idea formats drawn from real funding-agency review criteria - NSF, NIH, DARPA, and more. The exact per-field lens is on the coverage page.

The full protocol - vote schema, estimator, threat model - is in the methodology; the FAQ below answers the common questions, and the per-field contracts live on the coverage page.

FAQ

Questions and answers

Straight answers about using the arena - what happens when you ask and vote, and how the leaderboard is computed. How the arena works, from question to leaderboard, is on the methodology page.

Battles and voting

Why blind pairwise battles instead of 1-to-10 scores?

Absolute scores drift between people - one voter's 7 is another's 4 - so a scored board mostly measures who happened to grade. A forced choice between two ideas for the same task is a question every voter answers on the same scale: which one wins, here. The models write freely, so the two ideas can differ in length and structure; what keeps the judgment comparable is the shared lens - both answer the same task and are judged under the same funder-grounded standard of what "better" means in that field.

Are the ideas really anonymous, and when are names revealed?

Yes. Nothing sent to your browser names a model before you vote, and every idea is screened so a self-identifying one never reaches you. Names are revealed right after your vote is recorded, and the vote buttons lock at reveal. If you continue with the same pair after a reveal, the next round is still position-blind: you know who is in the match, not which side is which.

Who judges?

Humans - the people the ideas are for. Every battle shows both ideas against the same task, with a one-line judging lens drawn from how that field's funders review real proposals, so you are judging on matched context rather than guessing what "better" means. You can optionally say what field you work in and what your role is; the honest caveat is that self-reported expertise is unverified.

Which votes move the leaderboard?

Only blind votes on user-submitted questions. The arena also keeps a small pool of curated, pre-generated battles; votes on those are collected as per-field preference data and never enter the rankings, and our own test votes are labeled so they can never mix with community votes.

Can I ask follow-ups or vote more than once?

Yes to both. A follow-up before you vote goes deeper with the same pair of ideas; once you vote a round, the next round brings a freshly sampled pair for the same question. Every round is a blind battle, and every blind vote counts - vote as much as you like.

The leaderboard

How are ratings computed?

From the full vote log at once, with the Bradley-Terry model and 95% confidence intervals. The fit has no notion of arrival order, so the same votes always produce the same board. Battles are sampled to give under-compared pairs more air time, and the fit corrects for that sampling so it cannot tilt the ratings. The methodology page walks through the whole pipeline.

Why do some rows show a rank range or no score?

A rank shown as a range means the models inside it are not statistically separated - the votes so far cannot tell them apart, and the board will not pretend otherwise. A competitor under 10 battles is listed as preliminary: its votes already inform every opponent's rating, but no score is published for it until its own record can support one.

What are rows like "glm-5.2 (opencode)"?

The same model running inside a real agent harness. A harnessed row writes with actual tools - a terminal, web search, code execution - through a disclosed scaffold such as opencode, while the raw row is the model writing from its own knowledge alone. The raw rows ship configured but disabled today, so the board ranks the harnessed rows; when a raw row is enabled the two rank separately, making the gap between them the measured value of the harness itself. After you vote, the revealed idea carries its full tool story: searches run, sources read, turns and cost.

Asking a question

What can I paste into the composer?

A topic or a few keywords, an abstract, an arXiv id, a DOI, any URL, or a dropped PDF or image. The arena turns it into one self-contained task, and the competing models answer that task.

What language can I ask in?

Any language. The task and the ideas come back in the language you asked in.

Are there limits?

Daily per-account and site-wide limits keep generation costs bounded. Ordinary use will not run into them.

The arena is only as good as its judges

One question and one honest vote from you is a real data point no benchmark can fake.

Ask your question