Built-In PR Review with Adversarial Agent Conclaves

One review belongs to one revision

The review identity includes the repository, pull request, head SHA, model, and reviewer version. Source files, diffs, and optional repository guidance are read from pinned revisions rather than from whichever commit the branch points to later.

Runnable publishes a COMMENT review on the pull request head SHA, never on the synthetic merge commit used by some CI jobs. If a completed request is replayed, the control plane finds the existing bot-owned review and returns it instead of invoking the model or posting a duplicate.

That makes synchronize events unsurprising: a new head revision is new work; a retry of the same revision is not.

Read-only is an end-to-end boundary

The workflow step does not check out the repository or receive a workflow-visible GitHub token. Repository access and publication stay in the control plane, where authorization is tied to the active job, repository, organization, and declared action step.

The reviewer can inspect eligible text at the pinned base and head revisions. It cannot run a test, invoke a package script, follow a symlink, inspect a credential path, or turn instructions found in the repository into new permissions.

• Bounded contextUp to 50 eligible changed files and 500 KiB of cumulative source, diff, and instruction context.
• Bounded outputAt most 20 inline comments, attached only to valid lines in the reviewed diff.
• No-charge skipsForks, drafts, closed pull requests, and non-PR events skip before inference.

The reviewer has to argue against itself

A plausible bug story is not enough. Before a finding can survive, the reviewer must identify a triggering condition, compare behavior before and after the change under that same condition, show that the pull request introduced the defect, and justify severity by impact.

A fresh-context verification pass looks for counterexamples and equivalent behavior. Speculative, retracted, or disproved candidates are omitted. If the structured result or verification is invalid, the action fails operationally instead of converting uncertainty into a reassuring clean review.

This does not make model output a proof of correctness. It creates a stricter contract for what Runnable is willing to publish and a versioned evaluation gate whenever that contract changes.

Turn one reviewer into a conclave

The reviewers input lets the repository owner choose one to four independent agents. One preserves the lightweight root-reviewer and fresh-verifier path. Two to four activate a conclave: every agent reviews the same pinned change independently, then an equally sized adversarial round challenges the combined findings before anything reaches GitHub.

Independence matters more than a simple vote. Each reviewer forms its first assessment without seeing the others, an adversarial round tries to disprove every candidate, and a final adjudicator publishes only findings that survive. Disagreement becomes evidence for the adjudicator instead of being flattened into a majority vote.

Reviewer count stays inside the same explicit cost ceiling. Runnable parallelizes each round, reserves the configured maximum before starting, and refuses a model call the remaining budget cannot safely cover. More agents buy scrutiny, not an open-ended invoice.

The conclave preserves the same revision pinning, bounded repository access, evidence contract, replay safety, and failure semantics as the single-reviewer path. The backend validates the complete independent-review, challenge, adjudication sequence before it will publish a result.

• Independent first passA user-selected number of agents inspect the same pinned context without anchoring on one another's conclusions.
• Adversarial challengeAgents receive candidate findings and try to produce counterexamples, equivalence arguments, or stronger evidence.
• Adjudicated resultA final pass weighs the disagreements, publishes surviving findings, and never turns unresolved execution failure into consensus.

Spend is a workflow input, not a surprise

The action accepts three literal inputs: a model, which defaults to recommended; reviewers, which defaults to one and accepts up to four; and max-cost-usd, which defaults to $2.00. Runnable admits the review under both that per-review ceiling and the organization's billing-period limit before paid inference begins.

Under the current pricing contract, confirmed model usage is charged at provider cost divided by 0.70, targeting a 30% gross margin on model usage, plus $0.25 when a review is delivered. Runner compute and taxes are separate. A failed or cancelled review has no delivery fee, although confirmed model work already consumed can still be charged.

The step summary separates the resolved model, reviewed coverage, exclusions, confirmed model charge, delivery fee, and any provider cost still awaiting confirmation. Estimated usage is not invoiced.

Failure should look like failure

Paid inference is not automatically retried or switched to another model after an ambiguous response. Runnable records the intent before making the call, stores a completed response before returning it, and sends uncertain outcomes to reconciliation rather than purchasing another generation.

Publication follows the same rule. A stable marker lets Runnable recover a GitHub response that was delivered but lost on the network. Configuration errors, missing context, invalid output, budget rejection, and service failures fail the step; none are reported as no bugs found.

Teams that prefer review outages not to block another job can use standard continue-on-error. The important part is that the workflow makes that policy choice explicitly.

Setup stays visible in the repository

Repository owners first enable AI billing and accept the source-code processing and pricing terms. An owner or admin can then generate a standalone workflow or open a setup pull request. Runnable will not replace an existing pr-review.yml file.

The generated workflow uses an empty workflow permission set because the managed action does not depend on a checkout or a token exposed to the job. Rollout remains gated per organization, and setup appears only when the reviewer, billing path, and evaluated model are all enabled.

name: PR reviewon:  pull_request:    types: [opened, synchronize, reopened, ready_for_review] permissions: {}concurrency:  group: pr-review-${{ github.event.pull_request.number }}  cancel-in-progress: true jobs:  review:    runs-on: ubuntu-latest    timeout-minutes: 10    steps:      - uses: runnable/pr-review@v1        with:          model: recommended          reviewers: "3"          max-cost-usd: "2.00"