Installing ScarfBench

ScarfBench uses the scarf command line tool to pull benchmark assets, run evaluations, and work with submissions.

Install Options

Choose one installation method.

Shell Script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/scarfbench/scarf/releases/latest/download/scarfbench-cli-installer.sh | sh

Homebrew

brew tap scarfbench/tap
brew install scarfbench-cli

Cargo

cargo install scarfbench-cli

npm

npm install @scarfbench/scarfbench-cli

Source

git clone https://github.com/scarfbench/scarf.git
cd scarf
cargo install --path . --root "$HOME/.local"

Add $HOME/.local/bin to your PATH if needed.

Verify Installation

scarf --help

You should see CLI help output. For most users, the core commands needed post install are:

Group	Command	Purpose	Link
Benchmark setup	scarf bench pull	Download benchmark files for local evaluation	Command help
Agent evaluation	scarf eval	Evaluate agents on benchmark	Go to section
	scarf eval run	Run agent evaluation jobs	Command help

Getting the Benchmark

Use scarf bench pull to download the benchmark to a local directory.

Basic usage

scarf bench pull --dest /path/to/scarfbench-data

This creates or updates a local benchmark checkout in /path/to/scarfbench-data.

Pull a specific benchmark version

scarf bench pull --dest ./scarfbench-data --version <VERSION>

Replace <VERSION> with the release or version identifier you want to evaluate against.

Parameters

• --dest <DIR>: required destination directory where benchmark files will be pulled
• --version <VERSION>: optional version selector; if omitted, the latest available version is pulled

Next Step

After pulling, use scarf eval ... commands to run your agent evaluations. Continue with the Quickstart Guide to run your first evaluation.