eiboard

Static leaderboard site for evalscope LLM benchmark reports. Built with Astro + React + Tailwind, served as plain static files by nginx — no server process, no runtime secrets, no API endpoints.

The build reads JSON reports from an evalscope outputs/ tree and bakes them into a static bundle. Filtering, branding, and display options are configured via eiboard.config.yaml or environment variables.

Project structure

/
├── src/
│   ├── components/
│   │   ├── islands/          # Hydrated React components (client:load)
│   │   │   ├── LeaderboardTable.tsx     # Main sortable, filterable table
│   │   │   ├── QualityLatencyScatter.tsx# Quality vs. latency scatter chart
│   │   │   ├── CompareView.tsx          # Side-by-side model comparison
│   │   │   ├── BenchmarkBarChart.tsx    # Per-benchmark bar chart
│   │   │   ├── ModelChart.tsx           # Per-model radar/bar chart
│   │   │   └── PerfLatencyChart.tsx     # Latency distribution chart
│   │   ├── layout/           # Astro layout components
│   │   │   └── Layout.astro             # Header (brand+logo), footer, tab title
│   │   └── ui/               # Shared UI primitives (shadcn-style)
│   ├── lib/
│   │   ├── format.ts         # formatScore(score, asPercent, decimals?)
│   │   ├── utils.ts          # cn() class merge helper
│   │   ├── loader/
│   │   │   ├── loadReports.ts          # loadAllReports, loadRunCounts, loadModelIndex
│   │   │   ├── loadBenchmarkConfig.ts  # Config schema, loadSiteConfig(), env overrides
│   │   │   └── loadProvenance.ts       # Task-config provenance extraction
│   │   └── parser/
│   │       ├── parseReport.ts          # JSON → Report (with validation)
│   │       ├── parseTaskConfig.ts      # task_config.yaml parser
│   │       ├── schema.ts               # Zod schemas for report JSON
│   │       └── types.ts                # Report/Metric/Category TypeScript types
│   ├── pages/
│   │   ├── index.astro                  # Leaderboard (main page)
│   │   ├── about.astro                  # About / methodology
│   │   ├── compare.astro                # Compare page (renders CompareView)
│   │   ├── models/[model]/index.astro   # Per-model detail page
│   │   └── benchmarks/[benchmark]/index.astro  # Per-benchmark detail page
│   ├── stores/
│   │   └── compareStore.ts             # nanostores selection state for compare page
│   └── styles/
│       └── global.css                  # Tailwind + CSS chart tokens (--chart-1..5)
├── public/                  # Static assets served at site root (logos, favicon)
├── outputs/                 # evalscope report data (build-time input, gitignored)
├── test/                    # Vitest test suite
│   ├── benchmarkConfig.test.ts
│   ├── loaderFilter.test.ts
│   ├── parser.test.ts
│   └── fixtures/
├── scripts/
│   └── verify-build.sh      # Post-build HTML sanity check
├── deploy/
│   ├── nginx-eiboard.conf   # Bare-metal nginx config
│   ├── nginx-docker.conf    # Docker nginx config (absolute_redirect off)
│   └── README.md            # Deployment guide (bare-metal + watcher)
├── .forgejo/workflows/
│   ├── docker-build.yml     # Push/PR: builds tag {branch}-{sha}
│   └── docker-release.yml   # Tag v*: builds tag {tag}
├── Dockerfile               # Multi-stage: node:22-alpine build → nginx-unprivileged:alpine serve (non-root, port 8080)
├── docker-compose.yml.example
├── eiboard.config.yaml          # Active config (gitignored)
├── eiboard.config.yaml.example  # Documented template
├── Makefile                # build / deploy / dev / test / clean
├── astro.config.mjs
├── tsconfig.json           # Strict mode, @/* path alias → ./src/*
└── vitest.config.ts

Prerequisites

• Node.js ≥ 22.12.0
• evalscope reports under ./outputs/ (or point EIBOARD_DATA_DIR elsewhere)
• Optional: Docker, nginx, rsync (for deployment)

Quickstart

npm install
# Drop evalscope reports under ./outputs/ (or set EIBOARD_DATA_DIR)
cp eiboard.config.yaml.example eiboard.config.yaml   # edit benchmarks whitelist
npm run dev          # http://localhost:4321

Commands

Command	Action
npm install	Install dependencies
npm run dev	Dev server at localhost:4321
npm run build	Build static site to ./dist/
npm run preview	Preview the production build locally
npm run test	Run the Vitest test suite (npx vitest run)
npm run verify	Build + run scripts/verify-build.sh sanity check
npm run astro ...	Astro CLI (astro check, astro add, etc.)
make build	Alias for npm run build
make deploy	Build + rsync --delete dist/ /var/www/eiboard/
make clean	rm -rf dist

Configuration

eiboard is configured via eiboard.config.yaml (read at build time only). Copy the example to activate:

cp eiboard.config.yaml.example eiboard.config.yaml

Every field can also be set via an environment variable, which takes precedence over the file (env > file > built-in default). This is the primary configuration path for Docker/CI builds.

Field	Env var	Type	Default
title	EIBOARD_TITLE	string	eiboard
logo	EIBOARD_LOGO	string (path/URL)	none
description	EIBOARD_DESCRIPTION	string	LLM evaluation leaderboard...
score_as_percent	EIBOARD_SCORE_AS_PERCENT	"true"/"1"	false
show_sample_count	EIBOARD_SHOW_SAMPLE_COUNT	"true"/"1"	true
show_run_count	EIBOARD_SHOW_RUN_COUNT	"true"/"1"	false
show_title	EIBOARD_SHOW_TITLE	"true"/"1"	true
benchmarks	EIBOARD_BENCHMARKS	comma-separated	all (no file)
(data dir)	EIBOARD_DATA_DIR	string	./outputs
(config path)	EIBOARD_CONFIG_PATH	string	./eiboard.config.yaml

Benchmark whitelist

The benchmarks field lists dataset_name values to show. Every benchmark not listed disappears site-wide. If eiboard.config.yaml is absent, all benchmarks are shown. An empty list is rejected — list at least one. Find valid dataset_name values in your evalscope report JSON files.

Score formatting

formatScore(score, asPercent, decimals?) in src/lib/format.ts:

• asPercent=false (default): 0.3167 → "0.3167" (4 decimals)
• asPercent=true: 0.3167 → "31.67" (2 decimals)
• decimals overrides the default precision (e.g. chart axes use 2)

Data layout

eiboard reads evalscope reports from:

{EIBOARD_DATA_DIR}/{timestamp}/reports/{model_name}/{dataset_name}.json

• {timestamp}: YYYYMMDD_HHMMSS directory name (sorted lexicographically for dedup; non-conforming names fall back to file mtime)
• {model_name}: directory name (matches the model_name field in JSON)
• {dataset_name}.json: report file (filename matches dataset_name field)

collection_detailed_report.json is skipped — different format. Multiple runs of the same (model, benchmark) pair are merged: the displayed score is the median across all runs, while structural data (latency, sample counts, category breakdowns) comes from the most recent run. Sparse benchmarks (no report produced) simply do not appear — absent is not zero.

Deployment

Bare-metal (nginx + rsync)

See deploy/README.md for the full guide including an auto-rebuild watcher for new evalscope runs.

make deploy                    # build + rsync to /var/www/eiboard/
sudo cp deploy/nginx-eiboard.conf /etc/nginx/sites-available/eiboard
sudo ln -sf /etc/nginx/sites-available/eiboard /etc/nginx/sites-enabled/eiboard
sudo nginx -t && sudo nginx -s reload

Docker

eiboard is a static site — evalscope report data is baked into the Astro bundle at image build time, not read at runtime. Rebuild the image whenever evalscope emits new reports.

cp docker-compose.yml.example docker-compose.yml
# Edit: set additional_contexts "outputs=" to your evalscope outputs path
docker compose up -d --build
# → http://localhost:8080

The Dockerfile is multi-stage: node:22-alpine builds the Astro bundle, nginxinc/nginx-unprivileged:alpine serves it on port 8080 as non-root (UID 101). The evalscope outputs/ tree is brought into the build via additional_contexts (Compose v2.17+) so no data needs to be copied into the repo. All branding/display config is passed as build args. The logo SVG is a runtime volume mount so it can be swapped without rebuilding.

Forgejo CI

Two workflows in .forgejo/workflows/:

• docker-build.yml — triggers on push (any branch) and PRs. Builds and pushes forge.engelmann.me/$REPO:{branch}-{sha}.
• docker-release.yml — triggers on tags matching v*. Builds and pushes forge.engelmann.me/$REPO:{tag}.

Both require the PACKAGE_USER and PACKAGE_TOKEN repository secrets.

Testing

npm run test

Vitest test suite covers config parsing, env-var overrides, the report loader, and the parser. Test fixtures live under test/fixtures/.

Security

eiboard's build only extracts a whitelisted set of fields from task_config.yaml for display. It never emits to dist/:

• api_url, api_key, work_dir, dataset_dir

The nginx config and the deployed static bundle contain no credentials and no references to sensitive paths.

Documentation

• Astro: https://docs.astro.build
• Deployment guide: deploy/README.md