eiscore

Intelligence score API for EWCS LLMs. For every chat model offered at https://ai.ewcs.ch/v1/models, the API returns a score from 0.00 to 5.00 that expresses how capable the model is relative to current frontier models.

Data source: the Artificial Analysis Intelligence Index (via the official Data API).

Attribution: Data by Artificial Analysis — required by their terms when displaying API data; every response carries it in meta.attribution.

Scoring

score = aa_intelligence_index / max_aa_intelligence_index * 5.00   (rounded to 2 decimals)

The maximum across the whole AA feed is the anchor (currently claude-opus-5-5 ≈ 57.6 → 5.00). Example: gpt-oss-120b (11.6) → 11.6 / 57.6 * 5 = 1.01.

Models without an AA entry (internal names, embeddings, rerankers, speech, image models) get score: null and status: "NA".

Freshness

The AA feed is fetched once per UTC day — on the first request that day — and stored in a local SQLite database (data/eiscore.db, snapshots are kept as history). If Artificial Analysis is unreachable, the API serves the last snapshot with meta.stale: true instead of failing. A forced refresh is available via POST /v1/admin/refresh.

Endpoints

Method	Path	Description
GET	/v1/scores	Scores for all EWCS chat models
GET	/v1/scores/{model_id}	Score for one model (id as in the list)
GET	/v1/aa/models	Raw AA snapshot (debug)
POST	/v1/admin/refresh	Force AA refresh (guarded by EISCORE_ADMIN_TOKEN)
GET	/healthz	Liveness probe

Example

{
  "data": [
    {
      "model_id": "ew/inference-gpt-oss-120b",
      "normalized_name": "Gpt Oss 120B",
      "aa_slug": "gpt-oss-120b",
      "score": 1.01,
      "status": "ok",
      "aa_intelligence_index": 11.6,
      "match": "exact"
    },
    {
      "model_id": "ew/dex",
      "normalized_name": "Dex",
      "aa_slug": null,
      "score": null,
      "status": "NA",
      "match": "none"
    }
  ],
  "meta": {
    "reference": {"aa_slug": "claude-opus-5-5", "aa_intelligence_index": 57.6},
    "intelligence_index_version": "4.3",
    "snapshot_fetched_at": "2026-10-09T08:00:00+00:00",
    "stale": false,
    "models_stale": false,
    "model_count": 12,
    "attribution": "Data source: Artificial Analysis (https://artificialanalysis.ai)"
  }
}

Interactive docs: /docs (Swagger UI) and /redoc.

Configuration

Variable	Required	Default	Description
ARTAN_KEY	yes	—	Artificial Analysis API key
EISCORE_AA_BASE_URL	no	https://artificialanalysis.ai/api/v2	AA API base URL
EISCORE_EWCS_MODELS_URL	no	https://ai.ewcs.ch/v1/models	EWCS model list
EISCORE_DB_PATH	no	data/eiscore.db	SQLite file
EISCORE_EWCS_CACHE_TTL_SECONDS	no	3600	EWCS model-list cache TTL
EISCORE_CORS_ORIGIN_REGEX	no	.*	Allowed CORS origins (regex; matched origin is echoed). Default mirrors ai.ewcs.ch — any origin, no credentials
EISCORE_ADMIN_TOKEN	no	—	When set, protects /v1/admin/refresh
EISCORE_LOG_LEVEL	no	INFO	Log level

Run

uv sync
ARTAN_KEY=... uv run uvicorn eiscore.main:app --host 0.0.0.0 --port 8300

Docker:

docker build -t eiscore .
docker run -p 8300:8300 -e ARTAN_KEY=... -v eiscore-data:/app/data eiscore

Development

uv sync                      # install deps
uv run ruff check .          # lint
uv run ruff format --check . # formatting
uv run basedpyright          # types
uv run pytest                # tests

Mapping

EWCS ids are resolved to AA slugs in this order:

1. Alias overrides (table alias_overrides in SQLite; patterns support fnmatch wildcards)
2. Exact match — alphanumeric comparison of the EWCS normalized_name, then the cleaned model id (strip ew/, inference-, quantization suffixes like -fp8), against AA slugs
3. Unique prefix match against "base" slugs (AA effort/reasoning variants like -low/-high/-reasoning are excluded from base matching)
4. No match → status: "NA"