Getting started

Five minutes from clone to a probability. Everything runs locally; nothing phones home.

1. Install

Python 3.10 or newer. Install the PyTorch 2.8.0 wheel for your platform first (CUDA, CPU or Apple silicon), then the package and the trained checkpoint.

git clone https://github.com/erphq/openauditor && cd openauditor
pip install -e .

gh release download v0.3.0 -R erphq/openauditor -p openauditor-model.tar.gz -p SHA256SUMS -D artifacts
cd artifacts && shasum -a 256 -c SHA256SUMS --ignore-missing && tar -xzf openauditor-model.tar.gz && cd ..

artifacts/model/ is the one canonical checkpoint: tokenizer, configuration and a single merged weight file. Verify the checksum before loading.

2. First call

from openauditor import OpenAuditor, Choice, Noul, Score

auditor = OpenAuditor.from_pretrained("artifacts/model", device="cuda")  # or "mps", "cpu"

result = auditor.review(
    {"message": "Our payment integration is down and customers cannot check out."},
    {
        "team": Choice("Which team should own this?", {
            "billing": "Payments, charges, refunds, invoices",
            "technical": "Bugs, APIs, outages, integrations",
            "sales": "Pricing, quotes, upgrades, plans",
        }),
        "urgent": Noul("Does this need urgent attention?"),
        "frustration": Score("How frustrated is the customer?", ["calm", "frustrated", "very frustrated"]),
    },
)
answers = result.to_dict()["answers"]
answers["team"]["choice"]              # "technical"
answers["team"]["probabilities"]       # {"billing": 0.04, "technical": 0.93, "sales": 0.03}
answers["urgent"]["noul"]              # 0.91
answers["frustration"]["score"]        # 1.6  (expected rubric index)
result.usage                           # {"input_tokens": 41, "output_tokens": 0, "candidate_token_evaluations": 716}

The state can be any JSON value: a string, an object, a list. It is serialized canonically and shared across every question in the call. Questions are a map of your own keys to typed questions. See the API reference for every field.

3. CLI

openauditor infer --model artifacts/model --request examples/request.json --device cuda

The request file has the same state and questions shape as the HTTP body. Output is the JSON response on stdout.

4. Local server

openauditor serve --model artifacts/model --device cuda --port 8080

curl -s localhost:8080/v1/review \
  -H 'content-type: application/json' \
  -d '{"state": {"message": "Refund the duplicate charge"},
       "questions": {"refund": {"type": "noul", "instructions": "Is a refund requested?"}}}'

5. Train on your own data

One directory with train.jsonl, validation.jsonl and test.jsonl, plus an optional heldout.jsonl of task families the model never sees. One row per decision:

{"group": "ticket-001",
 "task": "routing",
 "state": {"message": "Please refund a duplicate charge"},
 "questions": {
   "team":     {"type": "choice", "instructions": "Which team handles this?", "criteria": {"billing": "Charges and refunds", "technical": "Software faults"}},
   "refund":   {"type": "noul",   "instructions": "Is a refund requested?"},
   "priority": {"type": "score",  "instructions": "Rate urgency.", "criteria": ["Routine", "Time-sensitive", "Emergency"]}},
 "targets": {"team": "billing", "refund": true, "priority": 0}}

• Choice targets: a candidate key, or a distribution over keys.
• Noul targets: true/false, or a probability in [0, 1].
• Score targets: a zero-based rubric index (fractions interpolate), or a distribution.
• group keeps related rows in one split. Never put the same customer, session or duplicate state in two splits.
• task is a free label used for per-task metrics and macro averages.

openauditor validate --data data/mine
openauditor train --data data/mine --output runs/mine --device cuda
openauditor infer --model runs/mine/model --request examples/request.json

Training runs supervised warmup, native environment RL, validation-only temperature calibration, export, and a frozen test evaluation. The output you keep is runs/mine/model/. Everything else in the run directory is resume state and evidence. Use --resume after an interruption with the same data and settings. Around five to twenty thousand labeled decisions is a sensible starting size; the public checkpoint used about fifteen thousand.

To rebuild the public recipe from scratch: openauditor data --output data/public downloads the pinned public datasets and generates the synthetic tasks. See TRAINING.md for the recipe, the curriculum extension, calibration and evaluation details.

Hardware

The model is 0.6B parameters. Inference needs about 1.5 GB of memory in BF16. Every candidate answer is one forward pass, so latency scales with the number of options, not with output length.