Coverage for orchestrant/benchmark/__init__.py: 100%

1"""Benchmark any OpenAI-compatible endpoint. 

2 

3The runner half of the family's LLM benchmark suite: one request path, endpoint 

4metrics (throughput, TTFT, decode vs prefill, time-to-answer), streaming and 

5batching probes, statistics, and provenance capture. The capability benchmarks 

6(coding, tool calling, the agent loop) live in ``benchmarks/`` -- they import 

7this package rather than owning their own transport. 

8 

9 orchestrant-bench speed --backend ollama --prompts 10 

10 orchestrant-bench lanes --batching --backend ollama 

11 orchestrant-bench report summary results.json 

12 

13Named backends come from the registry the serving stack owns 

14(``linux/llm-stack/backends.json`` in ANTfrastructure); resolution order is 

15``--base-url`` > ``LLM_BASE_URL`` / ``OLLAMA_BASE_URL`` > ``--backend <name>`` 

16> the registry's default entry. See ``orchestrant.benchmark.openai_api``. 

17""" 

18 

19from orchestrant.benchmark import client, lanes, openai_api, provenance, report, stats 

20 

21 

22__all__ = ["client", "lanes", "openai_api", "provenance", "report", "stats"]