Discover how to turn your organisation’s data and expertise into specialised AI models. Across three webinars, Reply experts explore when specialisation justifies the investment, the technical requirements behind launching an AI model, and how to build a coordinated approach to enterprise AI.
)
webinar
Open Arena + ARBITER: Choose It, Then Judge It
How do you choose the right AI model and judge its performance? In this session, discover how Open Arena compares models on your data and ARBITER evaluates them against your own quality and safety criteria.
)
Open Arena + ARBITER: Choose It, Then Judge It
Info
November 19, 2026
1:00 – 2:00 PM GMT+1
English
About this webinar
Choose the right AI model, then judge it by your own rules.
Open Arena: Choosing a model based on evidence
Open Arena is the Reply Model Factory’s evaluation platform and the open-source counterpart to public model leaderboards. It tests candidates on your use case: open-weight or closed models, accessed via API or self-hosted, running in a sandbox within your environment.
A single leaderboard ranks candidates by quality, cost and latency, with benchmarks rerun whenever a new model is released. Evaluation combines multiple scoring metrics for each use case with human voting: people compare responses side by side without knowing which model produced them.
The starting point is your current model. Is it still the right choice, or are you paying for a closed model when a smaller, open one delivers the same quality?
ARBITER: The judge, trained in the Factory
Choosing a model also means deciding who does the scoring. Evaluation today relies on hand-written rules or large, general-purpose models that are closed and expensive.
ARBITER is a family of small, specialised models that act as an AI judge—an approach known as “LLM-as-a-judge”. Trained quickly and at very low cost in the Reply Model Factory, using the same approach and infrastructure as GEV, it determines how other models and agents are evaluated.
ARBITER-G (Generate) creates evaluation criteria from specifications, task descriptions, stakeholder feedback or execution traces. ARBITER-J (Judge) assesses actions criterion by criterion, citing the exact supporting passage for every verdict. No evidence, no verdict. ARBITER-S responds in real time to the self-checks an agent runs while it works.
Compare AI models on your own data, then check that your chosen model meets your quality and safety standards before and after release.
Why you should participate
Choose models using your own evidence. Compare quality, cost and latency on your data, combine scoring metrics with blind human voting, and reassess your choice as new models are released.
Define quality and safety on your terms. Set your own evaluation criteria and apply them consistently. Scores are calculated in code you can inspect, and every judgment is explained and traceable, in line with the AI Act.
Keep evaluation on your infrastructure. Explore small, specialised judges that run within your environment, with no per-token costs or dependence on an external provider.
Apply evaluation before and after release. Discover quality control and guardrails for customer-facing finance assistants with approval workflows, engineering copilots in energy and industry, healthcare applications and software development.
Your models, your rules, your judge.
Speakers
)
Federico minutoli
Atena Reply
)
LUCA BONALI
Atena Reply
Where and when?
November 19, 2026
13:00 ‐ 14:00 CET / 12:00–13:00 UK Time
The webinar lasts 1 hour and is conducted in English.
You can find more webinars from our series here
;Resize,width=660)
Atena Reply specializes in building and optimizing generative models tailored to specific domains, modalities, or hardware. Part of the Reply group, consisting of a network of highly specialised companies, Atena Reply supports leading European organizations in Automotive, Banking, Healthcare, Insurance, Manufacturing, Real Estate, and Telco & Media in transforming personal, professional, and domain knowledge into AI‑native operating systems: we adopt a scientific approach to generative AI, offering dataset curation, model engineering, and infrastructure for AI workers that learn from real-world interaction.