- Pay
- $100–150k
- Where
- UK · Ireland · Delaware · France · Newfoundland and Labrador · Spain · Portugal · Italy · Switzerland · Poland · Sweden · Denmark · Belgium · Remote, Austria · Remote
- Level
- Mid
Plexus has partnered with an early-stage AI company building an evaluation platform that helps enterprise teams validate generative AI models and agents before they go live. The product is already live, with a public leaderboard benchmarking leading LLMs, and the team is now growing.
The role
You will build the infrastructure that runs evaluations of frontier models and agents at scale. This is a hands-on role with real ownership, working directly with the founders.
Responsibilities
Build scalable pipelines for benchmark execution, scoring and results aggregation across multiple LLM providers and custom endpoints
Develop systems that generate custom evals on demand from customer data
Design secure, multi-tenant services and APIs to enterprise privacy standards
Own orchestration, observability, performance and cost as usage grows
Experience required
Proven experience building AI systems and AI infrastructure in production
Strong Python (or similar) and backend or distributed systems skills
Hands-on experience with LLM APIs and open or self-hosted models
Cloud (AWS, GCP or Azure), Docker/Kubernetes and infrastructure-as-code
Comfortable in an early-stage, fast-moving environment
Nice to have
Experience with LLM evaluation, benchmarking or agent frameworks
Startup or scale-up background
Comp: Base + Equity
If this sounds like something you’d be interested in, please apply with your most up-to-date CV here (jayden.lazarus@plexusrs.com) or via TG: @JaydenLaz_12