Company:
Marsh
Description:
Marsh Management Consulting is looking for a hands-on Senior Software Engineer with strong software engineering fundamentals and deep practical experience building production Generative AI applications. The role will help build and own the next generation of AI-powered experiences across Marsh Management Consulting digital platforms.
The successful candidate will help establish reusable internal AI capability spanning retrieval, orchestration, evaluation, guardrails, content ingestion, APIs, observability, and integration with the website experience. The role will work closely with product, UX, content, AEM/web engineering, architecture, analytics, security, and external delivery partners, while building the knowledge required for Marsh Management Consulting to own and evolve the platform over time.
We will count on you to:
AI engineering and architecture
Design, build, and operate production-grade LLM applications using retrieval-augmented generation (RAG), semantic search, prompt orchestration, and agentic patterns where appropriate.
Build conversational experiences incorporating intent detection, session context, multi-turn interactions, structured outputs, tool/function calling, and streaming responses.
Develop reusable AI services and APIs rather than tightly coupling AI logic to individual website features.
Own technical decisions around model selection, retrieval strategy, orchestration, performance, and AI application architecture.
Search, content and RAG
Design retrieval strategies using embeddings, vector search, keyword/hybrid retrieval, metadata filtering, reranking, and contextual retrieval.
Build and maintain content ingestion pipelines that crawl, process, chunk, enrich, embed, and index Oliver Wyman website content.
Work with content and AEM teams to use taxonomy, tagging, metadata, and content lifecycle signals to improve retrieval quality and control AI inclusion and exclusion.
Implement source attribution and citation mechanisms so generated answers can be traced to authoritative Oliver Wyman content.
Continuously improve relevance and retrieval performance using quantitative evaluation and user feedback.
AI quality, evaluation and safety
Build systematic evaluation frameworks for retrieval accuracy, groundedness, answer quality, brand and tone alignment, and safety.
Create automated evaluation datasets and regression testing so model, prompt, and retrieval changes can be assessed before deployment.
Implement safeguards against hallucination, prompt injection, inappropriate content, PII exposure, and ungrounded responses.
Establish feedback loops and monitoring that enable continuous improvement of AI quality.
AWS and production engineering
Design and operate cloud-native AI services on AWS, ideally using services such as Amazon Bedrock, OpenSearch, Lambda, API Gateway, S3, ECS/Fargate, DynamoDB, and Redis.
Build secure, observable, and scalable services with appropriate logging, tracing, monitoring, and auditability.
Work with infrastructure teams on CI/CD and Infrastructure-as-Code using technologies such as Terraform and GitHub Actions.
Optimize AI services for reliability, latency, scalability, and cost.
Digital experience integration
Partner closely with AEM and front-end engineers to integrate AI capabilities into the Oliver Wyman website and its transition to AEM Cloud.
Design APIs and event/context models that allow the AI Companion to understand the current page, content, and user journey.
Enable conversational search, assisted navigation, contextual prompt suggestions, relevant content and tool recommendations, contextual calls-to-action, and TL;DR summaries.
Support the evolution from an AI assistant layered over the website toward AI-enhanced components and more personalized digital experiences.
What you need to have:
Approximately 6+ years of professional software engineering experience, with meaningful recent experience delivering Generative AI applications into…