About this role
Business Overview
The Risk & Quantitative Analysis (RQA) group provides independent oversight of BlackRock’s fiduciary and enterprise risks. RQA’s principal objectives are to advance the firm’s risk management practices and to deliver independent risk expertise and constructive challenge to drive better business and investment outcomes.
RQA is committed to investing in our people to increase individual enablement and ultimately build a stronger team. Our goal is to create a culture of inclusion which encourages collaboration, innovation, diversity and the development our future leaders. We actively engage in discussions on career growth and work with team members to understand how personal passions and strength connect with our purpose.
Role Description:
The Enterprise Risk team within RQA oversees those risks which directly impact the corporate entity (i.e. BlackRock, Inc., or one of its subsidiaries), rather than the fiduciary risks assumed by funds/clients, such as investment risks and counterparty risks. The team’s focus includes but is not limited to operational risk, technology, vendor, model and market risk, stress testing for market risk and the quantitative estimation of capital adequacy requirements.
The AI Validation Associate will perform independent reviews of AI tools and agents used across BlackRock, including agentic systems, RAG applications and other business workflows. This role requires strong technical expertise combined with risk management skills to challenge system design, data integrity, and performance monitoring. The role will be based onsite in our GGN office.
Key Responsibilities:
Conduct independent validation of AI usage in workflows, including testing for accuracy, efficiency of guardrails, orchestration, memory management and token usage.
Design and implement comprehensive validation strategies and test plans tailored to each AI use case. Define clear success criteria and quality benchmarks to measure whether the AI solution meets all requirements and standards.
Understand tool architecture, potential vulnerabilities and weaknesses.
Assess conceptual soundness, methodology, and alignment with intended business use.
Review data quality, preprocessing steps and engineering decisions for AI use cases.
Execute functional and performance tests.
Evaluate performance metrics, robustness, and limitations under various scenarios.
Document validation findings, provide recommendations
Stay current on emerging AI technologies, validation techniques, and regulatory developments.
Communicate validation conclusions to relevant stakeholders, including escalating identified findings, risks or gaps to senior leadership and approval committees
Collaborate with AI Engineering, Data Science, and Risk teams to enhance AI transparency and trustworthiness.
Experience, Skills and Qualification:
With 3-7 years’ works experience in the IT or financial industry.
Degree or Advanced degree (MS, PhD) in Computer Science, Data Science, Statistics, Applied Mathematics, or related field.
Strong understanding of generative AI technologies, including large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, prompt engineering, tool use, and modern AI application architectures.
Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow or similar).
Excellent analytical, communication, and documentation skills.
Ability to work independently and challenge stakeholders constructively.
Experience with one or more of the following:
Validating usage of large language models (LLMs) in business context
Evaluation methodologies, benchmark development, ground-truth creation, and AI quality assessment
AI observability, monitoring, performance measurement, and operational controls
AI orchestration frameworks, agent architectures, prompt engineering, memory management, and context optimization
Exposure to cloud-based AI platforms and scalable model…