We are seeking an experienced, hands-on Principal Data Engineer to play a key technical role within our Enterprise Data Products team.
Enterprise Data Products is progressing through a significant technology transformation and rearchitecture. You will join that journey in progress, quickly understand the existing landscape and agreed direction, and get hands-on in helping turn it into effective engineering solutions.
We are looking for someone who can add value quickly: bringing deep engineering experience, solving difficult problems, introducing new ideas and constructively challenging approaches where there is a better way. You will be technically curious, pragmatic and comfortable working with evolving or high-level requirements.
The Role
This is a senior individual contributor role with no direct people-management responsibility, reporting to the Technical Lead.
You will bring deep technical expertise and hands-on engineering experience across complex delivery, solution design, prototyping and problem solving. Working closely with the Technical Lead and wider engineering team, you will help evaluate technical options, develop pragmatic implementation approaches and turn agreed architectural direction into working solutions.
You will also act as a trusted technical sounding board, supporting engineers through practical collaboration, review, coaching and knowledge sharing across our geographically distributed team.
Solution Architecture & Technical Design
• Quickly understand the existing platform, architectural direction and transformation roadmap.
• Contribute hands-on expertise to the implementation and evolution of the target architecture.
• Develop solution designs and engineering approaches aligned with the agreed technical direction.
• Evaluate implementation options, balancing scalability, maintainability, performance, cost and delivery.
• Constructively challenge existing approaches and prototype alternatives where appropriate.
• Identify technical risks, dependencies and unknowns early and propose pragmatic solutions.
Data Engineering
• Design and build scalable, resilient and maintainable enterprise data solutions.
• Apply strong engineering practices using Databricks, Apache Spark/PySpark, Python, SQL and modern lakehouse technologies.
• Develop reusable patterns for data ingestion, transformation, integration and delivery.
• Design for incremental processing, schema evolution, reprocessing, performance and cost optimisation.
• Apply automated testing, CI/CD, observability, data quality, lineage and appropriate engineering controls.
• Investigate and resolve complex data, integration and platform problems.
Innovation & Continuous Improvement
• Explore technologies and engineering approaches that could improve Enterprise Data Products.
• Prototype promising ideas to validate their value and resolve technical unknowns.
• Identify opportunities to simplify solutions, automate activity and improve engineering effectiveness.
• Promote technical curiosity, experimentation and continuous improvement while maintaining engineering rigour.
Application & Integration Engineering
• Apply strong knowledge of modern application, API, authentication and cloud integration patterns.
• Work across data, application and platform technologies to develop coherent end-to-end solutions.
• Collaborate with product and engineering teams to translate business needs into practical technical solutions.
Technical Influence & Engineering Support
• Use technical expertise and hands-on experience to influence engineering decisions and approaches.
• Act as a technical sounding board for complex or unfamiliar problems.
• Support engineers through pairing, working examples, code and design reviews, coaching and knowledge sharing.
• Contribute to pragmatic engineering standards and reusable patterns.
• Help strengthen capability across a geographically distributed engineering team.
• Communicate technical options, trade-offs and recommendations…