Developer focused on data lineage to build systems that parse and analyze large, multi-language codebases and automatically generate accurate data lineage. The role pairs strong Python engineering with hands-on Gen AI/LLM work and ownership of build, deployment, and delivery pipelines. The emphasis is on clean, efficient, well-architected design — systems that are correct, performant, and maintainable as they scale.
What You'll Do
Design and build engines that parse multiple programming and query languages to extract accurate, column-level data lineage.
Build Gen AI/LLM-powered retrieval and assistant capabilities on top of lineage and metadata.
Develop backend services and the interfaces that expose them, from data processing through to user-facing views.
Own the delivery pipeline — build, test, deployment, and production support across environments.
Required Skills
Strong Python development with solid object-oriented and functional design skills.
Software engineering fundamentals: data structures, algorithms, and complexity analysis.
Data lineage / data flow analysis: parsing and analyzing source code (AST-based and pattern-based static analysis) and comfort reading multiple languages.
Hands-on Gen AI/LLM experience: prompt design, tool/function calling, and agent orchestration.
Understanding of retrieval-augmented generation and vector/semantic search concepts.
Experience designing APIs and backend services, plus the front-end fundamentals to build usable interfaces.
Pipeline and deployment experience: build/test automation, CI/CD, and releasing across environments.
Data manipulation and transformation skills.
Nice to Have
Graph algorithms and traversal techniques for dependency/lineage tracing.
Data governance, metadata management, and data cataloging concepts.
Experience designing systems for long-running or asynchronous workloads.
Infrastructure automation and deployment tooling.
Key Responsibilities
Translate ambiguous requirements into clean, efficient, well-structured designs.
Extend parsing and analysis capabilities to new languages and frameworks without compromising architecture.
Design and productionize Gen AI/LLM and retrieval features that are accurate and grounded in real data.
Build and maintain deployment pipelines; support delivery across environments.
Write tests and maintain quality across the development lifecycle.
Collaborate on design reviews and raise the engineering bar for the team.
Recommended Qualifications:
7-10 years of relevant experience
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Education:
Bachelor’s degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Design Principles We Value
Simplicity first — favor the simplest design that solves the problem; avoid unnecessary complexity and premature abstraction.
Separation of concerns — clear module boundaries, single-responsibility components, and well-defined interfaces so pieces can evolve independently.
Correctness and determinism — prefer predictable, testable behavior; ground AI outputs in verifiable data rather than guesswork.
Performance-aware design — think about time/space complexity, memory footprint, and efficient data structures and algorithms up front.
Scalability and extensibility — design so new languages, sources, and capabilities can be added without rewrites.
Robustness — graceful error handling, fault isolation, and defensive design so one bad input can't take the system down.
Readability and maintainability — code and designs the next engineer can understand, extend, and debug.
Reuse over duplication — shared models and common…