Are you looking for more in life than just building another data pipeline? Does upending cybersecurity resonate with you? We're a rapidly expanding cybersecurity company revolutionizing vulnerability management for organizations of all sizes. For our customers, vulnerability management has historically been a game of catch-up, with fragmented data, limited asset coverage, and manual processes.
Nucleus' primary mission is to create a fast, scalable platform that brings together large volumes of security data and transforms it into actionable insights, making vulnerability management simple, efficient, and effective.
We're looking for a Staff / Principal Data Engineer to help define and evolve the architecture behind our data platform. This is a senior technical leadership role for an engineer who thrives on solving complex data problems at scale, designing systems that process large and growing datasets, and establishing patterns that other engineers can build upon.
What You Will Do
Own the architecture and evolution of large-scale data processing and analytics systems, from ingestion and normalization through transformation, aggregation, and customer-facing analytics.
Design and evolve data models that support high-volume analytical workloads while balancing query performance, storage efficiency, maintainability, and scalability.
Design and optimize complex analytical SQL queries operating across large datasets, identifying performance bottlenecks and developing scalable solutions.
Architect distributed and asynchronous job processing systems capable of reliably processing large volumes of data across concurrent workloads.
Design and evolve Spark, Flink or similar technologies based data processing workflows for large-scale transformation, enrichment, aggregation, and computation.
Drive the effective use of SingleStore/MemSQL, including schema design, indexing and partitioning strategies, query optimization, and workload design.
Establish architectural patterns for batch processing, data transformation, job orchestration, failure recovery, idempotency, and reprocessing.
Diagnose complex performance and scalability issues across databases, distributed processing systems, application services, and infrastructure.
Partner with engineering and product teams to translate ambiguous analytical and data-processing requirements into scalable technical architectures.
Define technical standards and reusable patterns for data modeling, data processing, and analytical query development.
Provide technical leadership across engineering initiatives, influencing architecture and mentoring engineers while remaining deeply hands-on with implementation.
Identify architectural limitations before they become operational problems and lead improvements that increase the scalability, reliability, and performance of the platform.
The Following Skills and Experience Are Key to Succeeding in This Role
10+ years of professional software engineering, data engineering, or data platform experience, with significant experience designing and operating production data systems.
Deep expertise in relational and analytical data modeling, including designing schemas for large-scale analytical and reporting workloads.
Expert-level SQL, with demonstrated experience designing, analyzing, and optimizing complex analytical queries over large datasets.
Strong experience with SingleStore/MemSQL or comparable distributed SQL/analytical database technologies.
Significant hands-on experience with Spark, Flink or similar technologies, including designing and optimizing distributed data-processing workloads.
Strong understanding of distributed data processing, including partitioning, parallelism, data skew, resource utilization, and performance optimization.
Experience designing high-throughput job processing systems, including scheduling, concurrency management, retries, failure recovery, idempotency, and observability.
Experience designing data pipelines that support batch…