Senior Principal Software Engineer
(LLM Inference, CUDA & GPU Performance Optimization)
Compensation: $230,000–$280,000 base + annual bonus opportunity + potential equity
Location: Remote – United States
The Company
An established global AI technology company building intelligent, real-time conversational experiences for the automotive industry. Its technology is deployed at significant commercial scale through partnerships with major vehicle manufacturers worldwide, with continued investment in advanced generative AI and high-performance inference infrastructure.
The Opportunity
We are looking for a Senior Principal Software Engineer who understands what it takes to make large language models run faster, more efficiently, and reliably in production.
This is a hands-on engineering position focused on pushing the boundaries of LLM inference performance through low-level GPU optimization, inference runtime development, and advanced memory management. Your work will directly influence product responsiveness, infrastructure economics, and the performance of real-world AI systems operating at scale.
The Role
You will take ownership of identifying and removing performance bottlenecks in production LLM inference systems, with a particular focus on cloud-hosted workloads and real-time response requirements.
Working alongside an established AI infrastructure team, you will optimize inference engines, develop CUDA kernels, and improve GPU efficiency to deliver measurable reductions in latency and improvements in throughput.
This is not a conventional MLOps, infrastructure maintenance, or LLM application development role. We are looking for someone who can get deep into inference engine internals and make the underlying systems materially faster.
What You'll Do
Optimize LLM inference performance across production systems, improving latency, tokens/sec, throughput, and GPU utilization.
Develop and tune CUDA kernels , including attention implementations and other performance-critical inference operations.
Extend and optimize inference engines such as vLLM and other high-performance runtimes, identifying opportunities for low-level code improvements.
Improve GPU memory efficiency through KV cache optimization, memory layout design, paging, and cache-aware execution strategies.
Optimize prefill and decode performance , particularly for workloads involving substantial contextual input and short, latency-sensitive responses.
Evaluate advanced optimization techniques , including quantization, continuous batching, speculative decoding, and distributed inference strategies.
Benchmark and deploy production improvements , delivering measurable gains in inference performance, stability, and cost efficiency.
What You'll Bring
Significant hands-on experience with production ML or LLM inference systems , including direct responsibility for performance optimization.
Strong C++ programming experience , ideally five or more years, with the ability to understand, modify, and optimize complex low-level code.
Practical expertise in CUDA development, GPU profiling, and kernel-level optimization .
Deep understanding of GPU architecture, memory bandwidth, memory hierarchies, concurrency, and the trade-offs involved in high-performance inference.
Experience with inference frameworks such as vLLM, TensorRT-LLM, SGLang, or comparable runtimes , ideally including modifications beyond standard configuration.
A track record of diagnosing performance bottlenecks and producing measurable improvements in latency, throughput, or infrastructure efficiency.
An independent engineering mindset, strong technical judgment, and the ability to operate effectively in a fast-moving, globally distributed team.
What This Role Requires
Real production inference experience: You must have worked directly on deploying or optimizing inference workloads beyond proof-of-concept or research environments.
Low-level engineering capability: Strong C++ and hands-on CUDA experience, with the…