Is the Staff Software Engineer, LLM Inference Platform position at Zenlayer, Inc fully remote?

Yes, this Staff Software Engineer, LLM Inference Platform position at Zenlayer, Inc is a remote position. The role is available for candidates in United States.

Is this a full-time or part-time Staff Software Engineer, LLM Inference Platform role?

This is a full-time Staff Software Engineer, LLM Inference Platform position at Zenlayer, Inc.

What is the salary range for the Staff Software Engineer, LLM Inference Platform position?

The salary range for this Staff Software Engineer, LLM Inference Platform role is $300k - $600k.

How do I apply for the Staff Software Engineer, LLM Inference Platform position at Zenlayer, Inc?

You can apply for this Staff Software Engineer, LLM Inference Platform position by clicking the "Apply Now" button on this page, which will take you to Zenlayer, Inc's application portal.

Staff Software Engineer, LLM Inference Platform at Zenlayer, Inc

United States

$300k - $600k

About Zenlayer Zenlayer is the first massively distributed edge cloud provider, operating 290+ Points of Presence across 70+ countries. With accolades including PTC “Best Networking Innovation”, “Outstanding Cloud, Data Center, or Interconnection Company”, and Inc. 5000, we enable businesses to deploy and run applications at the edge, thus reducing network latency to the end user. Zenlayer focuses on improving connections worldwide, all while nurturing a unique company culture that values service, growth, and trust. As a “Zenplayer”, you’ll be joining a community of success-driven team players that share the dream of powering a better-connected world. About the Role: As the technical lead for inference performance, you will own end-to-end responsibility for the platform's Token Performance. For models ranging from hundreds of billions to trillions of parameters (including MoE), you will continuously reduce cost per million tokens and increase per-GPU Goodput while meeting TTFT/TPOT SLOs, and prove the gain from every optimization with reproducible data. What You Will Work On 1. End-to-End Performance Attribution and Benchmarking <ul> <li>Establish a unified Token Performance metrics framework (TTFT, TPOT, Goodput, GPU utilization, per-token cost), and build load-testing benchmarks that reflect real production traffic (high concurrency, long context, multi-turn Agent workloads) </li> </ul> <ul> <li>Perform bottleneck attribution for models such as DeepSeek, Kimi, GLM, Qwen, and MiniMax across compute, memory bandwidth, communication, scheduling, and queuing, and let the data determine which layer is the dominant constraint </li> </ul> 2. Kernel-Level Inference Engine Optimization <ul> <li>Dive deep into the source code of vLLM / SGLang / TensorRT-LLM, customizing and contributing upstream on critical paths such as the scheduler, Continuous Batching, Chunked Prefill, CUDA Graph, and KV block management </li> </ul> <ul> <li>Clearly define the responsibility boundary between the inference engine and orchestration layers such as Dynamo / llm-d, and reduce tail latency through request routing and load balancing </li> </ul> 3. PD Disaggregation and KV Cache Transfer <ul> <li>Lead the core design of Prefill-Decode disaggregation: the KV transfer path (NIXL/RDMA/UCX), transfer-compute overlap, and phase-aware autoscaling </li> </ul> <ul> <li>Define the failure domains and recovery semantics of cross-instance KV transfer, and answer the key question: under which workloads does PD disaggregation deliver gains, and under which does it make things slower </li> </ul> 4. MoE Parallelism and Wide-EP <ul> <li>Design hybrid TP/PP/DP/EP parallelism strategies and quantify the trade-offs between EP and TP </li> </ul> <ul> <li>Tackle expert load imbalance and All-to-All communication overhead, and drive Wide-EP into production </li> </ul> 5. Kernels and Quantization <ul> <li>Write and tune critical CUDA (and ROCm/HIP) operators: memory access optimization, kernel fusion, and Attention / MoE / communication-related kernels </li> </ul> <ul> <li>Build a closed-loop "quantization – accuracy – throughput" evaluation: the real-world gains and accuracy regressions of FP8/FP4/INT4/AWQ/GPTQ across different engines and hardware </li> </ul> Requirements <ul> <li>First-principles understanding of Transformer inference: the compute vs. memory-bound nature of Prefill and Decode, the KV Cache memory model, the differences among MHA/GQA/MLA, and how MoE routing constrains parallelism </li> </ul> <ul> <li>Source-level, in-depth experience with at least one mainstream inference engine, with real cases of identifying and fixing bottlenecks in scheduling, kernel selection, or KV management </li> </ul> <ul> <li>CUDA kernel tuning skills (Nsight profiling, memory optimization, fusion, CUDA Graph) with hands-on practical experience </li> </ul> <ul> <li>Bachelor’s degree in Computer Science, Electrical/Computer Engineering, or related field (HPC systems, ML Systems). </li> </ul> <ul> <li>Minimum 6 years in systems/Performance Engineering, with at least 2-3 years directly on LLM or GPU performance work </li> </ul> Preferred, but not required: <ul> <li>Experience with Merged PRs in vLLM </li> </ul> <ul> <li>Experience operating at cluster sale (hundreds to thousands of GPUs) </li> </ul>

Apply Now

Remote Opportunity

Staff Software Engineer, LLM Inference Platform

Join Zenlayer, Inc as a senior professional working remotely from United States. Explore the role, benefits, and apply in one place.

Staff Software Engineer, LLM Inference Platform

Zenlayer, Inc

Full Time

$300k - $600k

2 weeks ago

About Zenlayer Zenlayer is the first massively distributed edge cloud provider, operating 290+ Points of Presence across 70+ countries. With accolades including PTC “Best Networking Innovation”, “Outstanding Cloud, Data Center, or Interconnection Company”, and Inc.

United States

Senior

LLM Inference

CUDA

Performance Engineering

+5 more

Job Description

About Zenlayer

Zenlayer is the first massively distributed edge cloud provider, operating 290+ Points of Presence across 70+ countries. With accolades including PTC “Best Networking Innovation”, “Outstanding Cloud, Data Center, or Interconnection Company”, and Inc. 5000, we enable businesses to deploy and run applications at the edge, thus reducing network latency to the end user. Zenlayer focuses on improving connections worldwide, all while nurturing a unique company culture that values service, growth, and trust. As a “Zenplayer”, you’ll be joining a community of success-driven team players that share the dream of powering a better-connected world.

About the Role:

As the technical lead for inference performance, you will own end-to-end responsibility for the platform's Token Performance. For models ranging from hundreds of billions to trillions of parameters (including MoE), you will continuously reduce cost per million tokens and increase per-GPU Goodput while meeting TTFT/TPOT SLOs, and prove the gain from every optimization with reproducible data.

What You Will Work On

1. End-to-End Performance Attribution and Benchmarking

Establish a unified Token Performance metrics framework (TTFT, TPOT, Goodput, GPU utilization, per-token cost), and build load-testing benchmarks that reflect real production traffic (high concurrency, long context, multi-turn Agent workloads)

Perform bottleneck attribution for models such as DeepSeek, Kimi, GLM, Qwen, and MiniMax across compute, memory bandwidth, communication, scheduling, and queuing, and let the data determine which layer is the dominant constraint

2. Kernel-Level Inference Engine Optimization

Dive deep into the source code of vLLM / SGLang / TensorRT-LLM, customizing and contributing upstream on critical paths such as the scheduler, Continuous Batching, Chunked Prefill, CUDA Graph, and KV block management

Clearly define the responsibility boundary between the inference engine and orchestration layers such as Dynamo / llm-d, and reduce tail latency through request routing and load balancing

3. PD Disaggregation and KV Cache Transfer

Lead the core design of Prefill-Decode disaggregation: the KV transfer path (NIXL/RDMA/UCX), transfer-compute overlap, and phase-aware autoscaling

Define the failure domains and recovery semantics of cross-instance KV transfer, and answer the key question: under which workloads does PD disaggregation deliver gains, and under which does it make things slower

4. MoE Parallelism and Wide-EP

Design hybrid TP/PP/DP/EP parallelism strategies and quantify the trade-offs between EP and TP

Tackle expert load imbalance and All-to-All communication overhead, and drive Wide-EP into production

5. Kernels and Quantization

Write and tune critical CUDA (and ROCm/HIP) operators: memory access optimization, kernel fusion, and Attention / MoE / communication-related kernels

Build a closed-loop "quantization – accuracy – throughput" evaluation: the real-world gains and accuracy regressions of FP8/FP4/INT4/AWQ/GPTQ across different engines and hardware

Requirements

First-principles understanding of Transformer inference: the compute vs. memory-bound nature of Prefill and Decode, the KV Cache memory model, the differences among MHA/GQA/MLA, and how MoE routing constrains parallelism

Source-level, in-depth experience with at least one mainstream inference engine, with real cases of identifying and fixing bottlenecks in scheduling, kernel selection, or KV management

CUDA kernel tuning skills (Nsight profiling, memory optimization, fusion, CUDA Graph) with hands-on practical experience

Bachelor’s degree in Computer Science, Electrical/Computer Engineering, or related field (HPC systems, ML Systems).

Minimum 6 years in systems/Performance Engineering, with at least 2-3 years directly on LLM or GPU performance work

Preferred, but not required:

Experience with Merged PRs in vLLM

Experience operating at cluster sale (hundreds to thousands of GPUs)

Requirements

first-principles understanding of transformer inference: the compute vs. memory-bound nature of prefill and decode, the kv cache memory model, the differences among mha/gqa/mla, and how moe routing constrains parallelism
source-level, in-depth experience with at least one mainstream inference engine, with real cases of identifying and fixing bottlenecks in scheduling, kernel selection , or kv management
cuda kernel tuning skills (nsight profiling, memory optimization, fusion, cuda graph) with hands-on practical experience
bachelor’s degree in computer scienc e, el ectrical/computer engineering, or related field (hpc systems, ml systems).
minimum 6 years in systems/performance engineering, with at least 2-3 years directly on llm or gpu performance work
experience with merged prs in vllm
experience operating at cluster sale ( hundre d s to thousands of gpus)

Skills

LLM Inference

CUDA

Performance Engineering

vLLM

SGLang

TensorRT-LLM

Transformer Architecture

KV Cache

Ready to Apply?

Join Zenlayer, Inc today

Salary Range

$300k - $600k

Posted 2 weeks ago

More AI roles you might like

Discover similar opportunities from companies that are also hiring remotely.

Director of Sales - AI Digital Health

Navina

Full Time

1 day ago

Navina is a fast-growing digital health SaaS company that’s on a mission to transform the way physicians interact with patient data. Thousands of clinicians across the United States already use Navina’s AI-powered solution that transforms complex and fragmented patient data into concise “patient…

United States

Machine Learning

Senior

Sales

Consultative Selling

Healthcare Sales

+5 more

Explore more remote openings

Browse fresh listings from our global community of remote-friendly teams.

Senior/Staff Data Scientist

GRIDSIGHT

Full Time

18 hours ago

The Role As a Senior/Staff Data Scientist in the Connections Squad, you will provide the data insights needed to connect new facilities to the grid. You will design, implement and run the calculations needed to model new connections, and perform the statistical analysis needed to assess their…

Australia

Data

Data Science

Senior

Python

Data Science

Machine Learning

+5 more

Escalation Specialist

Flex Fitness

Full Time

$72k - $81.5k

19 hours ago

## Job Title Escalation Specialist ## Company Overview Flex Fitness is a dynamic and growing fitness company dedicated to helping people achieve their health and wellness goals.

United States

Engineering

Backend Engineering

Mid

Escalation Management

Customer Support

Problem Solving

+5 more

Project Manager

ICON plc

Full Time

19 hours ago

Project Manager ICON is a global healthcare intelligence and clinical research organisation united by a mission to bring new medicines and treatments to patients faster.

Japan

Engineering

Backend Engineering

Senior

Project Management

Team Leadership

Resource Management

+5 more

Software Automation & Platform Engineer — Federal Health IT

Rackner

Full Time

20 hours ago

Software Automation & Platform Engineer — Federal Health IT Location: Remote — United States Engineer Smarter Workflows. Deliver More Reliable Federal Health IT Services.

United States

Engineering

DevOps / SRE

Mid

Software Engineering

Platform engineering

ServiceNow

+5 more

Senior Underwriter, Central

Coalition, Inc.

Full Time

$120k - $150k

21 hours ago

About us Coalition is the world's first Active Insurance provider designed to help prevent digital risk before it strikes. Founded in 2017, Coalition combines comprehensive insurance coverage and innovative cybersecurity tools to help businesses manage and mitigate potential cyberattacks.

United States

Engineering

Senior

ServiceNow ITSM Engineer — Federal Health IT

Rackner

Full Time

21 hours ago

ServiceNow ITSM Engineer — Federal Health IT Location: Remote — United States Travel: Occasional travel for site go-lives and exercises Background Investigation: Eligibility for a Tier 3 investigation Build Reliable ITSM Workflows. Support Mission-Critical Federal Healthcare.

United States

Engineering

Backend Engineering

Mid

ServiceNow

ITSM

incident management

+5 more

Hazard Claims Administrator (REMOTE)

Dimont & Associates, LLC

Full Time

21 hours ago

Claims Administrator – Hazard Claims (Remote) Support the Claims Process From Anywhere DIMONT is seeking a Claims Administrator to join our fully remote claims operations team supporting hazard insurance claims for mortgage lenders, servicers, and investors nationwide.

United States

Engineering

Backend Engineering

Entry

Property Preservation

Mortgage Servicing

Data entry

+5 more

Country Lead Ecuador (Remote)

Clara

Full Time

22 hours ago

Clara is the leading platform in Latin America for companies to execute and manage every payment operation. More than 50,000 leading and fast-growing businesses run on our corporate cards, bill pay, financing and B2B payments, and in 2025 the Financial Times named Clara the region’s fastest-growing…

Worldwide

Engineering

Backend Engineering

Senior

B2B Sales

Enterprise Sales

Business Development

+5 more