Distributed Training & Inference Optimization Engineer (LLM) - GPU Optimization Department (GPUOD) job opportunity at Rakuten International.

Home Jobs In Japan Tag: Job Distributed Training & Inference Optimization Engineer (LLM) - GPU Optimization Department (GPUOD) position at Rakuten International


DateMore Than 30 Days Ago bot
Rakuten International Distributed Training & Inference Optimization Engineer (LLM) - GPU Optimization Department (GPUOD)
Experience: 3-years
Pattern: full-time
Country:
apply Apply Now
Salary:
Status:

Job

Copy Link Report
degreeHigh School (S.S.C.E)
loacation Tokyo, Japan, Japan
loacation Tokyo, Japan....Japan

Job Description: Business Overview AI & Data Division (AIDD) spearheads data science & AI initiatives by leveraging data from Rakuten Group. We build a platform for large-scale field experimentations using cutting-edge technologies to provide critical insights that enable faster and better and faster contribution for our business. Our division boasts an international culture created by talented employees from around the world. Following the strategic vision “Rakuten as a data-driven membership company”, AIDD is expanding its data & AI related activities across multiple Rakuten Group companies. 　 Department Overview GPU Optimization Department (GPUOD) is responsible for the strategic management, optimization, and governance of Rakuten's company-wide AI infrastructure, ensuring high-performance, cost-efficient utilization of compute resources for machine learning workloads. We oversee a large-scale hybrid infrastructure spanning thousands of accelerators, including the latest Hopper and upcoming Blackwell architectures. As a central enabler for AI innovation, we: - Optimize compute resource allocation across on-premises and multi-cloud environments, maximizing efficiency for training and inference workloads. - Manage hybrid orchestration of diverse accelerator resources, ensuring seamless scalability and cost-effective deployment. - Develop and enhance frameworks for large-scale distributed training, with special focus on LLMs and generative AI. - Optimize inference performance through model optimization techniques and system-level acceleration. - Collaborate with internal teams to deliver scalable, high-availability inference services tailored to business needs. - Continuously evaluate next-generation hardware solutions, including specialized AI chips optimized for LLM workloads. - By effectively managing both conventional and specialized compute resources across on-premises and cloud environments, our team ensures Rakuten's AI ecosystem remains at the forefront of performance, reliability, and cost-efficiency. Position: Why We Hire - Work on cutting-edge LLM training & inference optimization at scale. - Directly impact Rakuten’s AI infrastructure by improving efficiency and reducing costs. - Collaborate with global AI/ML teams on high-impact challenges. - Opportunity to research and implement state-of-the-art GPU optimizations. 　 Position Details As a GPU Training & Inference Optimization Engineer, you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten’s GPU clusters. You will deeply optimize training frameworks (e.g., PyTorch, DeepSpeed, FSDP) and inference engines (e.g., vLLM, TensorRT-LLM, Triton, SGLang), ensuring Rakuten’s AI models run at peak efficiency. This role requires strong expertise in GPU-accelerated ML frameworks, distributed training, and inference optimization, with a focus on reducing training time, improving GPU utilization, and minimizing inference latency. Key Responsibilities - Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time. - Profile and optimize distributed training bottlenecks (e.g., NCCL issues, CUDA kernel efficiency, communication overhead). - Implement and tune inference optimizations (e.g., quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang). - Collaborate with infrastructure teams to improve GPU cluster scheduling, resource allocation, and fault tolerance for large-scale training jobs. - Develop benchmarking tools to measure and improve training throughput, memory efficiency, and inference latency. - Research and apply cutting-edge techniques (e.g., mixture-of-experts, speculative decoding) to optimize LLM performance. 　 Mandatory Qualifications: - 3+ years of hands-on experience in GPU-accelerated ML training & inference optimization, preferably for LLMs or large-scale deep learning models. - Deep expertise in PyTorch, DeepSpeed, FSDP, or Megatron-LM, with experience in distributed training optimizations. - Strong knowledge of LLM inference optimizations (e.g., quantization, pruning, KV caching, continuous batching). - Bachelor’s or higher degree in Computer Science, Engineering, or related field. 　 Desired Qualifications: - Proficiency in CUDA, Triton kernel, NVIDIA tools (Nsight, NCCL), and performance profiling (e.g., PyTorch Profiler, TensorBoard). - Experience with LLM-specific optimizations (e.g., FlashAttention, PagedAttention, LoRA, speculative decoding). - Familiarity with Kubernetes (K8s) for GPU workloads (e.g., KubeFlow, Volcano). - Contributions to open-source ML frameworks (e.g., PyTorch, DeepSpeed, vLLM). - Experience with inference serving frameworks (e.g., vLLM, TensorRT-LLM, Triton, Hugging Face TGI). #engineer #applicationsengineer #AI #aianddatadiv  Languages: English (Overall - 3 - Advanced)

Other Ai Matches

Software Engineer - Measurement & Insight Section, Analytics Data Engineering Department (ADED) Applicants are expected to have a solid experience in handling Analytics Data Engineering Department (ADED) related tasks
Service Planning, Partner Sales 1 Section -Partner Sales Department(Group Sales Div) Applicants are expected to have a solid experience in handling Partner Sales 1 Section -Partner Sales Department(Group Sales Div) related tasks
Producer/Project Manager - System Division, Rakuten Bank, Ltd. Applicants are expected to have a solid experience in handling Rakuten Bank, Ltd. related tasks
Finance Processing Specialist – IT Financial Management Section, IT Planning Department(ITPD) Applicants are expected to have a solid experience in handling IT Planning Department(ITPD) related tasks
Project Manager (Quality Assurance Team Management) - Marketing Division Applicants are expected to have a solid experience in handling Job related tasks
Procurement Operations/Help Site Management - Procurement Operations Department, Procurement Operations Supervisory Department Applicants are expected to have a solid experience in handling Procurement Operations Supervisory Department related tasks
Sales Management Group Member, Solution Sales 1 Section -Enterprise Solution Business Department (Group Sales Div) Applicants are expected to have a solid experience in handling Solution Sales 1 Section -Enterprise Solution Business Department (Group Sales Div) related tasks
Security & Communication Services Section (Member or Manager) - Network Business Department Applicants are expected to have a solid experience in handling Job related tasks
PR staff, Corporate - Corporate Communications Department(PRD) Applicants are expected to have a solid experience in handling Corporate - Corporate Communications Department(PRD) related tasks
Senior Software Engineer (Engineer Lead), Ad Sales Support & Sales Management Service - Global Ad Technology Supervisory Department (GATD) Applicants are expected to have a solid experience in handling Ad Sales Support & Sales Management Service - Global Ad Technology Supervisory Department (GATD) related tasks
AI System Design/Architect Engineer - Business IT Platform Section, Corporate IT Service Department (CITSD) Applicants are expected to have a solid experience in handling Corporate IT Service Department (CITSD) related tasks
Business strategy and Service Planning (Leader) - Enterprise Solution Business Department, Group Sales Division (GSD) Applicants are expected to have a solid experience in handling Group Sales Division (GSD) related tasks
Sales, Solution Sales 3 Section -Enterprise Solution Business Department (Group Sales Div) Applicants are expected to have a solid experience in handling Solution Sales 3 Section -Enterprise Solution Business Department (Group Sales Div) related tasks
Data Science Engineer - Search Department (SED) Applicants are expected to have a solid experience in handling Job related tasks
Product Development and Service Planning (Rakuten All Services Integrated App) - Marketing Division Applicants are expected to have a solid experience in handling Job related tasks
Corporate Planning / Management - President Office, Commerce & Marketing Company(C&M) Applicants are expected to have a solid experience in handling Commerce & Marketing Company(C&M) related tasks
Application Development Engineer - Company CIO Office (CIOO) Applicants are expected to have a solid experience in handling Job related tasks
Marketing Specialist (Customer Engagement), Rakuma - Rakuma Business Department (C&M) Applicants are expected to have a solid experience in handling Rakuma - Rakuma Business Department (C&M) related tasks
Technical Project Manager - Conversational Experiences Department (CXD) Applicants are expected to have a solid experience in handling Job related tasks
System Planning, Corporate Business Operation 1 Section - Corporate Business Operation Department( Group Sales Div) Applicants are expected to have a solid experience in handling Corporate Business Operation 1 Section - Corporate Business Operation Department( Group Sales Div) related tasks
Cloud Engineer - Rakuten Wallet, Inc. Applicants are expected to have a solid experience in handling Inc. related tasks
Service planning & development (Leader Candidate):Solution Service Development 2 Section,Enterprise Solution Business Department(Group Sales Division) Applicants are expected to have a solid experience in handling Enterprise Solution Business Department(Group Sales Division) related tasks
Marketing Research Director - Research Direction Department, Rakuten Insight, Inc. (Hachinohe)(A&M) Applicants are expected to have a solid experience in handling Rakuten Insight, Inc. (Hachinohe)(A&M) related tasks