Machine Learning Intern - Dynamic KV-Cache Modeling for Efficient LLM Inference job opportunity at d-Matrix.

Home Jobs In United States Of America Tag: R&D - SW Kernels & Workloads Machine Learning Intern - Dynamic KV-Cache Modeling for Efficient LLM Inference position at d-Matrix


bot
d-Matrix Machine Learning Intern - Dynamic KV-Cache Modeling for Efficient LLM Inference
Experience: General
Pattern: Intern
Country:
apply Apply Now
Salary:
Status:

R&D - SW Kernels & Workloads

Copy Link Report
degreeBachelor's (B.Sc.)
loacation Santa Clara, United States Of America
loacation Santa Clara....United States Of America

At d-Matrix , we are focused on unleashing the potential of generative AI to power the transformation of technology. We are at the forefront of software and hardware innovation, pushing the boundaries of what is possible. Our culture is one of respect and collaboration. We value humility and believe in direct communication. Our team is inclusive , and our differing perspectives allow for better solutions. We are seeking individuals passionate about tackling challenges and are driven by execution.  Ready to come find your playground? Together , we can help shape the endless possibilities of AI. We are seeking a motivated and innovative Machine Learning Intern to join our team. The intern will work on developing a dynamic Key-Value (KV) cache solution for Large Language Model (LLM) inference, aimed at enhancing memory utilization and execution efficiency on D-Matrix hardware. This project will involve modeling at the PyTorch graph level to enable efficient, torch-native support for KV-Cache, addressing limitations in current solutions. Responsibilities: • Research and analyze existing KV-Cache implementations used in LLM inference, particularly those utilizing lists of past-key-values PyTorch tensors. • Investigate “Paged Attention” mechanisms that leverage dedicated CUDA data structures to optimize memory for variable sequence lengths. • Design and implement a torch-native dynamic KV-Cache model that can be integrated seamlessly within PyTorch. • Model KV-Cache behavior within the PyTorch compute graph to improve compatibility with torch.compile and facilitate the export of the compute graph. • Conduct experiments to evaluate memory utilization and inference efficiency on D-Matrix hardware. Key Objectives: • Develop an efficient support system for KV-Cache on D-Matrix hardware. • Create a torch-level modeling framework for dynamic KV-Cache. • Ensure compatibility of the KV-Cache model with torch.compile and other PyTorch features for optimized graph export. Requirements: • Currently pursuing a degree in Computer Science, Electrical Engineering, Machine Learning, or a related field. • Familiarity with PyTorch and deep learning concepts, particularly regarding model optimization and memory management. • Understanding of CUDA programming and hardware-accelerated computation (experience with CUDA is a plus). • Strong programming skills in Python, with experience in PyTorch. • Analytical mindset with the ability to approach problems creatively. Preferred Qualifications: • Experience with deep learning model inference optimization. • Knowledge of data structures used in machine learning for memory and compute efficiency. • Experience with hardware-specific optimization, especially on custom hardware like D-Matrix, is an advantage. This role is ideal for a self-motivated intern interested in applying advanced memory management techniques in the context of large-scale machine learning inference. If you’re passionate about optimizing machine learning models and are excited to explore cutting-edge solutions in model inference, we encourage you to apply. Equal Opportunity Employment Policy d-Matrix is proud to be an equal opportunity workplace and affirmative action employer. We’re committed to fostering an inclusive environment where everyone feels welcomed and empowered to do their best work. We hire the best talent for our teams, regardless of race, religion, color, age, disability, sex, gender identity, sexual orientation, ancestry, genetic information, marital status, national origin, political affiliation, or veteran status. Our focus is on hiring teammates with humble expertise, kindness, dedication and a willingness to embrace challenges and learn together every day. d-Matrix does not accept resumes or candidate submissions from external agencies. We appreciate the interest and effort of recruitment firms, but we kindly request that individual interested in opportunities with d-Matrix apply directly through our official channels. This approach allows us to streamline our hiring processes and maintain a consistent and fair evaluation of al applicants. Thank you for your understanding and cooperation.

Other Ai Matches

Machine Learning Research Intern Applicants are expected to have a solid experience in handling R&D - SW Machine Learning related tasks
Design Verification Engineer, Senior Staff Applicants are expected to have a solid experience in handling R&D - HW Verification related tasks
FPGA Design Engineer, Senior Staff Applicants are expected to have a solid experience in handling R&D - HW Silicon Design related tasks
AI Security Architect, Principal Applicants are expected to have a solid experience in handling R&D - CTO & Architecture related tasks
Analog Design Engineer, Senior Staff Applicants are expected to have a solid experience in handling R&D - HW Analog Circuits related tasks
Technical Recruiter, Senior Staff Applicants are expected to have a solid experience in handling G&A - Recruiting related tasks
Machine Learning Intern - Dynamic KV-Cache Modeling for Efficient LLM Inference Applicants are expected to have a solid experience in handling R&D - SW Kernels & Workloads related tasks
Micro-Architect / RTL Design - CPU, Principal Applicants are expected to have a solid experience in handling R&D - HW Silicon Design related tasks
Design Validation Engineering Intern Applicants are expected to have a solid experience in handling R&D - HW Validation related tasks
Application Performance Engineer Applicants are expected to have a solid experience in handling S&M - Product related tasks
Analog Design Engineer, Staff Applicants are expected to have a solid experience in handling R&D - HW Analog Circuits related tasks
Analog IC Design Intern Applicants are expected to have a solid experience in handling R&D - HW Analog Circuits related tasks
AI / ML System Software Engineer, Senior Staff Applicants are expected to have a solid experience in handling R&D - SW Kernels & Workloads related tasks
AI Software Applications Engineer Applicants are expected to have a solid experience in handling S&M - Product related tasks
Director, Partner and Supplier Management – AI Hardware Applicants are expected to have a solid experience in handling R&D - Operations & Supply Chain related tasks
HW Engineering Intern - PCB Layout Automation & AI Tooling Applicants are expected to have a solid experience in handling R&D - HW Systems Engineering related tasks
HW Systems Engineer, Customer Platforms - Tech Lead/Principal Applicants are expected to have a solid experience in handling S&M - Product related tasks
Design Verification Engineer, Staff Applicants are expected to have a solid experience in handling R&D - HW Verification related tasks
Digital Design Engineer, Micro-Architect, Principal Applicants are expected to have a solid experience in handling R&D - HW Silicon Design related tasks
Compiler Architect Applicants are expected to have a solid experience in handling R&D - CTO & Architecture related tasks