Engineering Manager, Machine Learning, Model Evaluations and Data Curation (AI Foundations) position at Netflix Inc.| United States Of America

Netflix Inc Engineering Manager, Machine Learning, Model Evaluations and Data Curation (AI Foundations)

Experience: General

Pattern: Remote

Country:

Copy Link Report

Bachelor's (B.Sc.)

United States ..........United States Of America

Netflix is one of the world's leading entertainment services, with over 300 million paid memberships in over 190 countries enjoying TV series, films and games across a wide variety of genres and languages. Members can play, pause and resume watching as much as they want, anytime, anywhere, and can change their plans at any time.The OpportunityFast-paced innovation in large language models (LLMs) and generative AI is reshaping personalization and discovery at Netflix. Our current bet is that an in-house LLM, customized on Netflix data, will be the cornerstone of this transformation. Model Evaluations and Data Curation (“Evals & Data”) are central to the development of LLMs and other high leverage foundation models at Netflix. Our Evals & Data team builds the benchmarks, evaluators, and baselines that guide LLM progress, and the data infrastructure that delivers high-quality, reproducible training and evaluation datasets to our AI/ML researchers. Together, these capabilities create the flywheel of continuous improvement between data, evals and modeling that drives innovation in LLMs that will power personalization and discovery in the future. We are incubating a centralized, first-class evaluation discipline, creating shared language, tools, and standards that enable application teams to measure progress consistently and with confidence, with an emphasis on Netflix’s LLM customized for personalization.We are looking for a seasoned engineering leader to build and scale this team. You will lead a world-class group of AI/ML researchers and engineers working at the intersection of evaluation science, data systems, and foundation models. Your team’s work will shape the trajectory of Netflix’s foundation model development by driving the continuous improvement loop between data, evaluation, and modeling.Questions Your Team Will Help AnswerThe Model Evaluations and Data Curation team exists to give Netflix clarity and confidence in the foundation models that power personalization and discovery. Some of the strategic questions your team will help answer include:Are our LLMs and other foundation models improving in ways that meaningfully benefit members?How do we rigorously measure LLM capabilities across personalization, content understanding, and text generation?What are the strongest baselines we can build with off-the-shelf SOTA models?How do data composition, diversity, and quality influence model performance?Which evaluation signals most reliably predict real-world product outcomes?How can we iterate quickly on data and evaluations while ensuring reproducibility and trust?How can we evaluate agentic systems for recommendations and content understanding?In this role, you will:Partner with downstream AI application teams to define shared evaluations that codify application expectations of LLMs and other foundation models, ensuring progress can be transparently tracked against real-world needs.Design rigorous benchmarks and evaluation methodologies across ranking & recommendations, content understanding, and language/text generation — grounded in a deep technical understanding of LLMs, their strengths, limitations, and failure modes.Lead the development of evaluators and strong baselines to ensure in-house LLMs and other foundation models demonstrate clear advantages over off-the-shelf alternatives.Build scalable, reproducible data and evaluation systems that make dataset creation and evaluation design as nimble and experiment-friendly as model development itself.Hire, grow, and nurture a world-class team, fostering an inclusive, high-performing culture that balances research innovation with engineering excellence.Work closely with the teams developing Netflix’s foundation models (including our core LLM) to ensure evaluation and data insights are folded back into the cadence of model development. Proactively influence the ML Platform and Data Engineering teams at key interfaces.Minimum QualificationsExperience building and leading high-performing teams of ML researchers and engineers.Proven track record of leading machine learning initiatives from research to production, ideally involving evaluation frameworks, ML infrastructure, or data-intensive systems.Strong technical expertise in LLMs, their evaluation, and practical methods for ensuring robustness, reproducibility, and quality.Broad knowledge of machine learning fundamentals and evaluation methodologies, including benchmark design, model-based evaluators, and offline/online metrics.Experience driving cross-functional projects, including close collaboration with AI application teams to translate product needs into evaluation frameworks.Excellent written and verbal communication skills, able to bridge technical and non-technical audiences.Advanced degree in Computer Science, Statistics, or a related quantitative field.Preferred Qualifications8+ years of overall experience, including 3+ years in engineering management.Experience with large-scale ML systems and foundation models, especially LLMs.Background in building evaluation frameworks, model benchmarking, or data infrastructure for LLM training.Familiarity with multi-modal data and evaluation.Culture & BenefitsNetflix's culture is an integral part of our success, and we approach diversity and inclusion seriously and thoughtfully. We are an equal opportunity employer and celebrate diversity, recognizing that bringing together different perspectives and backgrounds helps build stronger teams. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top-of-market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is$190,000 - $920,000.Netflix provides comprehensive benefits including Health Plans, Mental Health support, a 401(k) Retirement Plan with employer match, Stock Option Program, Disability Programs, Health Savings and Flexible Spending Accounts, Family-forming benefits, and Life and Serious Injury Benefits. We also offer paid leave of absence programs. Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off. Full-time salaried employees are immediately entitled to flexible time off. See more details about our Benefitshere.Netflix has a unique culture and environment. Learn morehere.Inclusionis a Netflix value and we strive to host a meaningful interview experience for all candidates. If you want anaccommodation/adjustmentfor a disability or any other reason during the hiring process, please send a request to your recruiting partner.We are an equal-opportunity employer and celebrate diversity, recognizing that diversitybuilds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.Job is open for no less than 7 days and will be removed when the position is filled.

Other Ai Matches

Partner Marketing Learning Program Manager Applicants are expected to have a solid experience in handling Marketing related tasks

Live Technical Launch Manager - Japan/APAC Applicants are expected to have a solid experience in handling Engineering Operations related tasks

Specialist, Communications - Entertainment Applicants are expected to have a solid experience in handling Communications & Corporate Affairs related tasks

Consumer Products Designer - APAC Applicants are expected to have a solid experience in handling Consumer Products & Experiences related tasks

Surfacing Artist Applicants are expected to have a solid experience in handling Surfacing Artist related tasks

Remote

Technical Program Manager (L5), Security M&A Applicants are expected to have a solid experience in handling Engineering Operations related tasks

Manager, Content Financial Planning & Analysis (FP&A) - Korea Applicants are expected to have a solid experience in handling management related tasks

Remote

Distributed Systems Engineer (L5), Content & Business Products Applicants are expected to have a solid experience in handling Engineering related tasks

Netflix House Digital Marketing Strategist Applicants are expected to have a solid experience in handling Marketing related tasks

Manager, Brand & Slate Marketing - Korea Applicants are expected to have a solid experience in handling Marketing related tasks

Coordinator, Marketing Production - SEA+ Applicants are expected to have a solid experience in handling Marketing related tasks

Investigations & Response Project Manager - Games Moderation Applicants are expected to have a solid experience in handling Customer Service related tasks

Software Engineer 5 - Device Reliability Applicants are expected to have a solid experience in handling Engineering related tasks

Senior Software Engineer — Commerce Experiences Testing Infrastructure Applicants are expected to have a solid experience in handling Engineering related tasks

Remote

Senior Motion Designer, Emerging Creative Applicants are expected to have a solid experience in handling Product Design related tasks

Remote

Manager, Machine Learning - Media Planning, DSE Applicants are expected to have a solid experience in handling Data & Insights related tasks

Software Engineer L4/L5 Training Platform, Machine Learning Platform Applicants are expected to have a solid experience in handling Data & Insights related tasks

Remote

Ads Privacy Engineer (L6) Applicants are expected to have a solid experience in handling Engineering related tasks

Engineering Manager, Data Platform Applicants are expected to have a solid experience in handling Engineering related tasks

Remote

Staff Product Designer, Cloud Infrastructure Applicants are expected to have a solid experience in handling Product Design related tasks

Recruiting Researcher - Korea Applicants are expected to have a solid experience in handling Talent related tasks

Software Engineer 5 - Data Platform Poland Applicants are expected to have a solid experience in handling Engineering related tasks

Solutions Software Engineer L4 - Studio and Enterprise Solutions Applicants are expected to have a solid experience in handling Engineering related tasks

Engineering Manager, Machine Learning, Model Evaluations and Data Curation (AI Foundations) job opportunity at Netflix Inc.

Saved Jobs

No Job Saved

Other Ai Matches