Project Maintainer – DPAI Arena Evaluation Infrastructure job opportunity at JetBrains.

Home Jobs In Netherlands Tag: IDE Project Maintainer – DPAI Arena Evaluation Infrastructure position at JetBrains


bot
JetBrains Project Maintainer – DPAI Arena Evaluation Infrastructure
Experience: General
Pattern: Remote
Country:
apply Apply Now
Salary:
Status:

IDE

Copy Link Report
degreeGeneral
loacation Amsterdam, ; Belgrade, Serbia; Berlin, Germany; Limassol, Cyprus; Munich, Germany; Paphos, Cyprus; Prague, Czech Republic; Remote, Germany; Warsaw, Poland; Yerevan, Armenia, Netherlands
loacation Amsterdam, ; B..........Netherlands

At JetBrains, code is our passion. Ever since we started, we have strived to make the strongest, most effective developer tools on earth. By automating routine checks and corrections, our tools speed up production, freeing developers to grow, discover, and create. We believe in building tools developers love – and we see the next frontier in AI-powered developer tools. The DPAI Arena project aims to define and maintain an open, community-driven benchmark for evaluating AI agents and IDE-embedded AI features at scale. We believe that the benchmark industry is only in its early stages, and we’re planning to take part in the evolution of the field.  We’re looking for a Project Maintainer to lead the technical backbone of this initiative – managing the evaluation pipeline, integrating agents, and enabling contributions from the broader community. As part of our team, you will: Design, build, and maintain the evaluation infrastructure pipeline with the Eval Infrastructure team, ensuring it supports the goals of DPAI Arena, including scalability, reproducibility, and extensibility. Integrate new agents, models, tools, and runtime environments into the pipeline, ensuring compatibility, stability, and maintainability across various configurations. Define, document, and maintain contribution guidelines and processes, making it easy and safe for internal teams and external contributors to add new tasks, agents, and evaluation setups. Curate the technical process of contributions by reviewing submissions, validating conformity with standards, coordinating merges, and ensuring the consistency, reproducibility, and quality of evaluation artifacts. Support the growth of a community-driven ecosystem by enabling contributions, maintaining clear documentation and onboarding flows, engaging with contributors, and ensuring long-term sustainability of the infrastructure. We’ll be happy to bring you on board if you have: Strong experience in building and maintaining evaluation or CI/CD-type infrastructure, pipelines, or related tooling. Comfort working with multiple models, runtime environments, tooling setups. The ability to create flexible, modular, maintainable infrastructure. An understanding of GenAI in the DevTooling domain Proficiency in writing and maintaining clear developer documentation, contribution guidelines, and onboarding processes. Good coding and system design skills, including the ability to work with codebases, integrate new components, and manage versioning and configurations. Meticulous attention to detail and a quality assurance mindset. The ability to coordinate with different teams (product, ML, external contributors) and ensure smooth collaboration. A self-motivated and proactive mindset in taking ownership of the long-term technical health of the project. We’ll be especially thrilled if you have: Experience with AI-agent integration, multi-model pipelines, and benchmarking frameworks. Familiarity with open-source and community-driven development workflows, contribution processes, and code review practices. #LI-KP1We process the data provided in your job application in accordance with the Recruitment Privacy Policy.

Other Ai Matches

remote-jobserver Remote
Research Engineer (Agentic Models) Applicants are expected to have a solid experience in handling JCP Core Machine Learning related tasks
remote-jobserver Remote
Data Analyst (JetBrains AI) Applicants are expected to have a solid experience in handling JCP Core Analytics related tasks
remote-jobserver Remote
Infrastructure Security Engineer Applicants are expected to have a solid experience in handling Infrastructure Security related tasks
remote-jobserver Remote
FullStack ML Developer Applicants are expected to have a solid experience in handling Junie related tasks
Machine Learning Evaluation Engineer (Agentic Mobile App Generator) Applicants are expected to have a solid experience in handling Ignite related tasks
Software Developer (IntelliJ Platform – Version Control Experience) Applicants are expected to have a solid experience in handling Version Control Experience related tasks
remote-jobserver Remote
Senior Software Developer - Kotlin Native Applicants are expected to have a solid experience in handling Kotlin Native related tasks
HR Specialist Applicants are expected to have a solid experience in handling Human Resources Netherlands related tasks
Head of Corporate Security Applicants are expected to have a solid experience in handling Security related tasks
Project Manager (Software Engineering Research) Applicants are expected to have a solid experience in handling Software Engineering Research related tasks
Finance Systems Analyst and Support Applicants are expected to have a solid experience in handling Financial Support and Analysis related tasks
Project Lead (Amper) Applicants are expected to have a solid experience in handling Amper related tasks
remote-jobserver Remote
Senior QA engineer (BAD) Applicants are expected to have a solid experience in handling Business Applications Quality Assurance related tasks
SDET Engineer in Kotlin Performance QA team Applicants are expected to have a solid experience in handling Kotlin Performance QA related tasks
remote-jobserver Remote
Staff Software Developer (Kotlin Libraries) Applicants are expected to have a solid experience in handling Kotlin Libraries related tasks
Payroll Specialist (US) Applicants are expected to have a solid experience in handling Global Payroll related tasks
remote-jobserver Remote
Support Engineer (Business Applications Development) Applicants are expected to have a solid experience in handling Business Applications Technical Support related tasks
Principal Compliance Analyst Applicants are expected to have a solid experience in handling Compliance related tasks
HR Generalist Applicants are expected to have a solid experience in handling Human Resources Czech Republic related tasks
Head of Product, Partner, and Customer PR and Communications Applicants are expected to have a solid experience in handling Public Relations (PR) and Communications related tasks
Kotlin Developer Advocate Applicants are expected to have a solid experience in handling [disbanded] Kotlin Advocacy related tasks
Software Developer (Platform/ Remote Development) Applicants are expected to have a solid experience in handling Remote Development Technology related tasks
remote-jobserver Remote
JetBrains Web Developer Advocate (Developer Advocacy) Applicants are expected to have a solid experience in handling Web and Data Advocacy related tasks