Wealth - Lead - GenAI Testing and Evaluation Framework - Vice President job opportunity at Citibank.

Home Jobs In United States Of America Tag: Lead Wealth - Lead - GenAI Testing and Evaluation Framework - Vice President position at Citibank


bot
Citibank Wealth - Lead - GenAI Testing and Evaluation Framework - Vice President
Experience: 1-year
Pattern: full-time
Country:
apply Apply Now
Salary:
Status:

Lead

Copy Link Report
degreeTechnical Certificate
loacation New York, United States Of America
loacation New York....United States Of America

We are seeking an #innovative and #detail-oriented professional to lead the development and management of the Generative AI (GenAI) testing and evaluation framework. This role focuses on creating patterns, methodologies, and iterative structures to optimize the performance and effectiveness of GenAI models, with a particular emphasis on prompt engineering and evaluation. The ideal candidate will have a strong background in GenAI, a deep understanding of natural language processing, and a passion for refining AI solutions through rigorous testing and iteration. __ Framework Development: Design and implement a comprehensive testing and evaluation framework for GenAI model outputs. Develop standards and patterns for assessing the quality and "goodness" of prompts across diverse use cases. Create iterative processes for testing and refining prompts to optimize model outputs. __ Prompt Engineering and Evaluation: Establish criteria for evaluating prompt performance, including accuracy, completeness, relevance, coherence, and alignment with desired outcomes. Experiment with prompt structures to identify optimal configurations for various business applications. Develop and document best practices for prompt design and refinement. __ Collaboration and Integration: Work closely with tech partners, engineers, and product teams to ensure testing frameworks integrate seamlessly into the development lifecycle. Partner with stakeholders to understand business requirements and tailor testing methodologies to address specific needs. Provide actionable insights and recommendations to improve model performance based on evaluation results. __ Tooling and Automation: Identify and implement tools for automating the testing and evaluation process. Develop dashboards and reporting mechanisms to monitor prompt and model performance metrics. Stay updated on emerging tools and techniques in AI testing and integrate them into the framework. __ Continuous Improvement: Establish feedback loops to iteratively improve testing methodologies and evaluation standards. Establish process for ongoing monitoring of prompts, once productionalized. Monitor industry trends and advancements in Generative AI to ensure the framework remains cutting-edge. Advocate for a culture of experimentation and continuous learning within the organization.

Other Ai Matches

Assistant Vice President, FX Settlement Apply Applicants are expected to have a solid experience in handling Operations related tasks
Head Universal Banker, Inglewood Branch Applicants are expected to have a solid experience in handling Universal Banker related tasks
Asset Servicing Intmd Analyst Applicants are expected to have a solid experience in handling Analyst related tasks
Loan Documentation Team Coordinator Applicants are expected to have a solid experience in handling Team Coordinator related tasks
Senior Executive Assistant Applicants are expected to have a solid experience in handling Executive Assistant related tasks
Vice President Associate Banker SAFE Act Applicants are expected to have a solid experience in handling Associate Banker related tasks
Banking – Corporate Baking, Internship, Bogotá – Colombia, 2026 Applicants are expected to have a solid experience in handling Corperate Banking related tasks
Part-Time Universal Banker (20 Hours), Bilingual English/Spanish Speaking Preferred, Collins Ave Branch Applicants are expected to have a solid experience in handling Universal Banker related tasks
Wealth - Account Opening Product Owner - VP Applicants are expected to have a solid experience in handling Accounting related tasks
Vice President - Audit Manager I Applicants are expected to have a solid experience in handling Auditor related tasks
Financial Institutions - Credit Risk Senior Analyst Applicants are expected to have a solid experience in handling Senior Analyst related tasks
Associate Banker - China Team Applicants are expected to have a solid experience in handling Private Banking related tasks
Personal Banker SAFE Act, Hawthorne Branch Applicants are expected to have a solid experience in handling Personal Banker related tasks
Syndicated Loan Operations Intermediate Associate Analyst Applicants are expected to have a solid experience in handling Analyst related tasks
Associate Banker - Hong Kong Team Applicants are expected to have a solid experience in handling Private Banking related tasks
Wealth Operations Change Manager Applicants are expected to have a solid experience in handling Manager related tasks
SUPER DAY KYC Operations Analyst Applicants are expected to have a solid experience in handling Analyst related tasks
Wealth Relationship Manager - Temple City, CA Applicants are expected to have a solid experience in handling Manager related tasks
IT Business Senior Analyst Applicants are expected to have a solid experience in handling Senior Analyst related tasks
Loan Documentation and Processing Manager AVP Applicants are expected to have a solid experience in handling Processing Manager related tasks
Sr Java Developer - Spring Kafka - Assistant Vice president Applicants are expected to have a solid experience in handling Developer related tasks
Assistant Vice President, FX Specialist (Treasury), Consumer Banking Applicants are expected to have a solid experience in handling Management related tasks
Wealth - Lead - GenAI Testing and Evaluation Framework - Vice President Applicants are expected to have a solid experience in handling Lead related tasks