Principal Data Scientist, Gemini Core Modeling and Evaluations
- linkCopy link
- emailEmail a friend
Minimum qualifications:
- Bachelor's degree in Machine Learning, Data Science, Information Management, Computer Science, or other Engineering discipline, or equivalent practical experience.
- 15 years of experience in data science or other quantitative fields such as: statistics, machine learning, software engineering, or ML engineering.
- Experience working on algorithmic products, production machine learning systems, and defining model evaluation metrics.
- Experience in using data insights to drive product excellence.
Preferred qualifications:
- Experience developing high-growth technology products and services.
- Ability to influence without direct authority, working on and with technical and cross-functional or highly matrixed teams who do not report into this role.
- Ability to turn ambiguous problem spaces into clear solutions, and comfortable with new technology and thinking outside the box to develop and implement short- and long-term creative solutions.
- Ability to work well cross-functionally, and manage organizational complexities and competing priorities.
- Excellent project/product management and technical architecture skills, and the ability to deliver projects on time.
- Excellent strategic thinking, data science, and analytical skills.
About the job
As a Principal Data Scientist, you will be a deeply technical and strategic data science leader for Gemini Core Modeling and Evaluations. You (along with a small team of data scientists) will be responsible for defining the model evaluation criteria, influencing the supervised fine tuning data and evaluation prompt sets, identifying valuable signals for reinforcement learning human feedback, and improving human rater efficiencies.
Responsibilities
- Develop deep product and user analytics that will help to drive quality and design directions, such as identifying signals that are predictive of user engagement or analyzing Gemini use cases (e.g., coding, fact finding, or creativity and summarization). Develop insights around trends and drive model improvements that help achieve goals.
- Drive forward the science of measuring Gemini, ensuring that our quality, factuality, and safety model evaluation metrics are indicative of the end-user experience, and help guide launch decisions for Gemini.
- Manage experimentation and comparative evaluation, including the science, methodologies, tools, and measurement approaches to accelerate Gemini experimentation and innovation.
- Guide the model's human evaluation process, including analyzing rater performance, influencing rater templates, updating training, and evolving our approach to ensure precise and accurate evaluations.
Information collected and processed as part of your Google Careers profile, and any job applications you choose to submit is subject to Google's Applicant and Candidate Privacy Policy.
Google is proud to be an equal opportunity and affirmative action employer. We are committed to building a workforce that is representative of the users we serve, creating a culture of belonging, and providing an equal employment opportunity regardless of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition (including breastfeeding), expecting or parents-to-be, criminal histories consistent with legal requirements, or any other basis protected by law. See also Google's EEO Policy, Know your rights: workplace discrimination is illegal, Belonging at Google, and How we hire.
If you have a need that requires accommodation, please let us know by completing our Accommodations for Applicants form.
Google is a global company and, in order to facilitate efficient collaboration and communication globally, English proficiency is a requirement for all roles unless stated otherwise in the job posting.
To all recruitment agencies: Google does not accept agency resumes. Please do not forward resumes to our jobs alias, Google employees, or any other organization location. Google is not responsible for any fees related to unsolicited resumes.