Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

American Home Design

Installer/Plumber Job at American Home Design

American Home Design has an immediate opening for a Plumbing Installer to join our team. If you have experience installing water softeners, hot water tanks, or general plumbing, we will train you to install our systems in residential homes. ~ Work Part time and receive...

Avamere

Director of Nursing/Registered Nurse Job at Avamere

 ...Directorof Nursing Services (RN) Status: Full-Time Schedule: Monday-Friday Location: Avamere Rehabilitation of Park West - 1703 California Avenue SW - Seattle, WA 98116 Apply at Teamavamere.com Responsible for management of the nursing services department... 

CMU Health

Patient Scheduler - Endocrinology Job at CMU Health

 ...GENERAL STATEMENT OF DUTIES The Endocrinology Scheduler is responsible for supporting the needs of the endocrinology department, receiving and directing phone call encounters, scheduling visits, recording insurance and demographic data, scheduling, and coordinating the... 

Drivo Rent a Car

Airport Operations Manager Job at Drivo Rent a Car

 ...rental industry, we are committed to delivering top-notch service to our customers while fostering a supportive and inclusive work environment...  ...Rent A Car? With 5 locations in New York and New Jersey airport plus offices in Brooklyn and Manhattan Growth plan for new... 

Oliver Group INC.

Independent Contractor/Owner Operator Job at Oliver Group INC.

 ...Job description Calling All Owner-Operators! Join Oliver Group INC today and get direct loads! Earn Top Rates: $0.60 - $3.00/mile...  ...choose your orders through the convenient mobile app No Experience Needed Just bring your reliable vehicle! We Accept: Cargo...