Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Russell Tobin

French Translator Job at Russell Tobin

 ...______________ Basic Qualifications High School Diploma or GED. Native speaker in one of the following languages: Canadian French, French, Dutch, Hindi, Japanese, Arabic. ________________________________________ Preferred Qualifications Prior transcription... 

Atlantic Health

Nursing Educator (RN), Full Time, Evening/Night Shift Flexible Hours, 3p - 3a, Chilton Medical Center Job at Atlantic Health

 ...written Patient, compassionate nature Ability to operate as part of a team, alongside physicians and nurses Proficiency in...  ...facilities, will receive the highest quality care delivered at the right time, at the right place, and at the right cost. This commitment is... 

gTANGIBLE Corporation

Activity Security Representative III Job at gTANGIBLE Corporation

 ...About the Company gTANGIBLE Corporation (gTC), , is a S corporation and a registered Government contractor that provides services and solutions in: National Security Programs Professional, Administrative, and Management Support Mission and Warfighter Support... 

Quatrro BSS

Chief Financial Officer Job at Quatrro BSS

 ...Quatrro Business Support Services is seeking a Chief Financial Officer to join our nonprofit team in a hybrid role based in Detroit, MI . This position offers a unique opportunity to lead end-to-end accounting processes for multiple nonprofit clients, with a strong... 

Equity Medical

Internal Medicine Physician (Clinical Trials Principal Investigator Role) Job at Equity Medical

 ...Medical, LLC (EM") is a multi-state, multi-therapeutic area clinical trial research company specializing in the dermatology, allergy/immunology/respiratory, and Ph1 areas. Founded by board-certified dermatologists Dr Michael Cameron and Dr James Allred, EM has dedicated...