Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Hobson Prior

Principal Investigator Job at Hobson Prior

 ...Hobson Prior is seeking a Principal Investigator to ensure the safety of participants in clinical trials and oversee the study process according to approved guidelines. You will play a crucial role in advancing medical research by ensuring studies are conducted safely... 

Ken Fulk Inc

Graphic Designer Job at Ken Fulk Inc

 ...Ken Fulk Inc. is seeking a Graphic Designer with 8+ years of experience to join our San Francisco studio. Requirements At least 8 years of experience working in graphic design, preferably in an agency, hospitality or fashion capacity Portfolio that includes a range... 

ACS Air Conditioning Specialist Inc

Lead Installer (RNC) Job at ACS Air Conditioning Specialist Inc

 ...HVAC Lead Installer Minimum 5 years of ductwork experience required. Looking for an experienced duct installer to install ductwork in residential new construction homes for a fast-growing HVAC company! We are one of many branches of ACS looking for someone to join... 

Inter-Con Security

Security Trainer Job at Inter-Con Security

 ...Opportunity Employer - Disability/Veteran. Job Summary As a Trainer, you will be involved in curriculum development and training...  ..., Web-based, etc.) and types (technical, professional, team, safety, etc.). Must be a certified instructor for CPR, AED and Adult... 

PreGel America

Pastry Chef Job at PreGel America

The Pastry Chef is responsible for maintaining the daily functions of a training center kitchen at a high level of quality. This position requires a thorough knowledge of pastry ingredients and their functions in baking, pastry, and frozen dessert applications. This person...