Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

BRYN MAWR MEDICAL SPECIALISTS ASSOCIATION

Certified Registered Nurse Anesthetist Job at BRYN MAWR MEDICAL SPECIALISTS ASSOCIATION

 ...to patients undergoing endoscopic procedures in our ambulatory surgery center. This position offers the opportunity to work in a physician owned medical practice dedicated to delivering high-quality patient care. Responsibilities Perform pre-anesthetic assessments... 

Russell Tobin

Trade Operations Soecialist Job at Russell Tobin

 ...Description: The role is responsible for supporting the full post-trade lifecycle, ensuring accurate processing, settlement, and...  ...counterparties to ensure operational efficiency. Qualifications & Skills: Experience in trade operations, middle office, or investment... 

Pocketbook Agency

Office Manager for Retail Company Job at Pocketbook Agency

 ...friendly Office Manager for a successful and fast growing e-commerce company based in Hollywood, Los Angeles. You will be the first point of...  ...visibility for key team Partner with People & Culture on new hire prep tasks, including paperwork and building/parking card setup,... 

Silver Line Recruiting LLC

Construction Superintendent 100% Travel Job at Silver Line Recruiting LLC

 ...Job Description The Construction Project Superintendent is responsible for the first-line supervision of a work crew (usually four crews with 1...  ...company will provide vehicle transportation and fuel for travel to and from projects. Work hours start and end at the location... 

Eurofins USA BioPharma Services

Entry Level Senior Administrator (Archive/Document Control) Job at Eurofins USA BioPharma Services

 ...and laboratory services for genomics, discovery pharmacology, forensics, advanced material sciences and in the support of clinical studies...  ...: High school diploma required Strong computer skills Authorization to work in the United States indefinitely...