Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

GreatNeckTools

In-House Videographer/Photographer Job at GreatNeckTools

 ...communications. Capture dynamic visuals in both studio and real-world industrial/construction environments. Collaborate Partner with...  ...Qualifications Technical Skills Bachelors degree in Film, Media, Communications, or related field (or equivalent... 

Guthrie

Supervisor, Nursing - Nursing Administration - Part Time Job at Guthrie

 ...may be assigned as necessary in the performance of this position. About Us Joining the Guthrie team allows you to become a part of a tradition of excellence in health care. In all areas and at all levels of Guthrie, youll find staff members who have committed... 

Synergy Care, Inc.

Speech Language Pathologist Job at Synergy Care, Inc.

 ...Job Summary The Speech-Language Pathologist (SLP) assesses, diagnoses, and treats individuals with speech, language, voice, fluency, and swallowing disorders. The SLP works with patients across the lifespan to improve communication skills, cognitive-linguistic abilities... 

Hospice of the Western Reserve

Pediatric Home Care Registered Nurse (RN) Job at Hospice of the Western Reserve

 ...INTEGRITY. SERVICE. STEWARDSHIP . We have a passion for purpose driven work! Do you? JOIN OUR FAMILY! Provide palliative nursing care and support to pediatric patients and families WHAT YOU WILL DO: Assess physical, psychosocial and spiritual needs of the... 

Solomon Page

Administrative Assistant Job at Solomon Page

 ...We are hiring an Administrative Assistant to join a top consumer goods company in their Bellevue, WA Headquarters. This position will run for 3 months with the potential to extend. ~ Pay Rate: $20/hr Responsibilities: Extend a warm and professional welcome...