Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Insight Global

Safety Manager Job at Insight Global

 ...Position: Safety Manager Location: Northern California, Southern Washington, or Oregon Schedule: Hybrid Pay Rate: $100,000 - $115,000 Duration: Permanent Start Date: ASAP Must Haves: CALOSHA (California's OSHA)2 -3 years of experience with construction... 

NOVACES

Engineer FEMA Public Assistance Job at NOVACES

 ...solutions for both government and commercial clients. We support disaster recovery efforts by delivering skilled professionals to assist FEMA and other agencies in rebuilding communities and enhancing resilience after major events. Our expertise spans across a wide range of... 

Construction Testing Services

CWI/ICC Structural Steel & Bolting Inspector Job at Construction Testing Services

 ...Nevada. We competitively bid projects and provide the highest level of service including project management and budget control services. CTS is seeking is seeking a CWI/ICC Structural Steel & Bolting Inspector to perform inspections for the shop/field. To include the... 

Memorial Healthcare System

Dishwasher - Porter - PT - Days - MHW Job at Memorial Healthcare System

 ...entrusted to our care. An unwavering commitment to our service vision is what makes the difference. It is the foundation of The Memorial Experience. Summary Cleans and maintains all areas of the Foodservice Department according to safety, sanitation and infection control... 

Mitsubishi UFJ Trust and Banking Corporation, New York Branc...

Global Securities Lending Solutions (GSLS) Summer Intern Job at Mitsubishi UFJ Trust and Banking Corporation, New York Branc...

 ...with an expected graduation date of May 2027 or May 2028 The internship is from June 1, 2026 - August 31, 2026. The typical base pay...  ...savings plan, educational assistance and training programs, paid maternity and parental bonding leave, and paid vacation, sick leave...