Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Amtec Staffing

Facility Security Officer Job at Amtec Staffing

Title: Facility Security Officer Reports to: Director of Security Works with: Mid-level Managers and Supervisors, Employees Location: Coachella, California, 92236 (relocation assistance is available) Type: Full-Time Schedule: 4/10 work week (M-Th 7 am...

luckyscent

Graphic Designer Job at luckyscent

About Us Since 2002, Luckyscent has been the playground for perfume lovers who refuse to smell like everyone else. We scout the globe for niche, rare, rule-breaking fragrances and bring them to our community through our online store and our boutiques in NYC, Hollywood...

Insight Global

Telecommunications Technician Job at Insight Global

 ...with opportunity for extension / conversion. Required Skills & Experience ~3+ years of large fiber installation experience ~...  ...employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable... 

Marshall Industrial Technologies Inc.

Sales Manager Job at Marshall Industrial Technologies Inc.

 .../7 availability, our team is dedicated to meeting the needs of businesses around the clock. Role Description This is a full-time, hybrid location Sales Manager role based in Trenton, NJ and Stockertown, PA. Experience with Mechanical, Electrical, HVAC/R and/or... 

Ardent Health

MA Endocrinology Job at Ardent Health

 ...Join our team as a day shift, full-time, Endocrinology Medical Assistant (MA) in Tulsa, OK. Why Join Us? Thrive in a People-First Environment and Make Healthcare Better Thrive: We empower our team with career growth opportunities, tuition assistance , and...