Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

The Glass Jar

Shift Lead - The Penny Ice Creamery Job at The Glass Jar

 ...The Penny Ice Creamery | Made From Scratch The Penny Ice Creamery is seeking Shift Leads for our Scotts Valley location! The Penny Ice Creamery is the only ice cream shop in Santa Cruz that makes ice cream completely from scratch, in house, using local and... 

LHH

Administrative Assistant Job at LHH

Administrative Assistant Location: Nashville, TN Job Type: Contract-to-Hire About the Role: We're looking for a highly organized and personable Administrative Assistant to join our client's team in Nashville! This role is perfect for someone who thrives in a ...

Cayetano Development

Bilingual Administrative Clerk Job at Cayetano Development

 ...Administrative Clerk: Greater Laredo, TX Part-Time | Onsite Description The Administrative Clerk is the first point of contact for clients and supports daily office operations. Responsibilities include managing calls, calendars, documentation, and application... 

BLDG Partners

Asset Manager (Affordable Housing) - Virginia Job at BLDG Partners

 ...LLC is a Southern California based real estate investment firm founded in 2010 focused on the preservation of workforce and affordable housing.We pursue opportunities to improve communities in urban and suburban markets across the country. Position: BLDG Partners... 

TORQ Coatings

Call Center Representative Job at TORQ Coatings

 ...Location: Lombard, IL Compensation: $19-$21/hr Job Type: Part-Time, Onsite Industry: Consumer Services / Construction /...  ...brand. Join our team and take your career to the next level in a company that values craftsmanship, leadership, and professional excellence...