Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Columbia Healthcare

General Dentist Job at Columbia Healthcare

 ...GENERAL DENTIST OPENING Shelby, NC Join a thriving, doctor-led group in a modern practice environment. ABOUT THE POSITION A well-established, doctor-led dental group is seeking a General Dentist to join their team in Shelby, NC. This modern, fully equipped office... 

PrismHR

Electrical Technician Job at PrismHR

 ...Electrical technician Location: Fort Lauderdale, FL Work schedule: 7:00 a.m. - 3:30 p.m. Competition: $31 - $39 dollars per hour (according to experience and skills) Essential Duties and Responsibilities Wear proper work attire/ jeans, cotton T-shirt... 

Worldwide Steel Buildings

Senior Building Consultant Job at Worldwide Steel Buildings

 ...and ship high-quality steel structures across the United States, Alaska and Hawaii. About the Position: As a Building Sales Consultant, you will be responsible for guiding customers through the entire steel building process, from initial contact to project... 

Upper East Side Tan

Weekend Salon Receptionist (11am to 7pm) Job at Upper East Side Tan

We are looking for a weekend front desk attendant to join our expanding team at Upper East Side Tan! As the first person that our members see when they enter the salon and the last person they see before they leave, you will become a face of our Upper East Side Tan brand...

American Income Life

Remote Union Rep - Life Benefits Job at American Income Life

 ...Remote Sales Representative Work from Anywhere 100% Remote | Flexible Schedule | Career Growth Opportunities Please Note: We are currently...  ...supplemental health products to help protect members of labor unions, credit unions, associations, and their families. AIL...