Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Mint Cannabis

Security Guard Job at Mint Cannabis

 ...future. Be part of the Mint Cannabis journey and see the difference! Job Summary Mint Cannabis is seeking a vigilant and reliable Security Guard to join our team. The ideal candidate will maintain a secure environment for employees, visitors, and property by observing... 

Total Quality Logistics

Fast Track to Leadership - Account Executive - $2,500 Sign-on Bonus, $7,500 in Rental Assistance Job at Total Quality Logistics

 ...~$2500 sign on bonus ~$7500 Housing Stipend paid in bi-weekly increments for the first 12 months ~ PAID relocation (up to $2500 in relocation reimbursement) to our headquartered city of Cincinnati, OH to be trained by the top producers in our company ~ PAID training... 

Domino's Franchise

Customer Service Rep - 5219 Dixie Hwy Job at Domino's Franchise

Job Description Customer Service Representatives with Dominos Pizza undertake a variety of duties. Job duties include: Taking phone calls Taking orders Making pizzas Completing cash transactions Providing customers a great customer service experience...

Balbec Capital LP

Frontend Developer (React, Net C#) Job at Balbec Capital LP

 ...employment applicants. The firm strives to maintain an environment free of discrimination based on race, color, religion, gender,...  ...transfer, reductions in force, social and recreational programs, training, employee development, compensation and fringe benefits, discipline... 

Insigneo

Trader Job at Insigneo

 ...As a Trader, your primary responsibility is to execute trades in fixed income, equity, and option securities optimizing portfolio performance and managing market risk. You will work closely with investment professionals, portfolio managers, analysts, and other traders...