Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Jamail & Smith Construction, LP

Project Manager/Estimator Job at Jamail & Smith Construction, LP

 ...entities within the State of Texas. Specializing in Job Order Contracting (JOC), Design Build, and CSP Construction Services, we...  ...heart of our business model lies a vibrant focus on the K-12, government, and municipal construction sectors which drives our sustained... 

Volt

Order Fulfillment Associate Job at Volt

 ...inventory status, and maintain customer information and other relevant data for each transaction. Confirm orders, unit prices, shipping dates, update shipping statuses, and notify customers of any backorders or delivery delays. Provide price quotations, complete... 

Intrepid

Mine Operator Job at Intrepid

DescriptionJob Title:Mine OperatorReports To: Mine Shift SupervisorLocation: New Mexico East Plant(Relocation bonus or...  ...~Minimum one (1) year general labor and/or industrial plant experienceOPPORTUNITIES~Medical plans with prescription drug coverage... 

Memorial Sloan Kettering Cancer Center (MSK)

Tenure-Track Faculty Position in Chemistry or Chemical Biology Job at Memorial Sloan Kettering Cancer Center (MSK)

 ...Member or Associate Member level, or tenured positions at the Member (Professor) level, with strong research accomplishments in organic chemistry or chemical biology and interests in bringing chemical approaches to bear upon problems at the interface with biomedical... 

Springborn Staffing

CDL A Truck Driver Job at Springborn Staffing

 ...equipment (tractor/trailer/yard truck/ straight truck. As a Delivery Driver, you are also responsible for reviewing paperwork for...  ...to the Distribution Center first. Must have a valid Class A CDL. Must have a safe driving record / clear MVR. Able to operate...