Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Russell Tobin

Data Integrity Analyst Job at Russell Tobin

 ...Job Title: Data/Quality Analyst Location: Cupertino, CA Fully onsite Duration: 12+ Months with Possibility Extension Pay Rate: $35/hr on W2 Job Description: Quality Assurance Analyst Data Integrity General Description: The client is dedicated to creating... 

Wellpath

Mental Health Administrative Assistant Job at Wellpath

 ...members and their family to support physical, mental, and financial wellbeing including:-...  ...your money as you earn it!- Tuition Assistance and dependent Scholarships- Employee Assistance...  ...(EAP) including free counseling and health coaching- Company paid life insurance-... 

SoCalJCB

SoCal JCB - Service Specialist Job at SoCalJCB

Company Profile SoCal JCB is a highly successful, fast-growing business. We pride ourselves in customer service. We invest in our workforce and offer a highly competitive compensation and benefit program. SoCal JCB is an authorized dealer for JCB construction equipment...

The Judge Group

Physician Assistant / Nurse Practitioner - Occupational Health Job at The Judge Group

 ...About the Role Were hiring a Physician Assistant or Nurse Practitioner to join a weekday outpatient clinic focused on occupational health. This role offers a consistent schedule, competitive compensation, and the opportunity to sharpen procedural skills in a collaborative... 

Provider Solutions & Development

Orthopedic Surgeon - Foot & Ankle Job at Provider Solutions & Development

Seeking a board-certified/board-eligible Orthopedic Foot and Ankle surgeon for a full-time position in Lubbock, Texas. Join a well-established team of general Orthopedists, hand surgeons and a Trauma surgeon at Covenant Medical Group's Orthopedic service. Work at a Level...