Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Optum

Nursing Manager Job at Optum

 .... Work with one of the nations leading health care organizations and build your career...  ...Supervise daily clinical operations and manage nursing staff across oncology service lines...  ...~ Strong computer skills and ability to travel between centers Preferred Qualifications... 

elite personnel

Office Administrator Job at elite personnel

 ...The anticipated annual salary for this position is approx. $55,000+ plus bonuses. Our client offers a comprehensive benefits package that includes health insurance, retirement savings plans, paid time off, and other employee programs. Interested? Apply today!... 

Onward Search Education

School Nurse [80846] Job at Onward Search Education

 ...with the nations schools. Were partnering with a school district in Providence County, RI to hire a dedicated Licensed Practical Nurse (LPN) or Registered Nurse (RN) to support a kindergarten student during daily transportation. This role involves riding the school... 

RoslinCT

Process Engineer Job at RoslinCT

 ...corporate culture, peoples development, growth, and the ability to impact patients. ACCELERATING YOUR FUTURE The Process Engineer I/II will develop and execute technology transfer plans for RoslinCTs client processes, including working with key stakeholders... 

Excelsia Injury Care

Licensed Clinical Social Worker Job at Excelsia Injury Care

 ...providers are leaders in personal injury and workers compensation care, with a proven track...  ...those injured in motor vehicle or work-related accidents. We take an interdisciplinary...  ...citizens, we integrate environmental, social, and governance (ESG) considerations into...