Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

dE91M3ZjNWxza3BUMG52d2d6U2xDdVR2NVE9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Insight Global

Dentist Job at Insight Global

 ...JOB DESCRIPTION Our client is seeking a dentist for either full-time or part-time work. This person will be working with various demographics, including children occasionally. Types of procedures: ---Restorative Adult: composites, amalgams, crowns --Endo:... 

Glenmark Pharmaceuticals

Warehouse Associate Job at Glenmark Pharmaceuticals

 ...warehouse associates when needed.OVERALL JOB RESPONSIBILITIES:~Manage time and work load for shift operations~Prepare shipments by processing requests and supply orders; pulling materials; packing boxes ~Organize warehouse and work area for orderliness at all times.~... 

KMK Consulting Inc.

Principal, Real World Evidence Job at KMK Consulting Inc.

KMK is a global data analytics and technology consulting company empowering leaders across the Life Sciences industries to make better data-driven decisions. Our data analytics and software platforms support data science, commercial operations, real world evidence, ...

Complete Staffing LLC

Fence Laborer Twic card required Job at Complete Staffing LLC

Our client is passionate about providing their customers with the best products and installation. We are looking for experienced fence installers or someone eager to learn the trade to join our team as an installer. Requirements are : T.W.I.C ( must have on hand and...

Peak Recruiter, Sanford Rose and Associates

Interim Director and Executive Positions - Acute Care Job at Peak Recruiter, Sanford Rose and Associates

 ...leading hospitals and medical groups through permanent search and interim leadership services. The company focuses on customized recruitment services to help candidates succeed in reaching their healthcare-related initiatives. Peak Recruiter is known for its industry-...