Job Description

Russell Tobin & Associates is currently seeking a Hybrid Data Model Quality Analyst to work for our client in the AI/tech industry. Apply now for consideration!

Contract: W2, 6- Months

Location: Hybrid | Tues–Thurs On-site in San Jose, CA

Pay Range: $75-85/Hour depending on experience

Responsibilities:

• Ensure data quality during task development for human-created content used in training Large Language Models (LLMs).
• Define detailed data requirements based on model goals and analysis of model loss patterns.
• Design high-quality, nuanced instructions for expert content creators with illustrative examples.
• Rapidly iterate with content-production vendors to improve data quality in small batch reviews.
• Develop benchmark datasets to assess and validate data quality.
• Benchmark vendor outputs against competitor models using both quantitative and qualitative methods.
• Analyze data from vendor teams and optimize rater pools for quality.
• Audit datasets, identify quality issues, and help develop tools to streamline data review processes.
• Use experimentation to demonstrate data impact and uncover model blind spots.
• Generate regular stakeholder reports on data quality metrics, iteration times, and vendor performance.
• Author and maintain best practices documentation and instructional guidelines.

Requirements:

• Bachelor’s degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
• 3+ years of experience in Technical Writing, Quality Assurance, or a related field.
• Excellent writing and editing skills with adaptability to varied styles and tones.
• Strong analytical and problem-solving skills.
• Ability to work independently and collaboratively across technical and non-technical teams.

Preferred Qualifications:

• PhD in a relevant field (English, Linguistics, Computer Science, etc.)
• 5+ years experience in relevant roles with exposure to LLM development or content creation at scale.
• Experience with data benchmarking, quality auditing, or content vendor management.
• Familiarity with model evaluation techniques and qualitative dataset analysis.

Russell Tobin / Pride Global offers eligible employees comprehensive healthcare coverage (medical, dental, and vision plans), supplemental coverage (accident insurance, critical illness insurance, and hospital indemnity), a 401(k)-retirement savings plan, life & disability insurance, an employee assistance program, identity theft protection, legal support, auto and home insurance, pet insurance, and employee discounts with preferred vendors.

#LI-JJ2

Job Tags

Similar Jobs