Data Model Quality Analyst Job at Russell Tobin, San Jose, CA

VnJnZkcyL1lhN1F0Rjk0TjNtbDhocHZrT3c9PQ==
  • Russell Tobin
  • San Jose, CA

Job Description

Russell Tobin & Associates is currently seeking a Hybrid Data Model Quality Analyst to work for our client in the AI/tech industry. Apply now for consideration!

Contract: W2, 6- Months

Location: Hybrid | Tues–Thurs On-site in San Jose, CA

Pay Range: $75-85/Hour depending on experience

Responsibilities:

  • Ensure data quality during task development for human-created content used in training Large Language Models (LLMs).
  • Define detailed data requirements based on model goals and analysis of model loss patterns.
  • Design high-quality, nuanced instructions for expert content creators with illustrative examples.
  • Rapidly iterate with content-production vendors to improve data quality in small batch reviews.
  • Develop benchmark datasets to assess and validate data quality.
  • Benchmark vendor outputs against competitor models using both quantitative and qualitative methods.
  • Analyze data from vendor teams and optimize rater pools for quality.
  • Audit datasets, identify quality issues, and help develop tools to streamline data review processes.
  • Use experimentation to demonstrate data impact and uncover model blind spots.
  • Generate regular stakeholder reports on data quality metrics, iteration times, and vendor performance.
  • Author and maintain best practices documentation and instructional guidelines.

Requirements:

  • Bachelor’s degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
  • 3+ years of experience in Technical Writing, Quality Assurance, or a related field.
  • Excellent writing and editing skills with adaptability to varied styles and tones.
  • Strong analytical and problem-solving skills.
  • Ability to work independently and collaboratively across technical and non-technical teams.

Preferred Qualifications:

  • PhD in a relevant field (English, Linguistics, Computer Science, etc.)
  • 5+ years experience in relevant roles with exposure to LLM development or content creation at scale.
  • Experience with data benchmarking, quality auditing, or content vendor management.
  • Familiarity with model evaluation techniques and qualitative dataset analysis.

Russell Tobin / Pride Global offers eligible employees comprehensive healthcare coverage (medical, dental, and vision plans), supplemental coverage (accident insurance, critical illness insurance, and hospital indemnity), a 401(k)-retirement savings plan, life & disability insurance, an employee assistance program, identity theft protection, legal support, auto and home insurance, pet insurance, and employee discounts with preferred vendors.

#LI-JJ2

Job Tags

Contract work,

Similar Jobs

Vaughan Nelson Investment Management

Equity Research Associate Job at Vaughan Nelson Investment Management

 ...Houston-based investment management firm of highly experienced, research-oriented professionals. Since being founded in 1970, we have...  ...We are seeking a highly motivated and detail-oriented Research Associate to support our investment team. As a Research Associate, you... 

Joseph Gunnar & Co., LLC

Registered Investment Advisor Job at Joseph Gunnar & Co., LLC

***CURRENTLY SERIES 7 REGISTERED REPS ONLY******SALARY & BONUS and Full benefits*******PRIVATE EQUITY SALES *** Joseph Gunnar & Co., LLC is a full-service broker-dealer founded in 1997 dedicated to assisting investors in achieving their financial goals. The... 

PTS Advance

Computer Aided Design Drafter Per diem Job at PTS Advance

 ...Background and drug test required. Must have the ability to aquire a TWIC card as required for entry. Local Candidate only (no per diem) Start date - June 2025 Pay rate: $40 + hr. DOE Hours 5/8 schedule (40 hrs) On-Site Job Responsibilities: Update... 

Bentley Meeker Lighting & Staging, Inc.

Associate Event Designer, Lighting & Audio Visual for Live Events Job at Bentley Meeker Lighting & Staging, Inc.

 ...Designer will also work with clients to understand their needs, budget, and timeline. Qualifications ~ Bachelor's degree in Theatre Design or a related field is preferred ~2+ years of experience in lighting and audio visual design, production, and implementation... 

A-V Services, Inc.

Lead Audio Engineer Job at A-V Services, Inc.

A-V Services Inc., a leading AV company with multiple fortune 100 contracts within the financial, pharmaceutical, and computing industry, seeks a lead audio engineer (A1) to operate advanced sound systems in a corporate environment. These systems are predominantly permanent...