Data Model Quality Analyst Job at Russell Tobin, San Jose, CA

ZDdWajgvZWcyZmwybWtab09kU2Z4UTFGU2c9PQ==
  • Russell Tobin
  • San Jose, CA

Job Description

Russell Tobin & Associates is currently seeking a Hybrid Data Model Quality Analyst to work for our client in the AI/tech industry. Apply now for consideration!

Contract: W2, 6- Months

Location: Hybrid | Tues–Thurs On-site in San Jose, CA

Pay Range: $75-85/Hour depending on experience

Responsibilities:

  • Ensure data quality during task development for human-created content used in training Large Language Models (LLMs).
  • Define detailed data requirements based on model goals and analysis of model loss patterns.
  • Design high-quality, nuanced instructions for expert content creators with illustrative examples.
  • Rapidly iterate with content-production vendors to improve data quality in small batch reviews.
  • Develop benchmark datasets to assess and validate data quality.
  • Benchmark vendor outputs against competitor models using both quantitative and qualitative methods.
  • Analyze data from vendor teams and optimize rater pools for quality.
  • Audit datasets, identify quality issues, and help develop tools to streamline data review processes.
  • Use experimentation to demonstrate data impact and uncover model blind spots.
  • Generate regular stakeholder reports on data quality metrics, iteration times, and vendor performance.
  • Author and maintain best practices documentation and instructional guidelines.

Requirements:

  • Bachelor’s degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
  • 3+ years of experience in Technical Writing, Quality Assurance, or a related field.
  • Excellent writing and editing skills with adaptability to varied styles and tones.
  • Strong analytical and problem-solving skills.
  • Ability to work independently and collaboratively across technical and non-technical teams.

Preferred Qualifications:

  • PhD in a relevant field (English, Linguistics, Computer Science, etc.)
  • 5+ years experience in relevant roles with exposure to LLM development or content creation at scale.
  • Experience with data benchmarking, quality auditing, or content vendor management.
  • Familiarity with model evaluation techniques and qualitative dataset analysis.

Russell Tobin / Pride Global offers eligible employees comprehensive healthcare coverage (medical, dental, and vision plans), supplemental coverage (accident insurance, critical illness insurance, and hospital indemnity), a 401(k)-retirement savings plan, life & disability insurance, an employee assistance program, identity theft protection, legal support, auto and home insurance, pet insurance, and employee discounts with preferred vendors.

#LI-JJ2

Job Tags

Contract work,

Similar Jobs

Pennant

CEO-In-Training, Executive Director Job at Pennant

 ...teams and has a passion for the senior living industry. Join us in making an impact and shaping the future of senior care! We have CEO In Training opportunities in the following areas: Wausau, Stevens Point, Green Bay, Madison and Milwaukee, WI. About the Opportunity... 

Hurray

Market Sales Manager - Hurray’s GIRL BEER (Austin) Job at Hurray

 ...unserious brands. Launched in Los Angeles in September 2024, we take an outrageous approach to beverages. Our first-born is Hurrays Girl Beer, the beer to out-girl all beers - flipping the pre-existing cultures of the beer category through diabolical anti-branding. We... 

Trinity Health Mid-Atlantic

Physician Assistant - Trauma Job at Trinity Health Mid-Atlantic

 ...Trinity Health Mid-Atlantic is seeking a Trauma Physician Assistant to provide evening coverage. The position is located at our main campus, St. Mary Medical Center, in Langhorne, Pennsylvania. Schedule is Night Shift: Four 10-hour shifts: 11:00 PM - 9:30 AM working... 

Tri Therapy

Physical Therapist and Physical Therapy Assistant Job at Tri Therapy

 ...wellness? If so, we have an exciting opportunity for you! At Tri Therapy, we believe in making a difference in people's lives through...  ...and personalized treatment plans. As a leading provider of physical therapy services in our community, we are dedicated to helping... 

BIOWORLD MERCHANDISING

Product Designer Job at BIOWORLD MERCHANDISING

 ...obsessed ~ Excels in highly collaborative environment with different work groups ~ Passion for fan culture and pop culture product ~...  ..., Fashion/Industrial BUSINESS HOURS We have a Work from Home flexible schedule. We work 4 days in office and 1 from home....