AI evaluation jobs
Open roles at other companies — generalist, legal, medical, data science, finance, languages and more. Free to browse, no account needed.
19 open roles · listings checked on
Referral disclosure
Links on this board are referral links. When you click, sign up or get hired through one, the platform may pay us a commission under its own referral program.
- We don't hire. You apply on the employer's site, and the employer decides.
- We are not affiliated with these platforms.
No roles match these filters
Try a broader search or clear a filter to see more opportunities.
Generalist · 6
- Generalist
AI Trainer - Fluent French Speakers - Fully Remote
We are looking for fluent French speakers to help train the next generation of AI models - through real conversation. You'll join live video calls with other French-speaking participants, chatting naturally about everyday topics while your conversation is used to help AI models better understand how people actually speak.
- Generalist$50 - $90/hour pay
Generalist
Scope of Work Analyze, evaluate, and annotate a variety of data sets to support AI system training and validation. Deliver clear, high-quality written input and feedback based on project requirements. Participate in reviewing, refining, and improving model outputs to ensure relevance and accuracy. Engage in problem-solving tasks that require critical thinking and depth of understanding across multiple domains. Follow project guidelines and participate in quality assurance processes to consistently meet deliverable standards. Communicate findings and insights effectively through both written and verbal channels. Demonstrate strong attention to detail in all aspects of contribution. Preferred Qualifications Exceptional command of written and spoken English. Strong critical thinking and analytical skills. Proven ability to deliver precise, detail-oriented work in independent and collaborative settings. Experience with any type of data labeling or annotation projects (a plus, but not required). Comfort working in a fully remote, digital-first environment. Strong commitment to fulfilling project requirements on time and to a high standard.
Details: GeneralistApply for Generalist on micro1.aiReferral link — how we earn
- Generalist
AI Quality Analyst - English
As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. You will assess how well the model uses information from your past Gemini conversations, Gmail, Google Search, and YouTube activity to make responses more relevant and helpful. This role requires a unique blend of creativity and analytical rigor. You will actively design prompts from the perspective of your own personal experiences. You will then use your analytical skills to assess the quality of the model's personalized responses, evaluating dimensions like Grounding, Integration, and Helpfulness.
- Generalist$15-$20 per hour
Video Evaluation Generalist
Job Details: Evaluate Paired Video Outputs: Watch two short video clips presented side by side and compare them against a provided rubric across dimensions such as prompt adherence, motion quality, visual artifacts, and physical plausibility. Score and Justify Preferences: Assign rubric scores to each clip, select an overall preference between the two, and document a brief written justification supporting the decision where prompted. Maintain Scoring Consistency: Apply the same standard across high task volumes, holding calibration to the rubric steady so that judgments remain reliable and comparable throughout a production run.
- Generalist$50-$70
Generalist Expert (US/Canada)
We’re looking for US or Canada-based generalist experts to help train and evaluate frontier AI models. In this role, you’ll review and assess a wide range of everyday professional content — documents, slides, spreadsheets, and other written materials — judging them for quality, accuracy, clarity, and completeness. Your feedback directly shapes how AI systems reason about and produce real-world work product. This is a great fit if you’re a sharp, detail-oriented generalist who’s comfortable moving across different formats and subject areas. You don’t need deep specialization in any one field — strong judgment, careful reading, and clear written reasoning matter most. What you’ll do Review and evaluate documents, slides, spreadsheets, and similar materials for quality and correctness Provide clear, well-reasoned written feedback and ratings Compare and rank AI-generated outputs against defined criteria Flag errors, inconsistencies, and gaps in reasoning or formatting
- Generalist$20 - $20/hr
Exocentric Video Recorder
We are looking for individuals to collect video of themselves performing natural, everyday activities across a variety of indoor, outdoor, and vehicle environments. Collection may include third-person recordings of everyday activities as well as vehicle navigation tasks. Camera Requirements You must already own or have access to a compatible camera with: A minimum 170° field of view (FOV) A minimum recording resolution of 720p
TELUS Digital AI · 1
- TELUS Digital AI$45 - $75
Global AI Expert & Evaluator
TELUS Digital is building a global community of experts to evaluate, challenge and improve advanced AI systems. We are looking for university students, graduates, researchers and experienced professionals with expertise across Artificial Intelligence, Machine Learning, LLMs, Mathematics, Science, Engineering, Linguistics, Cybersecurity, Psychology, Law, Healthcare, Finance, Education and other specialist disciplines.
Expert · 12
- Expert$65-$100 per hour
Corporate Designer — Project Style
What you'll do Compare AI-generated versions of the same deliverable and score them across layout & spacing, typography, colour, domain realism, and polish Take the strongest candidate and rework it to reference quality Provide clear justifications tied to specific layout evidence (alignment, spacing, balance, grouping, density) Bring your design judgment to real corporate client materials You're a good fit if you Have a background as a designer, marketing/comms professional, or knowledge-work specialist — management consulting, investment banking, in-house corporate design, agency work, and marcom all fit Spot subtle layout inconsistencies most people miss: alignment gaps, unequal card heights, dead space, margin discipline Are opinionated about what "good" looks like in a business deliverable and can articulate it in writing Care about how work is presented, not just what it says
- Expert$125 per hour
Data Science Expert, AI Training Sprint (12 Weeks)
Key Responsibilities Work through tasks on our partner's platform, applying the same judgment you would apply on the job; Write and review model responses, explanations, and reference material to a professional standard; Correct model output where it is wrong and set out clearly why it is wrong; Produce context files and worked examples the model can learn from; Act on reviewer feedback and hold your quality across the full 12 weeks; Log 30 to 40 hours a week and keep your throughput steady; Work from your own expertise, without LLM assistance on projects that do not permit it. Ideal Qualifications PhD or MBA in a quantitative field, completed or in progress; At least 3 years of non-academic professional experience in data science or applied analytics; Strong Python and SQL, with real statistical grounding behind the tooling; Experience designing and reading experiments; Able to write up an analysis clearly enough for a peer to challenge it; Based in the United States, United Kingdom, Ireland, Canada, or Australia.
- ExpertUp to $90 / hr
React and Next.js Engineer
Develop AI Training Content: Create detailed prompts across React/Next.js topics and write high-quality reference answers to guide AI learning. Optimize AI Performance: Evaluate and rank AI responses to enhance correctness, clarity, accessibility, and performance. Ensure Model Integrity: Test AI models for inaccuracies or unsafe recommendations, validating reliability across real-world use cases.
- Expert$0 - $90 / hr
Dentist
Develop AI Training Content: Create detailed prompts and gold-standard answers across oral health, dental diagnosis, treatment planning, emergencies, prescribing, and patient counseling. Optimize AI Performance: Evaluate and rank AI responses to improve correctness, safety, clarity, and contextual relevance. Ensure Model Integrity: Test AI outputs for inaccuracies, unsafe recommendations, and bias; validate reliability across patient populations and use cases.
Details: DentistApply for Dentist on sme.careersReferral link — how we earn
- Expert$0 - $65 / hr
Nurse Practitioner
Develop AI Training Content: Create detailed prompts and gold-standard answers across nursing care topics (assessment, triage, safety, education, documentation). Optimize AI Performance: Evaluate and rank AI responses to enhance accuracy, safety, clarity, and contextual relevance. Ensure Model Integrity: Test AI outputs for potential inaccuracies, unsafe guidance, and bias; validate reliability across use cases.
- Expert$60-$70 per hour
Sales and Marketing Expert
Role Overview Mercor is seeking senior sales and marketing professionals to build evaluation tasks for AI systems operating in Fortune 500 go-to-market contexts. The workflows are calibrated to the deal sizes, stakeholder complexity, and brand stakes of Fortune 500 and large public companies. Contributors design enterprise GTM scenarios, draft reference outputs, and write rubrics that capture how senior F500 operators think. Key Responsibilities Construct enterprise sales scenarios spanning $1M+ ACV deals, multi-stakeholder buying committees, and complex procurement cycles at F500 accounts. Build marketing tasks across F500 brand strategy, enterprise ABM, demand generation at scale, lifecycle, and category positioning. Develop RevOps and GTM scenarios involving Salesforce Enterprise, Marketo, 6sense, Gong, and Outreach in F500 stacks. Apply enterprise sales methodologies (MEDDIC, Challenger, Force Management) and produce reference deal strategies, account plans, and executive narratives. Author rubrics that distinguish authentic enterprise GTM judgment from generic playbook recall.
- Expert$80-$120
Legal / compliance Evaluator
About the role We are hiring expert Evaluators in Legal / compliance to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise to grade outputs. This is a remote, hourly engagement.
- Expert$130-$170 per hour
Clinical Regulatory Expert
As a regulatory expert on this project, you would own a part in multiple workflows: 1. Clinical Data Generation: we need to validate that each data room and its documents hold up against FDA evidentiary standards and formatting expectations, including the deliberate defects planted to test regulatory judgment. 2. FDA RFI task authorship: you must be capable of authoring information requests and model responses, and designing prompts, golden outputs, and rubrics to quantify the ability of an AI agent attempting to do the same thing.
- Expert$45-$65
Certified Medical Coder (ICD-10) — Multilingual Clinical Documentation & AI Evaluation Hourly contract Remote New opportunity
Key Responsibilities Documentation & Coding Review Review clinical documentation for accuracy, completeness, and coding integrity Evaluate AI-generated clinical notes and summaries against real-world documentation and coding standards Validate alignment between clinical documentation and assigned codes
- Expert$100-$120 per hour
Legal Expert — Disputes & International (Project Donna)
What you'll do Read a synthetic legal matter and its underlying documents Review six blinded AI deliverables (labelled A–F) for that matter Make five head-to-head comparisons against a common comparator output Write a short, specific rationale for every preference Practice scope International arbitration and award enforcement, sanctions and export controls, cross-border disputes and emergency interim relief.
- Expert$138 per hour
Legal Expert — Disputes & International, UK (Project Donna)
What you'll do Read a synthetic legal matter and its underlying documents Review six blinded AI deliverables (labelled A–F) for that matter Make five head-to-head comparisons against a common comparator output Write a short, specific rationale for every preference Practice scope International arbitration and award enforcement, sanctions and export controls, cross-border disputes and emergency interim relief. Requirements Qualified lawyer with substantive practice experience in the areas above Able to judge work product holistically — correctness, completeness, judgment, usefulness Precise written English Able to work independently to written instructions and a deadline
- Expert$100 per hour
Technical Accounting and FP&A Experts: AI Training Scenarios
What We're Researching We're hiring technical accounting and FP&A experts to help build reinforcement learning environments for AI systems. This work focuses on creating complex financial scenarios, defining edge cases, and establishing evaluation rubrics. Your contributions will directly improve how artificial intelligence handles sophisticated financial and accounting tasks.
Get ready before you apply
A free account gives you your Readiness Index. Paid plans add practice and the AI interviewer.
Start free