SWE-Bench Task Auditor

About this role

Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository-level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric-based…

Disclosure: Skillora is independent of Mercor. The apply button carries our referral code, and if you are placed, Mercor pays Skillora a fee. It does not change what you are paid, our ranking does not know which listings pay us, and selection is decided by Mercor’s own vetting.