Software Engineer - Evaluation (Remote)
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
We are seeking a Senior Software Engineer with Go expertise to work part-time on LLM evaluation and repository validation. You will analyze GitHub issues, set up code repositories, and evaluate LLM performance on real-world coding tasks.
Key Skills for This Role
Responsibilities
- Analyze and triage GitHub issues across trending open source libraries to assess LLM performance
- Set up and configure code repositories, including Dockerization and environment setup for testing
- Evaluate unit test coverage and quality across complex codebases
- Modify and run codebases locally to assess how LLMs perform in bug fixing scenarios
- Collaborate with researchers to design and identify repositories and issues that challenge LLMs
Requirements
- Minimum 3+ years of overall software engineering experience
- Strong proficiency in Go
- Familiarity with Git, Docker, and basic software pipeline setup
- Ability to understand and navigate complex codebases efficiently
- Experience running, modifying, and testing real world projects locally
- Comfortable contributing to or evaluating open source projects
Full Job Posting
Role Overview
- We are hiring for one of our clients, seeking a Senior Software Engineer – Go (LLM Evaluation & Repository Validation) to work on a part time basis.
- The project involves building LLM evaluation and training datasets to train models on realistic software engineering problems.
- This includes developing verifiable software engineering tasks based on public repository histories using a synthetic approach with human in the loop input.
Key Responsibilities
- Analyze and triage GitHub issues across trending open source libraries to assess LLM performance.
- Set up and configure code repositories, including Dockerization and environment setup for testing.
- Evaluate unit test coverage and quality across complex codebases.
- Modify and run codebases locally to assess how LLMs perform in bug fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that challenge LLMs.
Required Skills & Qualifications
- Minimum 3+ years of overall software engineering experience.
- Strong proficiency in Go and familiarity with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases efficiently.
- Experience running, modifying, and testing real world projects locally.
- Comfortable contributing to or evaluating open source projects.
More About the Opportunity
- This role offers a unique opportunity to work with a global leader in the technology industry, contributing to the advancement of AI assisted software development.
- The position blends practical software engineering with AI research, enabling hands on impact in evaluating LLMs against real world coding challenges.
Equal Opportunity Employer
- We hire based on skills and expertise. All qualified candidates are welcome regardless of background, experience, or prior employment history.
- Applications are reviewed solely on demonstrated technical ability and qualifications.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Hire Feed
Research Analyst (Remote)
Abu Dhabi, UAE
Hire Feed is hiring a Research Analyst for a client to work on fine-tuning large language models. The role involves reading, summarizing, and breaking down content, conducting online research, and validating claims. Requ
Machine Learning Analyst (Remote)
Abu Dhabi, UAE
Role: Machine Learning Analyst (Remote) Location: Remote (Work from Anywhere) Job Type: Part-Time Payout: Competitive, based on experience Role Overview: We are hiring for one of our clients, seeking a MLE Bench – Dat
Technical Content Writer (Remote)
Abu Dhabi, UAE
Hiring a part-time Technical Content Writer to produce clear technical documentation and support data-driven work using JSON and public datasets. Requires 3+ years of experience in technical writing, analysis, or softwar
Analytical Chemist (Remote)
Abu Dhabi, UAE
Hire Feed is hiring a Chemistry Expert (PhD/Master's) to work on fine-tuning large language models by creating and solving advanced chemistry problems. The role requires strong analytical skills, proficiency in organic c
SQL Developer (Remote)
Abu Dhabi, UAE
We are hiring a SQL Developer for a contract role. You will design, develop, and maintain SQL-based solutions, optimize database performance, and collaborate with cross-functional teams. Proficiency in SQL, query optimiz
Biologist (Remote)
Abu Dhabi, UAE
Hire Feed is hiring a Biology Expert (PhD/Master's) to design and solve biology problems for evaluating large language models. The role involves creating high-quality solutions, collaborating with LLM researchers, and de
ML Engineer (Remote)
Abu Dhabi, UAE
Role : ML Engineer (Remote) Location : Remote (Work from Anywhere) Job Type : Part-Time Payout : Competitive, based on experience Role Overview: We are hiring for one of our clients, seeking a MLE Bench – ML Engineers
Business Analyst - Arabic Language (Remote)
Abu Dhabi, UAE
Role: Business Analyst - Arabic Language (Remote) Location: Remote (Work from Anywhere) Job Type: Full-Time Payout: Competitive, based on experience Role Overview: We are hiring for one of our clients, seeking a Busin
Research Analyst (Remote)
Abu Dhabi, UAE
Machine Learning Analyst (Remote)
Abu Dhabi, UAE
Technical Content Writer (Remote)
Abu Dhabi, UAE
Analytical Chemist (Remote)
Abu Dhabi, UAE
SQL Developer (Remote)
Abu Dhabi, UAE
Biologist (Remote)
Abu Dhabi, UAE
ML Engineer (Remote)
Abu Dhabi, UAE
Business Analyst - Arabic Language (Remote)
Abu Dhabi, UAE