Software Engineer - AI Code Evaluation (Remote)
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
We are seeking a Senior Software Engineer for LLM Evaluation & Repository Validation on a part-time basis. The role involves hands-on software engineering including GitHub issue triaging, repository setup, and evaluating test coverage to train AI models.
Key Skills for This Role
Responsibilities
- Analyze and triage GitHub issues across trending open source libraries
- Set up and configure code repositories, including Dockerization and environment setup
- Evaluate unit test coverage and quality in real world codebases
- Modify and run codebases locally to assess LLM performance in bug fixing scenarios
- Collaborate with researchers to identify repositories and issues challenging for LLMs
Requirements
- Strong experience with at least one of: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby
- Proficiency with Git, Docker, and basic software pipeline setup
- Ability to understand and navigate complex codebases
- Comfortable running, modifying, and testing real world projects locally
- Experience contributing to or evaluating open source projects is a plus
Full Job Posting
Role Overview
- We are hiring for one of our clients, seeking a Senior Software Engineer – LLM Evaluation & Repository Validation to work on a part time basis.
- The team is building LLM evaluation and training datasets to train AI models on realistic software engineering problems, including verifiable SWE tasks based on public repository histories.
- This role involves hands on software engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality.
Key Responsibilities
- Analyze and triage GitHub issues across trending open source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluate unit test coverage and quality in real world codebases.
- Modify and run codebases locally to assess LLM performance in bug fixing scenarios.
- Collaborate with researchers to identify repositories and issues challenging for LLMs.
Required Skills & Qualifications
- Strong experience with at least one of the following languages: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing real world projects locally.
- Experience contributing to or evaluating open source projects is a plus.
More About the Opportunity
- This role offers a unique opportunity to work with a global leader in the Software Development | Technology, Information and Internet | Information Services | Data Infrastructure and Analytics industry.
- Contributing to the advancement of AI assisted software development.
- The work directly influences how LLMs interact with real code, shaping the future of automated software engineering.
Equal Opportunity Employer
- We hire based on skills and expertise.
- All qualified candidates are welcome regardless of background, experience, or prior employment history.
- Applications are reviewed solely on demonstrated technical ability and qualifications.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Hire Feed
Research Analyst (Remote)
Abu Dhabi, UAE
Hire Feed is hiring a Research Analyst for a client to work on fine-tuning large language models. The role involves reading, summarizing, and breaking down content, conducting online research, and validating claims. Requ
Machine Learning Analyst (Remote)
Abu Dhabi, UAE
Role: Machine Learning Analyst (Remote) Location: Remote (Work from Anywhere) Job Type: Part-Time Payout: Competitive, based on experience Role Overview: We are hiring for one of our clients, seeking a MLE Bench – Dat
Technical Content Writer (Remote)
Abu Dhabi, UAE
Hiring a part-time Technical Content Writer to produce clear technical documentation and support data-driven work using JSON and public datasets. Requires 3+ years of experience in technical writing, analysis, or softwar
Analytical Chemist (Remote)
Abu Dhabi, UAE
Hire Feed is hiring a Chemistry Expert (PhD/Master's) to work on fine-tuning large language models by creating and solving advanced chemistry problems. The role requires strong analytical skills, proficiency in organic c
SQL Developer (Remote)
Abu Dhabi, UAE
We are hiring a SQL Developer for a contract role. You will design, develop, and maintain SQL-based solutions, optimize database performance, and collaborate with cross-functional teams. Proficiency in SQL, query optimiz
Biologist (Remote)
Abu Dhabi, UAE
Hire Feed is hiring a Biology Expert (PhD/Master's) to design and solve biology problems for evaluating large language models. The role involves creating high-quality solutions, collaborating with LLM researchers, and de
ML Engineer (Remote)
Abu Dhabi, UAE
Role : ML Engineer (Remote) Location : Remote (Work from Anywhere) Job Type : Part-Time Payout : Competitive, based on experience Role Overview: We are hiring for one of our clients, seeking a MLE Bench – ML Engineers
Business Analyst - Arabic Language (Remote)
Abu Dhabi, UAE
Role: Business Analyst - Arabic Language (Remote) Location: Remote (Work from Anywhere) Job Type: Full-Time Payout: Competitive, based on experience Role Overview: We are hiring for one of our clients, seeking a Busin
Research Analyst (Remote)
Abu Dhabi, UAE
Machine Learning Analyst (Remote)
Abu Dhabi, UAE
Technical Content Writer (Remote)
Abu Dhabi, UAE
Analytical Chemist (Remote)
Abu Dhabi, UAE
SQL Developer (Remote)
Abu Dhabi, UAE
Biologist (Remote)
Abu Dhabi, UAE
ML Engineer (Remote)
Abu Dhabi, UAE
Business Analyst - Arabic Language (Remote)
Abu Dhabi, UAE