Software Engineer - Python | Remote

Position: Senior Software Engineer – Python (LLM Evaluation & Repository Validation) Type: Contractor Assignment (3 months) Compensation: $19-$20/hr Location: Remote Commitment: 40 hrs/week with some overlap with PST Role Responsibilities Evaluate and analyze GitHub issues across widely used open-source repositories. Set up and configure development environments for repositories, including Docker setup. Modify and run codebases locally to evaluate LLM performance in debugging and bug-fixing scenarios. Evaluate unit test coverage and assess the quality of existing tests. Work with real-world Python repositories to identify engineering tasks suitable for LLM evaluation. Collaborate with researchers to expand datasets used for training and evaluating AI models. Identify repositories and issues that present challenging real-world scenarios for LLMs. Support the creation of realistic software engineering tasks based on repository histories. Requirements Strong experience in software development. Strong experience with Python programming. Proficiency in Git, Docker, and development environment setup. Experience working with well-maintained public GitHub repositories. Ability to understand, modify, and test complex codebases. Strong analytical thinking and problem-solving skills. Excellent written English and ability to document findings clearly. Ability to work independently in a remote environment. Desktop or laptop with a stable internet connection. Preferred Qualifications Experience contributing to open-source projects. Exposure to LLM evaluation, AI research, or machine learning projects. Experience with developer tools, automation agents, or CI/CD pipelines. Application Process Application Form ICF + Profile Review Assessment Submit

Evaluate and analyze GitHub issues across open-source repositories and set up development environments. Collaborate with researchers to expand datasets for AI model training and identify engineering tasks for LLM evaluation.

Candidates should have strong experience in software development and Python programming, along with proficiency in Git and Docker. They must also possess analytical thinking skills and the ability to work independently in a remote environment.

Identified from the job description. Confirm important requirements above.

• Python
• Git
• Docker
• Software Development
• Analytical Thinking
• Problem-Solving
• Unit Testing
• AI Research
• Machine Learning
• Open-Source Projects
• Development Environment Setup
• Code Modification
• Documentation
• Collaboration
• CI/CD Pipelines

• Technology
• Software
• Engineering
• Science & Research

Minimum experience

5+ years

Posting language

English

Working hours

40 hours per week

Seniority

Associate

Application method

Direct apply is available