Senior Python Data Scraping Engineer (Freelance)

About the Role

What Mindrift Does

Key Responsibilities

• Own end-to-end extraction workflows across complex websites and deliver accurate structured datasets.
• Use tools and custom workflows to accelerate collection, validation, and task execution.
• Handle dynamic and interactive sources, including JavaScript-rendered content and changing site behavior.
• Enforce data quality through validation checks, cross-source consistency, formatting controls, and verification.
• Scale large-dataset scraping with batching or parallelization, monitor failures, and maintain stability.

Educational Qualifications

• At least five years of relevant experience in data engineering, web scraping, automation, or software development is required.
• A Bachelor's or Master's degree in Engineering, Applied Mathematics, Computer Science, or a related technical field is a plus.
• Candidates should have a strong technical foundation in scripting, automation, and data extraction workflows.
• The role requires problem-solving ability, independent work, and a methodical, detail-oriented approach.

Technical Skills

• Strong Python web scraping experience using BeautifulSoup, Selenium, or similar tools.
• Experience handling JavaScript, AJAX, infinite scroll, and APIs via proxies.
• Ability to extract data from hierarchies, archived pages, inconsistent HTML, and other complex structures.
• Experience with data cleaning, normalization, validation, and structured CSV, JSON, or Google Sheets output.
• Experience with anti-bot mechanisms, cloud infrastructure, Docker, and LLM frameworks such as LangChain or OpenRouter.
• A GitHub link is a plus.
• English proficiency at upper-intermediate level, B2 or above, is required.

Workplace

• This is a remote opportunity for technical professionals.

Project Time Expectations

• Tasks are estimated at approximately 10–20 hours per week during active project phases.
• The estimate is not a guaranteed workload and applies only while the project is active.

Compensation

• Contributors can earn up to $37 per hour equivalent on this project, depending on level and pace of contribution.
• Compensation varies by project scope, complexity, and required expertise.