Software Engineer - Post-training Data

As a Software Engineer working on our post-training data infrastructure, you write efficient and robust data pipelines and work with the research and Applied teams to devise strategies for gathering and maintaining specific, diverse datasets at scale.

Develop creative ways to obtain post-training datasets teaching the model targeted capabilities.

Develop creative ways to reliably generate synthetic datasets

Iterate on filtering and scoring heuristics for a post-training dataset

Contribute to our data infrastructure by implementing, maintaining and testing data pipelines serving workloads across scales from gigabytes to 100s of petabytes

Extreme attention to detail and commitment to data quality.

Ability to write reliable, well-tested code and quickly learn new tools, programming languages or frameworks needed for a given job.

Versatility in end-to-end data pipelines, from scraping to processing.

High intellectual agility and grit to tackle tough challenges.

Magic strives to be the place where high-potential individuals can do their best work. We value quick learning and grit just as much as skill and experience.