greenhouse
Data Engineer
M9 Solutions
Springfield, USA
Full-time
Senior · 10+ years experience
Onsite
USD 60000-180000 yearly / year
Discovered 1 weeks ago
PythonPandasPySparkFastAPIAWSAzure
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonPandasPySpark
Key Skills for This Role
PythonPandasPySparkFastAPIAWSAzure
Full Job Posting
Responsibilities
- Data Ingestion & Acquisition: Collect and integrate data from a wide variety of structured and unstructured sources, including APIs, RDBMS, file systems, third-party services, and real-time streams.
- Pipeline Development: Design and implement scalable ETL/ELT pipelines to clean, enrich, normalize, and semantically align data (ontology-driven transformations).
- Cloud Deployment: Build and deploy data pipelines and associated infrastructure on AWS or Azure, using managed services like Lambda, Glue, Step Functions, Azure Data Factory, etc.
- Database Architecture: Understand and optimize for different storage engines—relational (PostgreSQL, MySQL), columnar (Redshift, BigQuery), indexing engines (ElasticSearch), key-value stores (DynamoDB, Redis), Object stores (S3 or similar), and caching layers.
- Streaming Data Processing: Work with Apache Kafka (or similar platforms) to handle high-volume, low-latency data streams.
- Workflow Orchestration: Utilize Apache Airflow (or equivalent) to schedule and monitor complex data workflows.
- AI/ML Integration: Collaborate with data scientists to integrate LLMs and ML models into pipelines for inference, tagging, enrichment, or intelligent routing of data.
Required Skills and Qualifications
- Active TS/SCI security clearance.
- Bachelor's or master’s degree in computer science, engineering, or related field.
- 10+ years of experience in data engineering or software development roles.
- Strong proficiency in Python, including experience with libraries like pandas, PySpark, FastAPI, or similar.
- Solid experience with cloud services (AWS or Azure) and Cloud native data engineering tools.
- Proven experience in building and maintaining data pipelines using Kafka, Airflow, and other open-source frameworks.
- Strong grasp of database internals and trade-offs between different storage technologies.
- Familiarity with data governance, lineage, and metadata management concepts.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at M9 Solutions
Senior Advanced TIDES Engineer
Scottsdale, USA
Senior
M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance
Discovered YesterdayView →
IT Systems Engineer
Sunnyvale, USA
Mid
M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance
Discovered YesterdayView →
Senior Firmware Engineer
Taunton, USA
Senior
M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance
Discovered YesterdayView →
Senior Digital Hardware Designer
Scottsdale, USA
Senior
M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance
Discovered YesterdayView →
ETS Senior Systems Engineer
Manassas, USA
Senior
M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance
Discovered YesterdayView →
Senior Advanced TIDES Engineer
Scottsdale, USA
Discovered Yesterday
IT Systems Engineer
Sunnyvale, USA
Discovered Yesterday
Senior Firmware Engineer
Taunton, USA
Discovered Yesterday
Senior Digital Hardware Designer
Scottsdale, USA
Discovered Yesterday
ETS Senior Systems Engineer
Manassas, USA
Discovered Yesterday
DevOps Engineer
Chantilly, USA
Discovered 1 weeks agoFull-time
Network Engineer
, USA
Discovered 1 weeks agoFull-time
Data Engineer
Bethesda, USA
Discovered 1 weeks agoFull-time