adp
Mid-Level Data Engineer (On-Site in Washington, DC)
Agile5 Technologies, Inc.
USA
Full-time
Senior · 5+ years experience
Onsite
USD 68000-152000 yearly / year
Discovered 1 weeks ago
PythonPySparkSQLInformaticaDelta LakeDatabricks
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonPySparkSQL
Key Skills for This Role
PythonPySparkSQLInformaticaDelta LakeDatabricks
Full Job Posting
Mid-Level Data Engineer Job Duties
- Execute daily data migration operations including data profiling, schema mapping, pipeline conversion, and automated reconciliation for Low and Medium complexity Informatica artifacts.
- Convert Informatica mappings into well-documented Python/PySpark code, ensuring all business logic and data quality controls are preserved.
- Migrate legacy Hive tables to Delta Lake format on S3 using Databricks ingestion tools.
- Build and execute automated data reconciliation scripts to validate migration accuracy and establish Change Data Capture (CDC) pipelines for ongoing synchronization.
- Commit all converted code into Azure DevOps with clear documentation and inline comments while maintaining Unity Catalog configurations.
- Perform daily data profiling and side-by-side validation within legacy enclave environments.
- Support Power BI and ESRI integration testing and validation.
- Participate actively in peer code reviews, daily Agile ceremonies, and collaborative data validation sessions.
- Contribute to Data Quality Assessment Reports and support training and knowledge transfer activities.
- Performs other duties as assigned.
Security Clearance Requirements
- Public Trust / Tier 4 Eligible: No clearance required to apply; must be a U.S. citizen willing to undergo a background check to obtain a Public Trust / Tier 4 clearance.
Experience Requirements
- Minimum experience required varies by degree level: PhD with 0 years; Master's degree with 3 years; Bachelor's degree with 5 years; or High School Diploma with 9 years of relevant experience.
- Proficiency in Python for data transformation and pipeline development, as well as SQL for query development and schema analysis.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Agile5 Technologies, Inc.
Part Time Data Scientist (1099 - Contractor)
, USA
SeniorPart Time, Contract
Discovered 1 weeks agoView →
Junior Systems Administrator (On Site in Clarksburg, WV)
Clarksburg, USA
EntryFull-time
Discovered 1 weeks agoView →
Senior Systems Administrator (On-Site in Clarksburg, WV)
Clarksburg, USA
SeniorFull-time
Discovered 1 weeks agoView →
QA & Test Engineer
, USA
Discovered 1 weeks agoFull-time
Senior Data Engineer (On Site, Washington, DC)
, USA
Discovered 1 weeks agoFull-time
Part Time Data Scientist (1099 - Contractor)
, USA
Discovered 1 weeks agoPart Time, Contract
Junior Systems Administrator (On Site in Clarksburg, WV)
Clarksburg, USA
Discovered 1 weeks agoFull-time
Senior Systems Administrator (On-Site in Clarksburg, WV)
Clarksburg, USA
Discovered 1 weeks agoFull-time