Senior Data Engineer - Databricks

Ability to hold a position of public trust with the US government.

5-7 years industry experience coding commercial software and a passion for solving complex problems.

5-7 years direct experience in Data Engineering with experience in tools such as:

Big data tools: Databricks, Apache Spark, Delta Lake, etc.

Relational SQL (Preferably T-SQL. Alternatively pgSQL , MySQL).

Data pipeline and workflow management tools: Databricks Workflows, Airflow, Step Functions, etc.

AWS cloud services: Databricks on AWS, S3, EC2, RDS (or Azure equivalents).

Object-oriented/object function scripting languages: PySpark /Python, Java, C++, Scala, etc.

Experience working with Data Lakehouse architecture and Delta Lake/Apache Iceberg

Advanced working SQL knowledge and experience working with relational databases, query authoring and optimization (SQL) as well as working familiarity with a variety of databases.

Experience manipulating, processing, and extracting value from large, disconnected datasets.

Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.

Experience manipulating structured and unstructured data.

Experience architecting data systems (transactional and warehouses).

Experience the SDLC, CI/CD, and operating in dev/test/prod environments.

Experience with data cataloging tools such as Informatica EDC, Unity Catalog, Collibra, Alation, Purview, or DataZone is a plus.

Commitment to data governance.

Experience working in an Agile environment.

Experience supporting project teams of developers and data scientists who build web-based interfaces, dashboards, reports, and analytics/machine learning models