Application Developer- Airflow / Astronomer Migration

Design the migration approach from ADF to Astronomer / Airflow .

Convert existing orchestration logic into clean Airflow DAG design.

Define DAG structure, task dependencies, retry logic, and schedule patterns.

Design workflows that are easy to maintain and support.

Suggest improvements instead of only doing one-to-one migration.

Design and build DAGs in Apache Airflow / Astronomer .

Develop and adapt data pipelines as per migration requirement.

Modify existing ingestion steps in Astronomer where required.

Modify existing dbt flows and transformations when needed.

Use Python and SQL for workflow logic, validation, and automation.

Follow coding standards and project guidelines.

Configure schedules and dependencies for Airflow DAGs.

Integrate Airflow workflows with dbt , Snowflake , and Azure Blob Storage .

Ensure dbt jobs are properly triggered and monitored through Airflow.

Validate integration between ingestion, transformation, and consumption layers.

Configure workflow parameters, environment settings, and required connections.

Perform unit testing for DAGs and workflow components.

Perform integration testing for end-to-end data workflows.

Validate migrated workflows against existing ADF output wherever applicable.

Check data accuracy, completeness, and processing status.

Fix defects and retest before production readiness.

Prepare test evidence and validation notes.

Review existing workflows and identify improvement areas.

Optimize Airflow DAG performance and execution time.

Improve reliability using proper retries, failure handling, and dependency management.

Suggest cost-efficient execution patterns.

Improve maintainability by creating reusable workflow components.

Proactively recommend better design wherever useful.

Follow client security and data handling guidelines.

Use secure connection and access patterns as per project standards.

Avoid hardcoding secrets, passwords, or sensitive values.

Ensure access and workflow configurations are controlled and traceable.

Follow required governance process for data pipeline changes.

Support deployment of Airflow / Astronomer workflows across environments.

Prepare deployment steps and release notes.

Support production readiness checks before go-live.

Coordinate with the Data Engineering team during release activities.

Validate workflows after deployment.

Support rollback or quick fix approach if any deployment issue occurs.

Monitor workflow execution and identify failures.

Debug failures using Airflow logs, task history, dbt logs, and Snowflake queries.

Perform root cause analysis for recurring issues.

Implement preventive fixes to improve workflow stability.

Support production issues during migration and stabilization phase.

Prepare technical documentation for developed DAGs and workflows.

Document schedule, dependency, parameter, and configuration details.

Prepare development and testing documentation.

Maintain handover notes and support instructions.

Provide knowledge transfer to the Data Engineering team.

Explain design decisions, known issues, and support steps clearly.

Work closely with the Principal Data Engineer and Data Engineering team.

Provide regular status updates on progress, risks, and blockers.

Collaborate with team members in a structured and transparent way.

Take ownership of assigned tasks and deliver on time.

Work as a hands-on team reinforcement for the migration project.

Cloud / Platform: Azure

Orchestration Tools: Apache Airflow, Astronomer, Azure Data Factory

Data Transformation: dbt

Storage: Azure Blob Storage

Database / Warehouse: Snowflake

Programming Languages: Python, SQL

DevOps Tools: Git / Azure DevOps, if used in project

Monitoring Tools: Airflow UI, Astronomer monitoring, dbt logs, Snowflake query history

Documentation Tools: Confluence / SharePoint / Project documentation repository, as applicable

3 to 5 years of Data Engineering experience.

Strong hands-on experience in Airflow / Astronomer .

Good working knowledge of dbt, Snowflake, Azure Blob Storage, Python, and SQL .

Prior experience in orchestration migration or data pipeline modernization.

Ability to understand ADF workflows and redesign them in Airflow.

Good delivery ownership and clear communication.

Able to work independently with minimum supervision.

At Fujitsu, we are committed to an inclusive recruitment process that values the diverse backgrounds and experiences of all applicants. We believe that hiring people from a wide variety of backgrounds makes us stronger, not because it's the right thing to do, but because it allows us to draw on a wider range of perspectives and life experiences.