Data Engineer - Mid Level

• 4-10 years of experience in Data Engineering, Backend Engineering, and

DevOps.

• Designs, implements, and maintains large-scale ETL pipelines and ingestion

architecture for collecting, retrieving, normalizing, validating, and processing

intelligence data.

• Builds and maintains distributed, event-driven architecture using cloud-native

services such as AWS Lambda, EventBridge, SQS, RDS/PostgreSQL, and

serverless computing platforms.

• Develops backend services, APIs, and worker-based system for parsing,

transforming, enriching, and publishing structured data.

• Drafts database schemas and indexes, and manages data integrity and

performance of PostgreSQL database environment.

• Creates automated validation, deduplication, and quality assurance

processes to ensure accuracy, consistency, and reliability of ingested data.

• Implements monitoring, alerting, and logging mechanisms to detect pipeline

failure and processing issues in a distributed environment.

• Participates in collaboration with engineers, analytics, and mission

stakeholders to determine data processing requirements

• Investigates, analyzes, and implements new and innovative technologies to

enhance the scalability and automation of cloud-native data platforms.

• Implements best practices of software development and cloud infrastructure

in accordance with mission and security (RMF) guidelines.

• Supports CI/CD (GitLab CI/CD, or similar) pipelines, automation of

infrastructure (Terraform, Terragrunt), and deployment processes for cloud-

native application and serverless services.

• Documents and creates technical architecture and operational procedures.