Data Engineer

Role Summary

Senior Data Engineer

Intelligent Supply Chain

Leveredge

Key Responsibilities

• **Data Lakehouse Architecture (Supply Chain)**
• **ERP Integration:**
• Architect and deploy ingestion pipelines to extract high-volume transactional data from SAP S/4HANA, Ariba, and PLM systems, ensuring near real-time availability for forecasting models.
• **External Feed Integration:**
• Build connectors for external market intelligence feeds (e.g., S&P Global, Orbis, EcoVadis) to enrich internal procurement data with macroeconomic and geopolitical signals.
• **Unified Data Model:**
• Design and implement a standardized procurement data model and taxonomy across multiple entities, harmonizing fragmented datasets into a cohesive layer for analytics.
• **Unstructured Data Pipelines (LeverEDGE)**
• **Complex Ingestion:**
• Engineer pipelines to ingest and process unstructured technical data, including PDF tender documents, CAD metadata, and historical CONOPS, transforming them into vector-ready formats for RAG (Retrieval-Augmented Generation) applications.
• **Vector Database Management:**
• Manage and optimize Vector Databases (e.g., Weaviate) to store embeddings of archival proposals and engineering snippets, ensuring high-speed retrieval for AI drafting assistants.
• **Digital Thread Implementation:**
• Establish data lineage and traceability protocols that link requirements to physical components, supporting the Model-Based Systems Engineering (MBSE) "Digital Thread".
• **Governance & Security**
• **Defense-Grade Security:**
• Implement Role-Based Access Control (RBAC), audit logging, and data redaction policies to ensure compliance with export controls and strict on-premise security requirements.
• **Quality Control:**
• Deploy automated data quality frameworks to validate BOM (Bill of Materials) completeness and cost data accuracy before it reaches AI models.
• **Infrastructure Optimization:**
• Optimize pipelines for on-premise GPU clusters and air-gapped environments, ensuring efficiency within existing infrastructure limits.

Technical Requirements

• **Core Stack:**
• Expert proficiency in Python, SQL, and modern data engineering frameworks (Apache Spark, Kafka, Airflow).
• **Enterprise ERP:**
• Strong experience extracting data from complex ERP environments, specifically SAP S/4HANA and SAP Ariba.
• Familiarity with SAP BTP is a plus.
• **Database Technologies:**
• Deep understanding of Data Lakehouse architectures (Databricks/Delta Lake), Relational Databases (PostgreSQL), and Vector Databases (Weaviate/Milvus).
• **Data Pipeline Development:**
• Experience building pipelines for RAG solutions, Conversational agents and classical ML models with tools like dbt, dagster, or prefect
• **DevOps/DataOps:**
• Proficiency with containerization (Docker, Kubernetes) and CI/CD pipelines for deploying data workflows in secure environments.

Professional Qualifications

• **Experience:**
• 5+ years of experience in Data Engineering, with at least 2 years focused on building pipelines for Machine Learning or Generative AI applications in an enterprise setting.
• **Domain Knowledge:**
• Experience in Supply Chain, Manufacturing, or Defense sectors is highly desirable.
• Ability to understand "Bill of Materials" (BOM) structures and procurement lifecycles.
• **Problem Solving:**
• Ability to navigate the "Governance Collision" between agile data work and rigid systems engineering requirements, ensuring data deliverables meet formal Stage Gate reviews.
• **Collaboration:**
• Proven ability to work alongside Data Scientists and Backend Engineers to define data schemas that support predictive modeling and AI agents.
• Why This Role?
• You will be the architect of the data foundation that secures national defense capabilities.
• Your work will directly enable AI agents to negotiate procurement contracts and assist engineers in designing next-generation systems.
• If you are ready to build the robust infrastructure that turns raw data into strategic advantage, join AI71.