Data Engineer

Role Summary

Senior Data Engineer

Intelligent Supply Chain

Leveredge

• Data Lakehouse Architecture (Supply Chain)

• ERP Integration: Architect and deploy ingestion pipelines to extract high-volume transactional data from SAP S/4HANA, Ariba, and PLM systems, ensuring near real-time availability for forecasting models.
• External Feed Integration: Build connectors for external market intelligence feeds (e.g., S&P Global, Orbis, EcoVadis) to enrich internal procurement data with macroeconomic and geopolitical signals.
• Unified Data Model: Design and implement a standardized procurement data model and taxonomy across multiple entities, harmonizing fragmented datasets into a cohesive layer for analytics.

• Unstructured Data Pipelines (LeverEDGE)

• Complex Ingestion: Engineer pipelines to ingest and process unstructured technical data, including PDF tender documents, CAD metadata, and historical CONOPS, transforming them into vector-ready formats for RAG (Retrieval-Augmented Generation) applications.
• Vector Database Management: Manage and optimize Vector Databases (e.g., Weaviate) to store embeddings of archival proposals and engineering snippets, ensuring high-speed retrieval for AI drafting assistants.
• Digital Thread Implementation: Establish data lineage and traceability protocols that link requirements to physical components, supporting the Model-Based Systems Engineering (MBSE) "Digital Thread".

• Governance & Security

• Defense-Grade Security: Implement Role-Based Access Control (RBAC), audit logging, and data redaction policies to ensure compliance with export controls and strict on-premise security requirements.
• Quality Control: Deploy automated data quality frameworks to validate BOM (Bill of Materials) completeness and cost data accuracy before it reaches AI models.
• Infrastructure Optimization: Optimize pipelines for on-premise GPU clusters and air-gapped environments, ensuring efficiency within existing infrastructure limits.

Technical Requirements

• Core Stack: Expert proficiency in Python, SQL, and modern data engineering frameworks (Apache Spark, Kafka, Airflow).
• Enterprise ERP: Strong experience extracting data from complex ERP environments, specifically SAP S/4HANA and SAP Ariba. Familiarity with SAP BTP is a plus.
• Database Technologies: Deep understanding of Data Lakehouse architectures (Databricks/Delta Lake), Relational Databases (PostgreSQL), and Vector Databases (Weaviate/Milvus).
• Data Pipeline Development: Experience building pipelines for RAG solutions, Conversational agents and classical ML models with tools like dbt, dagster, or prefect
• DevOps/DataOps: Proficiency with containerization (Docker, Kubernetes) and CI/CD pipelines for deploying data workflows in secure environments.

Professional Qualifications

• Experience: 5+ years of experience in Data Engineering, with at least 2 years focused on building pipelines for Machine Learning or Generative AI applications in an enterprise setting.
• Domain Knowledge: Experience in Supply Chain, Manufacturing, or Defense sectors is highly desirable. Ability to understand "Bill of Materials" (BOM) structures and procurement lifecycles.
• Problem Solving: Ability to navigate the "Governance Collision" between agile data work and rigid systems engineering requirements, ensuring data deliverables meet formal Stage Gate reviews.
• Collaboration: Proven ability to work alongside Data Scientists and Backend Engineers to define data schemas that support predictive modeling and AI agents.
• Why This Role?
• You will be the architect of the data foundation that secures national defense capabilities.
• Your work will directly enable AI agents to negotiate procurement contracts and assist engineers in designing next-generation systems.
• If you are ready to build the robust infrastructure that turns raw data into strategic advantage, join AI71.

Job Details

About The Company

Report