Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
AIQ is seeking a Senior Data Engineer to build and maintain data infrastructure for AI solutions in the energy sector. You will design scalable data pipelines, manage data storage, and collaborate with data scientists and ML engineers.
Key Skills for This Role
Responsibilities
- Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics
- Work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable
- Develop and maintain APIs for data access and manipulation
- Manage and optimize data storage solutions for both structured and unstructured data
- Implement processes to monitor data quality and ensure production data is accurate and available
- Collaborate with ML engineers to assist in data related technical issues and provide architectural guidance
- Ensure compliance with data security and privacy policies
- Maintain clear and up to date documentation including data dictionaries, metadata, and architectural diagrams
Requirements
- Bachelor's degree in Computer Science, Engineering, Mathematics, or related field
- 7+ years of experience in a Data Engineering role
- Proficiency in programming languages like Python, Java, and Scala
- Hands on experience with big data technologies like Hadoop, Spark, and Flink
- Experience with cloud platforms like Azure
- Strong understanding of data warehousing concepts, ETL processes, and data modeling
- Experience with API development and integration with data services
- Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes
- Experience with real time / streaming data processing
Full Job Posting
About The Company
- AIQ is an Abu Dhabi based technology company that develops and deploys industrial artificial intelligence (AI) technologies at scale, focused on the energy sector.
- As a venture between Presight (G42) and ADNOC, AIQ has productized 15 AI enabled solutions.
- The organization embraces an innovative and entrepreneurial spirit.
About The Role
- As a Senior Data Engineer at AIQ, you will be responsible for building and maintaining the infrastructure that supports data collection, processing, and storage.
- You will work closely with data scientists, analysts, and other stakeholders to ensure data systems are reliable, scalable, and secure.
Responsibilities
- Data Pipeline Architecture and Development: Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.
- Data Integration: Work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable.
- API and Data Services: Develop and maintain APIs for data access and manipulation, and integrate with external data services.
- Data Storage: Manage and optimize data storage solutions for both structured and unstructured data.
- Data Quality and Governance: Implement processes to monitor data quality and ensure production data is always accurate and available.
- Collaboration and Support: Collaborate with ML engineers to assist in data related technical issues and provide architectural guidance.
- Security and Compliance: Ensure compliance with data security and privacy policies.
- Documentation: Maintain clear and up to date documentation including data dictionaries, metadata, and architectural diagrams.
Qualifications
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.
- 7+ years of experience in a Data Engineering role.
- Programming languages like Python, Java, and Scala and managing huge scale data potentially Terabyte to Petabyte.
- Hands on experience with big data technologies like Hadoop, Spark and Flink.
- Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar.
- Strong understanding of data warehousing concepts, ETL processes, and data modeling.
- Experience with API development and integration with data services.
- Experience with cloud platforms like Azure.
- Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.
- Experience with real time / streaming data processing.
Technical Stack
- Programming Languages: Python, Java, Scala, SQL, Bash
- Big Data Technologies: Hadoop, Spark, Flink
- Unstructured Data: Text, Image, Audio & Video
- Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis
- Cloud Platforms: Azure
- API Development: RESTful APIs, GraphQL, OpenAPI
- Data Services: Kafka, RabbitMQ
- Containers: Docker, Kubernetes
What Working At AIQ Offers
- Culture: Encouraging initiative, work in a fast paced and varied environment with talented peers.
- Career: Join a team where contribution is recognized and rewarded.
- Rewards: Attractive remuneration package including healthcare, education support for dependents, leave benefits, and more.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at AIQ
Senior Developer - Full Stack
Abu Dhabi, UAE
AIQ is seeking a Senior Full Stack Developer to build next-generation cloud-native applications for AI-driven oil and gas operations. You will design scalable full-stack systems, integrate AI/ML services, and collaborate
Product Manager (Subsurface Domain)
Abu Dhabi, UAE
AIQ is seeking a Product Manager (Subsurface Domain) to lead AI-driven product development for upstream oil and gas operations. The role requires deep subsurface domain expertise, experience with reservoir engineering wo
Product Manager (Subsurface Domain)
Abu Dhabi, UAE
AIQ is an Abu Dhabi-based technology company that develops and deploys industrial artificial intelligence (AI) technologies at scale, focused on the energy sector. As a venture between Presight (G42) and ADNOC, AIQ has p
Accounting Specialist
Abu Dhabi, UAE
AIQ is an Abu Dhabi-based technology company that develops and deploys industrial artificial intelligence (AI) technologies at scale, focused on the energy sector. As a venture between Presight (G42) and ADNOC, AIQ has p
Product Manager (Drilling Domain)
Abu Dhabi, UAE
AIQ is an Abu Dhabi-based technology company that develops and deploys industrial artificial intelligence (AI) technologies at scale, focused on the energy sector. As a venture between Presight (G42) and ADNOC, AIQ has p
Manager - Technical Project
Abu Dhabi, UAE
AIQ is an Abu Dhabi-based technology company that develops and deploys industrial artificial intelligence (AI) technologies at scale, focused on the energy sector. As a venture between Presight (G42) and ADNOC, AIQ has p
Senior Solution Architect
Abu Dhabi, UAE
AIQ is an Abu Dhabi-based technology company that develops and deploys industrial artificial intelligence (AI) technologies at scale, focused on the energy sector. As a venture between Presight (G42) and ADNOC, AIQ has p
Senior Developer - Full Stack
Abu Dhabi, UAE
Product Manager (Subsurface Domain)
Abu Dhabi, UAE
Product Manager (Subsurface Domain)
Abu Dhabi, UAE
Accounting Specialist
Abu Dhabi, UAE
Product Manager (Drilling Domain)
Abu Dhabi, UAE
Manager - Technical Project
Abu Dhabi, UAE
Senior Solution Architect
Abu Dhabi, UAE