Job Overview We are seeking a Big Data Infrastructure Engineer with a strong AI-first and automation-driven mindset to join our infrastructure team and support the administration and operation of enterprise Big Data platforms and their underlying infrastructure.
The ideal candidate should have strong hands-on experience in Linux operating system administration and database administration, with particular focus on configuration, troubleshooting, performance tuning, optimization, monitoring, and production support.
The candidate is expected to actively leverage AI-assisted tools and automation to improve troubleshooting, operational efficiency, scripting, documentation, research, and day-to-day engineering activities, while maintaining the technical judgment required to validate solutions before implementation.
The role also requires practical knowledge of Docker and Kubernetes, along with a good understanding of clustering, high availability, distributed systems, and Big Data concepts.
A strong willingness to continuously learn and stay current with emerging AI, automation, infrastructure, and Big Data technologies is essential.
Duties And Responsibilities
Leverage AI-assisted tools to improve troubleshooting, log analysis, scripting, documentation, research, and operational efficiency while validating outputs before implementation.
Identify repetitive operational activities and develop automation solutions using Shell/Bash, Python, Ansible, APIs, or other appropriate technologies.
Continuously evaluate emerging AI and automation capabilities and identify practical opportunities to improve infrastructure operations and engineering workflows.
Administer, configure, manage, troubleshoot, and optimize Linux operating systems supporting production and non-production environments.
Monitor and analyze CPU, memory, disk, filesystem, network, processes, and system services, and perform configuration and performance tuning when required.
Administer and manage PostgreSQL, MySQL/MariaDB, and Redis, including configuration, access management, backup and recovery, monitoring, troubleshooting, maintenance, and performance optimization.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
Support database replication, high availability, backup/recovery, and capacity management requirements.
Support and maintain Docker and Kubernetes environments, including deployment, configuration, monitoring, troubleshooting, scaling, and cluster administration.
Support clustered and distributed platforms with focus on high availability, replication, failover, load balancing, quorum, capacity management, and disaster recovery.
Support the installation, configuration, monitoring, administration, and upgrade of Cloudera/Hortonworks and Hadoop-based environments.
Support and troubleshoot Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, HBase, and Kafka, as well as related platforms such as Airflow, Superset, and Trino/Presto where applicable.
Perform production monitoring and support using tools such as Zabbix and Grafana, and participate in incident management, root cause analysis, and corrective/preventive actions.
Support security integrations and technologies such as Ranger, LDAP, and Kerberos.
Collaborate with development, infrastructure, and other technical teams on deployments, upgrades, infrastructure changes, troubleshooting, and production support.
Maintain technical documentation, operational procedures, automation, and infrastructure configuration records. Skills and Qualifications
3-5 years of relevant hands-on experience in Linux/System Administration, Database Administration, Big Data Infrastructure, DevOps, or a related infrastructure role.
Bachelor's Degree in Computer Science, Computer Engineering, Information Technology, or a related field.
Strong AI-first and automation-driven mindset, with demonstrated ability to use AI-assisted tools effectively in technical workflows and critically validate generated recommendations before applying them. Big Data Infrastructure Engineer - Job Description Good scripting and automation skills using Shell/Bash; knowledge of Python, Ansible, APIs, or similar technologies is highly desirable.
Strong hands-on knowledge of Linux administration, including system configuration, service management, resource management, storage/filesystems, permissions, networking, troubleshooting, and performance optimization.
Good hands-on knowledge of PostgreSQL, MySQL/MariaDB, and Redis administration, including configuration, backup and recovery, users and privileges, monitoring, maintenance, and performance tuning.
Good understanding of database concepts including connections, transactions, locks, indexing, query performance, replication, and high availability.
Good hands-on understanding of Docker and Kubernetes, including containers, images, pods, deployments, services, storage, networking, monitoring, resource management, and troubleshooting.
Good understanding of clustering and distributed system concepts, including high availability, replication, failover, load balancing, and quorum.
Good understanding of networking fundamentals, including TCP/IP, DNS, ports, routing, connectivity, and network troubleshooting.
Good understanding of Big Data concepts and the Hadoop ecosystem, with familiarity or hands-on experience in HDFS, YARN, Hive, Spark, Kafka, and HBase.
Familiarity with Cloudera or Hortonworks platforms is highly desirable.
Familiarity with Zabbix/Grafana, Ranger/LDAP/Kerberos, CI/CD tools, and Trino/Presto is an advantage.
Strong troubleshooting, analytical, and problem-solving skills, with the ability to investigate issues systematically and identify root causes.
Ability to work effectively in production environments, collaborate across technical teams, take ownership of assigned activities, and continuously develop technical knowledge. Preferred Certifications
Relevant Linux certifications such as RHCSA or RHCE.
Kubernetes certification such as CKA.
PostgreSQL or MySQL-related certifications/training.
Red Hat Ansible or other relevant automation certifications. Note: Certifications are considered an advantage and are not a substitute for practical hands-on experience.
About LigaData
Software & SaaS200 employeesFounded 2014
Enterprise data and AI software company serving communications providers and enterprises with governed platforms and managed services.
