Build and maintain platform observability through development of KPIs, executive dashboards, and monitoring solutions that provide visibility into system performance, resource consumption, cost trends, and compliance adherence.
Develop and implement scalable user provisioning and de-provisioning processes.
Design and manage RBAC (Role-Based Access Control) frameworks and security groups.
Integrate with enterprise identity management systems (SAML, SCIM, Active Directory).
Manage Unity Catalog implementation and governance structures.
Oversee metastore configuration, catalog hierarchies, and data organization.
Implement data lineage, auditing, and compliance frameworks.
Design and enforce compute policies, cluster configurations, and pool management strategies.
Implement cost optimization frameworks and resource allocation policies.
Monitor and optimize cluster utilization and performance.
Manage serverless SQL warehouses and compute resource governance.
Establish and manage connections to diverse source systems (databases, APIs, file systems).
Configure and maintain storage integrations (S3, ADLS, GCS, external locations) in addition to setting up and data access patterns.
Implement and monitor secure credential management and secrets handling.
and mount points.
Build and configure Databricks workspaces, clusters, notebooks, jobs, and Delta Lake storage, integrating with AWS services such as S3, IAM, and KMS.
Implement and maintain security controls including IAM policies, cluster security configurations, KMS-based encryption, and data masking to protect sensitive data.
Monitor and tune Databricks clusters and job workloads for reliability, cost optimization, and performance, leveraging autoscaling and workload management.
Apply data governance best practices including cataloging, metadata management, and data lineage using Databricks Unity Catalog and AWS-native capabilities.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
Collaborate with cloud infrastructure teams to configure supporting AWS components such as S3 storage, networking, logging, monitoring, and access controls.
Maintain detailed technical documentation including solution designs, data flow diagrams, configuration standards, and operational procedures.
Stay up to date with advancements in Databricks, Delta Lake, Spark, and AWS services, integrating new features that improve automation and efficiency.
Partner with data engineers, analysts, and scientists to implement data models and reusable transformation patterns within Databricks and AWS.
Troubleshoot and resolve platform-level issues including but not limited to workspace connectivity, cluster startup failures, authentication problems, and infrastructure performance degradation.
Ensure compliance with data governance, privacy, and security requirements by applying secure architecture patterns and validating controls throughout the data lifecycle.
Support the evaluation and hands-on testing of new AWS and Databricks features or services that enhance the Data Lake environment.
Salary Range: $140,000 - $165,000
General Description of Benefits
Required Skills
Ability to obtain and maintain a DHS Entry on Duty (EOD)/Public Trust
Bachelor's degree +5 years of relevant experience (or 9 years of experience minimum in lieu of degree)
Proven experience in building and configuring enterprise-level data lake solutions using Databricks in an AWS or Azure environment.
In-depth knowledge of Databricks architecture, including workspaces, clusters, storage, notebook development, and automation capabilities.
Deep expertise in Databricks Unity Catalog, workspace management, and security models
Experience with big data technologies such as Apache Spark, Apache Hive, Delta Lake, and Hadoop.
Solid understanding of data governance principles, data modeling, data cataloging, and metadata management.
Hands-on experience with cloud platforms like AWS or Azure, including relevant services like S3, EMR, Glue, Data Factory, etc.
Proficiency in SQL and one or more programming languages (Python, Scala, Bash, or PowerShell).
Knowledge of data security and privacy best practices, including data access controls, encryption, and data masking techniques.
Strong problem-solving and analytical skills, with the ability to identify and resolve complex data-related issues.
Excellent interpersonal and communication skills, with the ability to collaborate effectively with technical and non-technical stakeholders.
Experience in a senior role, providing technical guidance and mentorship to junior team members.
Must hold a certification such as Databricks Certified Developer or Databricks Certified Professional
Desired Skills
Experience in designing and implementing data ingestion pipelines, data transformations, and data quality processes using Databricks.
Experience working with federal government customers and compliance requirements (FedRAMP, FISMA)
Experience with CI/CD practices and version control for infrastructure (Git, GitHub/GitLab/Bitbucket, CI/CD pipelines)
#EverforthECS1
ECS Federal LLC is an equal opportunity employer and does not discriminate or allow discrimination on the basis any characteristic protected by law. All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, or local jurisdiction law.
Everforth ECS is the federal segment of Everforth , a $4B global organization with over 10,000 employees . Our nearly 3,500 professionals deliver advanced technology solutions in data and AI, cybersecurity, and enterprise transformation, serving defense, intelligence, and federal civilian agencies. Our work powers mission-critical outcomes, strengthens technology partnerships, and creates meaningful opportunities for our people. We are defined by a commitment to excellence in delivery, a culture of innovation, and an environment where talent can thrive and grow. We value:
Attracting and developing top talent and high-performing teams
Fostering a culture that is engaging, accountable, and mission-driven
Meet the challenge. Make a difference with Everforth ECS!
About ECS FEDERAL LLC
Verified company details for this employer are not available yet.
