This is a remote position. We are seeking a hands-on Databricks Platform Lead on a part-time, remote basis. We are not looking for a high-level architect; we need a brilliant, senior engineer who can take operational ownership of our enterprise environment while actively building and optimizing our data pipelines. You will serve as the foundational steward of our Databricks ecosystem—establishing Unity Catalog governance, managing workspace security, and enforcing cost-control configurations—while simultaneously delivering clean code for our lakehouse analytics. Key Information Commitment: Part-time (Hours to be discussed, flexible) Location: 100% Remote (Must overlap with Eastern Time Zone / EST business hours) Role Type: Technical Platform Lead — 100% Hands-on execution Key Responsibilities

1. Platform Administration & Governance Environment & Security: Provision Databricks workspaces, cloud storage integrations, and RBAC frameworks. Data Governance: Lead the implementation of Unity Catalog for centralized data lineage and access controls. Cost & Performance Optimization: Enforce compute policies, cluster sizes, and auto-scaling rules to minimize cloud spend. Diagnose bottlenecks and establish workspace best practices (e.g., Liquid Clustering).
2. Hands-on Data Engineering & Automation Pipeline Development: Design and maintain scalable data pipelines using Spark (PySpark/Scala) and Delta Lake (Medallion architecture). CI/CD & Automation: Automate infrastructure and code deployment using Terraform, Databricks Asset Bundles (DABs), and Git/CI-CD pipelines. Modern Features: Leverage cutting-edge features including Delta Live Tables (DLT), Serverless compute, and Photon engine.
3. Collaboration & Enablement Support data engineers, analysts, and data scientists by safely enabling workflows like MLflow, dbt, and Databricks SQL.

Requirements

Required Technical Skills & Qualifications Experience Tier: 7+ years of data engineering experience utilizing Apache Spark (PySpark/Scala) and SQL. Databricks Expertise: Strong history of managing enterprise-scale Databricks infrastructure and implementing latest updates. Data Governance: Practical experience configuring Unity Catalog and Delta Lake storage optimization from scratch. Infrastructure as Code (IaC): Proficient with Terraform or Databricks Asset Bundles (DABs). Cloud & Networking: Experience securing Databricks environments (VPCs, Private Link, IAM/Active Directory) across AWS, Azure, or GCP. Execution: Self-driven and capable of working autonomously in a remote, part-time capacity. Nice-to-Haves Databricks Certified Platform Administrator or Data Engineer Professional. Experience with Databricks Mosaic AI / MLflow or dbt Core/Cloud integration.