TPM- Data Engineering

Job summary

Lead the end-to-end execution of large-scale data and AI transformation programs using Databricks Lakehouse architecture. Coordinate cross-functional teams to design and implement scalable data platforms while aligning technical execution with business requirements.

Job details

Technical Program Manager (TPM) – Databricks Platform Role Overview We are seeking an experienced Technical Program Manager (TPM) with strong expertise in Databricks and modern data platforms to lead and drive large-scale data & AI transformation programs. This role will work at the intersection of business, data engineering, and platform architecture, ensuring successful delivery of enterprise-grade data solutions on Databricks. Key Responsibilities Program Leadership Drive end-to-end execution of data platform programs built on Databricks (Lakehouse architecture). Manage multiple cross-functional teams including data engineering, data science, DevOps, and BI teams. Own program roadmap, delivery timelines, risk management, and stakeholder communication. Establish governance frameworks for program delivery, quality, and performance tracking. Databricks Platform Ownership Lead design and implementation of Databricks-based data platforms (Azure Databricks / AWS Databricks / GCP). Ensure best practices in: Delta Lake Data pipelines (batch + streaming) Medallion architecture (Bronze, Silver, Gold layers) Drive platform optimization including cost, performance, and scalability. Technical Strategy & Execution Collaborate with architects to define data architecture standards and patterns. Oversee development of: ETL/ELT pipelines Data ingestion frameworks Data transformation and curation layers Ensure integration with ecosystem tools like: Azure Data Factory / Airflow Power BI / Tableau MLflow, Feature Store Stakeholder & Business Alignment Act as a bridge between business leaders and engineering teams. Translate business requirements into technical execution plans. Provide leadership updates and executive reporting on program progress. Delivery Excellence Define and track KPIs, SLAs, and program success metrics. Identify and mitigate risks, dependencies, and bottlenecks. Ensure adherence to Agile / Scrum / SAFe methodologies. Data Governance & Security Ensure compliance with: Data governance policies Security and privacy regulations Implement best practices for: Data lineage Data quality Access controls Skills & Qualifications Mandatory Skills 10+ years in technology roles, with 5+ years in program management. Strong hands-on or working knowledge of: Databricks (must-have) Apache Spark / PySpark / SQL Delta Lake architecture Experience in cloud platforms: Azure (preferred), AWS, or GCP Proven experience delivering enterprise data platform programs. Technical Expertise Data Engineering concepts: Data modeling, ETL/ELT, pipelines Real-time streaming (Kafka / Event Hub) CI/CD for data pipelines DataOps / MLOps frameworks Program Management Skills Strong experience in: Agile / Scrum / SAFe frameworks Program governance and reporting Excellent stakeholder management skills, including C-level engagement.

What you’ll do

Lead the end-to-end execution of large-scale data and AI transformation programs using Databricks Lakehouse architecture. Coordinate cross-functional teams to design and implement scalable data platforms while aligning technical execution with business requirements.

Requirements

Requires over 10 years of technology experience with at least 5 years in program management and deep expertise in Databricks and Spark. Candidates must be proficient in cloud platforms and Agile methodologies to deliver enterprise-grade data solutions.

Listed skills

• Execution · Preferred
• Technical · Preferred
• Teams · Preferred
• management · Preferred
• Agile · Preferred
• Microsoft Azure · Preferred
• Amazon Web Services · Preferred
• Google Cloud · Preferred
• Leadership · Preferred
• Reporting · Preferred
• Scrum · Preferred
• batch · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks
• Apache Spark
• PySpark
• SQL
• Delta Lake
• Azure
• AWS
• GCP
• Program Management
• Agile
• Scrum
• SAFe
• Data Engineering
• ETL/ELT
• Stakeholder Management
• Data Governance

Job areas

• Data & Analytics
• Technology
• Management & Leadership
• Software
• Engineering

More jobs from iLink Digital