Lead Data Engineer / Data Platform Lead

Job summary

Design, build, and maintain the EMFI Data Application Platform, delivering secure and reliable data products and scalable backend microservices. Develop Snowflake-native processing applications and APIs, optimize data queries, and own features from requirements and design through testing and deployment.

Job details

Job Overview The primary focus of this role will be to design, build, and maintain the EMFI Data Application Platform, a financial analytics application. The incumbent will work across the full stack to deliver reliable, performant, and secure data products. Reports To Director Data and AI, Embedded Finance Responsibilities and Duties • Collaborate with internal stakeholders to translate business and analytical requirements into well-defined backend features and data products. • Design and deploy scalable microservices within the FastAPI ecosystem, establishing clear service boundaries for domain-specific services and ensuring loose coupling through well-defined API contracts. • Design and develop Snowflake-native applications including Snowpark (Python), stored procedures, tasks, streams, and dynamic tables to support day-end processing engines that run natively within the Snowflake compute layer. • Design and implement RESTful API endpoints using FastAPI ensuring data contracts are clearly defined, versioned, and documented via OpenAPI/Swagger. • Write and optimize SQL queries against Snowflake and PostgreSQL for reporting, aggregation, and analytical use cases. • Own end-to-end feature delivery: from requirements gathering and API contract definition through implementation, testing, and deployment. • Ensure platform security by applying best practices around authentication, authorization (RBAC), input validation, and safe error handling, aligned with OWASP Top 10. Technical Skills • Experience designing and implementing microservices architectures, including service discovery, inter-service communication, API versioning, and distributed tracing/logging for observability. • Proficiency in Python for backend development, including FastAPI async patterns, and Pydantic-based data validation. • Hands-on experience with SQL and analytical databases for writing optimized queries, aggregations, and data transformations. • Experience developing Snowflake-native applications: stored procedures (JavaScript/Python), Tasks, Streams, Dynamic Tables, and Snowpark for building backend processing engines. • Understanding of REST API design, OpenAPI specifications, and API integration patterns in frontend applications. • Knowledge of authentication and authorization patterns including JWT, OAuth2, and role-based access control. • Experience with Docker and Docker Compose for local development and containerized deployments. • Familiarity with CI/CD pipelines for automated testing and deployment. • Experience with version control using Git, including branching strategies and code review workflows. • Ability to work with cloud platforms (AWS, Azure, or GCP) for data storage, compute, and deployment. Qualifications and Skills • Bachelor’s degree (Master’s preferred) in Computer Science, Software Engineering, Information Systems, or a related discipline. • 10+ years of professional full stack development experience, with demonstrated delivery of production-grade web applications. • Experience in the Payments or Fintech industry would be a strong asset. • Proven ability to work independently, manage multiple concurrent workstreams, and deliver with minimal oversight. • Excellent communication and interpersonal skills, with the ability to engage both technical and non-technical audiences. • Unquestionable personal and business ethics and integrity.

What you’ll do

Design, build, and maintain the EMFI Data Application Platform, delivering secure and reliable data products and scalable backend microservices. Develop Snowflake-native processing applications and APIs, optimize data queries, and own features from requirements and design through testing and deployment.

Requirements

Requires 10+ years of full-stack development experience delivering production-grade web applications, with strong Python, FastAPI, SQL, and Snowflake expertise. A bachelor’s degree in a related discipline is required or preferred, and fintech or payments experience is an asset; candidates should also be independent, communicative, and ethically grounded.

Listed skills

• Python · Preferred
• SQL · Preferred
• PostgreSQL · Preferred
• Docker · Preferred
• CI/CD · Preferred
• Git · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• FastAPI
• Microservices Architecture
• Snowflake
• Snowpark
• SQL
• PostgreSQL
• REST API Design
• OpenAPI
• Pydantic
• Authentication and Authorization
• Role-Based Access Control
• Docker
• CI/CD
• Git
• Cloud Platforms

Job areas

• Technology
• Software
• Data & Analytics
• Finance & Accounting
• Management & Leadership

More jobs from Apexon