Jobs.ca
Jobs.ca
Language
Atlantis IT Group logo

Lead Data Engineer (.net Python/PySpark & DevOps)

Toronto, ON
Senior Level

About the role

Role - Lead Data Engineer (.net Python/PySpark & DevOps)

Location - Toronto ( Hybrid )

Essential Skills: Job DescriptionDevelop and maintain backend systems using Python, PySpark ensuring high performance, scalability, and reliabilityParticipate in the design and implementation of data engineering and ETL pipelines using PySpark Collaborate with cross-functional teams to identify and prioritize project requirements Mentor and guide engineers, providing technical guidance and code reviewsStay up-to-date with the latest technologies and frameworks, and apply this knowledge to improve existing systems and processesLead the technical direction of the team, including architecture, design, and implementation of software systemsCollaborate with product owners to define and prioritize requirementsDevelop and maintain technical documentation, including architecture diagrams, and technical specificationsParticipate in code reviews, ensuring high-quality code and adherence to coding standardsCollaborate with DevOps and Operations teams to ensure smooth deployment and operation of applicationsExpert in DevOps practices and tools for CICD pipelines.Willingness to learn new technologies and adapt to new challengesProven experience in leading technical teams, including technical guidance, mentoring, and coachingStrong technical leadership skills, including the ability to make technical decisions, prioritize tasks, and manage technical resources Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teamsAbility to drive technical innovation, including researching new technologies, evaluating technical options, and recommending technical solutionsStrong problem-solving skills, with the ability to debug complex issues, optimize system performance, and ensure high-quality software deliveryExperience with Agile development methodologies, including Scrum or Kanban, and the ability to apply these principles to lead the teamExperience with cloud platforms, including Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP)Experience with PySpark for Data Engineering ETL pipelinesFamiliarity with containerization using Docker and orchestration using Kubernetesy
Desirable Skills:

8+ years of strong design, development experience using c#, .net core, Web API, angular/react, asp.net, mvc

3+ years of strong development experience using sqlserver/oracle/sybase, complex procedures and SSIS/SSRS

Working experience in financial domain

Experience working with devops, source code management tools like git, svn, jira, ucd, jenkins.

Experience working with Agile/Scrum/RAD development methodology

Nice to have container based technologies like docker, kubernetes, openshift ,cloud platform and familiar with big-data and cloud based technologies

Nice to have experience in setting application in Linux/Windows, IIS/nginx

Nice to have working experience with APAC Regulatory Reporting

Strong technical analytical skills to solve complex problems

Strategic thinker with excellent interpersonal skills to work across functions and businesses

About Atlantis IT Group

IT Services and IT Consulting
51-200

"Atlantis IT Group" is an IT venture that offers end-to-end services and products. The focus of "the Company" is to cater one-stop solutions to small and medium-sized businesses.

"Atlantis" had its first branch in Toronto, Canada, and was established in 1990. The Company has emerged from a small farm to a medium company providing all sorts of solutions, headquartered in Toronto, Canada The Company has had its branches and offices in different countries such as the USA, Canada, and India for the last 32 years.

Atlantis is enlarging its global footprint by offering complementary product and service lines by partnering and strategically working with businesses. The primary specialized area of the Company includes ERP solutions, with cloud-based and on-premise solutions available to meet the requirements of all Industries.

OUR PRODUCTS:

"WORKSWARE ERP" The emergence of WORKSWARE ERP is noticed from a ground level to an upgraded form because of the integrated ERP system, dedicated towards forward-thinking discrete manufacturing enterprises.

"BM-Cloud" BM-Cloud is One of the key offerings of Atlantis consulting as a Cloud-based business management ERP software since 2013. The significant feature of BM-Cloud includes easy usage with the enhancement of simplicity and usability as a complete integrated ERP system. The fundamental modules covered by BM-Cloud are accounting, inventory management, purchasing, sales, and HR/payroll systems. In other words, it is designed primarily to support businesses in the trading and retail sectors and includes all the essential components required for a firm to assist in decision-making.

"BM-POS"

We provide a cloud-based retail solution with all the tools required to manage a retail business, including a sales screen, inventory management, real-time dashboard, employee administration, and other functions that keep retailers in charge of their operations.