About the role
Required Skills.
- Strong hands-on experience with Data-bricks and Apache Spark (PySpark/Scala).
- Experience in SQL and data transformation techniques.
- Knowledge of ETL tools and data pipeline development.
- Experience working with cloud platforms (Azure/AWS/GCP)
- Strong Azure cloud background
- Understanding of data warehousing concepts.
- Strong problem-solving and analytical skills.
- Hands-on experience with Azure Data bricks or Delta Lake, in building ETL pipelines : batch (autoloader) and Spark structured streaming
- Knowledge of data modelling and performance tuning in Spark.
- Exposure to CI/CD pipelines and DevOps practices.
- Familiarity with data governance and security practices.
- Strong hands on working experience of Unity catalog:
- Hands on exposure to Creating end to end environments : creating catalogs, schemas, tables . materialized views, functions, volumes
- Experience in building SCD 1 and SCD2 (slowly changing dimensions ) on dimension tables
- Experience in building CDC (change data capture pipelines)
- Strong hands on experience with Lakehouse federation , creating foreign catalogs to get data from external sources
- Strong understanding of databricks partitioning , Liquid clustering
Not the right fit? Search for Data Engineer jobs in Toronto, Ontario, Canada
About Kumaran Systems
Kumaran Systems is an IT Services Company with imprints in three countries. The past two decades has seen us provide our global clientele with high-end IT services that include migration support, system integration and infrastructure management solutions, providing one-stop-solution to all your IT demands.
Our expertise and in-depth knowledge of businesses help us cater to a variety of industries. Our team strives to know your industry better, by observing current trends and the way it works, to tailor-make our solutions to your needs.With over 20 years of customer orientation, over 2000 engagements across Telecom, Education and Banking & Financial services spread across the globe, Kumaran stands as a key advisor to some of the largest Fortune 500 companies in their business driven technology enablement drive. Kumaran System's Customer Orientation is driven by a global delivery business model giving its customers to choose between an Onshore-Offshore mix. The delivery models enable multilevel touch points between the client, partner networks and Kumaran thereby enabling business driven customer sensitivity and agility.
ENGAGE, EMERGE AND EXCEL
Involving ourselves in a purposeful action, using our abilities to its maximum, helps us feel positive about ourselves. Such positivity gives raise to positive thinking and positive ideas. Ideas evolve into vision and set our aim! Our aspirations engage us with the incentive of sweet success!
Every endeavor has its own pitfalls. Perseverance helps us emerge a successful entrepreneur after any deterrent. In our attempt to achieve our aim, we keep upgrading our knowledge and skill, to surface successful!
A genius mind does not work to excel, but excels in work! We challenge ourselves with every next step! We change every deterrent to a stepping stone to success.
We essentially, engage, emerge and excel!
Similar Jobs
About the role
Required Skills.
- Strong hands-on experience with Data-bricks and Apache Spark (PySpark/Scala).
- Experience in SQL and data transformation techniques.
- Knowledge of ETL tools and data pipeline development.
- Experience working with cloud platforms (Azure/AWS/GCP)
- Strong Azure cloud background
- Understanding of data warehousing concepts.
- Strong problem-solving and analytical skills.
- Hands-on experience with Azure Data bricks or Delta Lake, in building ETL pipelines : batch (autoloader) and Spark structured streaming
- Knowledge of data modelling and performance tuning in Spark.
- Exposure to CI/CD pipelines and DevOps practices.
- Familiarity with data governance and security practices.
- Strong hands on working experience of Unity catalog:
- Hands on exposure to Creating end to end environments : creating catalogs, schemas, tables . materialized views, functions, volumes
- Experience in building SCD 1 and SCD2 (slowly changing dimensions ) on dimension tables
- Experience in building CDC (change data capture pipelines)
- Strong hands on experience with Lakehouse federation , creating foreign catalogs to get data from external sources
- Strong understanding of databricks partitioning , Liquid clustering
Not the right fit? Search for Data Engineer jobs in Toronto, Ontario, Canada
About Kumaran Systems
Kumaran Systems is an IT Services Company with imprints in three countries. The past two decades has seen us provide our global clientele with high-end IT services that include migration support, system integration and infrastructure management solutions, providing one-stop-solution to all your IT demands.
Our expertise and in-depth knowledge of businesses help us cater to a variety of industries. Our team strives to know your industry better, by observing current trends and the way it works, to tailor-make our solutions to your needs.With over 20 years of customer orientation, over 2000 engagements across Telecom, Education and Banking & Financial services spread across the globe, Kumaran stands as a key advisor to some of the largest Fortune 500 companies in their business driven technology enablement drive. Kumaran System's Customer Orientation is driven by a global delivery business model giving its customers to choose between an Onshore-Offshore mix. The delivery models enable multilevel touch points between the client, partner networks and Kumaran thereby enabling business driven customer sensitivity and agility.
ENGAGE, EMERGE AND EXCEL
Involving ourselves in a purposeful action, using our abilities to its maximum, helps us feel positive about ourselves. Such positivity gives raise to positive thinking and positive ideas. Ideas evolve into vision and set our aim! Our aspirations engage us with the incentive of sweet success!
Every endeavor has its own pitfalls. Perseverance helps us emerge a successful entrepreneur after any deterrent. In our attempt to achieve our aim, we keep upgrading our knowledge and skill, to surface successful!
A genius mind does not work to excel, but excels in work! We challenge ourselves with every next step! We change every deterrent to a stepping stone to success.
We essentially, engage, emerge and excel!