About the role
We are seeking a highly skilled and experienced Senior Spark Developer to join our team. This role demands a deep understanding of Apache Spark and its ecosystem, along with a strong foundation in Java/J2EE application development. You will be responsible for designing, developing, and optimizing Spark-based solutions to process large volumes of data, collaborating with various teams to deliver high-performance and scalable data applications. This is a challenging and rewarding opportunity to contribute to a cutting-edge data engineering team.
Responsibilities:
- Design, develop, and implement complex Spark applications for processing large datasets, leveraging your expertise in distributed computing and data manipulation techniques.
- Partner with management teams to integrate Spark-based solutions into existing data infrastructure, ensuring seamless data flow and optimal performance.
- Resolve critical data processing challenges by evaluating complex business processes, system processes, and industry standards, applying your deep understanding of Spark's capabilities.
- Provide technical leadership and mentorship to other engineers, guiding them on Spark best practices and promoting efficient development strategies.
- Develop and maintain coding standards, testing procedures, and debugging techniques specifically for Spark applications.
- Contribute to the overall data architecture, ensuring that Spark solutions align with business goals and integrate effectively with other data technologies.
- Appropriately assess risk when making technical decisions, considering factors such as data security, performance, and scalability.
- Drive compliance with applicable laws, rules, and regulations, adhering to company policies and demonstrating sound ethical judgment.
Qualifications:
- 6+ years of application development experience, with a significant focus on Java/J2EE and Apache Spark.
- Strong command of Object-Oriented Programming (OOP) principles and design patterns in Java.
- Excellent understanding of data structures and algorithms, and their application in Spark development.
- Proven experience developing complex Java/J2EE applications, encompassing all areas of the technology stack.
- Thorough knowledge and hands-on experience with Hadoop, MapReduce, YARN, Sqoop, Pig, Hue, Unix, Impala, and Cassandra on Mesos.
- Extensive experience designing, developing, and deploying Spark applications in a production environment, handling large volumes of data.
- Deep understanding of Spark architecture, including Spark SQL, Spark Streaming, RDDs, and DataFrames.
- Experience with ETL and BI tools is a plus.
- Experience working in an agile environment using Scrum methodologies.
- Expertise in designing and optimizing software solutions for performance and stability, particularly in the context of Spark.
- Expertise in troubleshooting and problem-solving complex technical issues.
- Experience with test-driven development (TDD).
- Cloudera certification (CCDH) is an added advantage.
Preferred Qualifications:
- Experience with cloud-based Spark platforms (e.g., Databricks, AWS EMR, Azure HDInsight).
- Experience with other big data technologies (e.g., Kafka, Hive, HBase).
- Experience with data visualization tools and techniques.
Education:
- Bachelor's degree in Computer Science, Engineering, or a related field.
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Job Family Group: Technology
Job Family: Applications Development
Time Type: Full time
Most Relevant Skills Please see the requirements listed above.
Other Relevant Skills For complementary skills, please see above and/or contact the recruiter.
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster.
About Citi
Citi's mission is to serve as a trusted partner to our clients by responsibly providing financial services that enable growth and economic progress. Our core activities are safeguarding assets, lending money, making payments and accessing the capital markets on behalf of our clients. We have 200 years of experience helping our clients meet the world's toughest challenges and embrace its greatest opportunities. We are Citi, the global bank – an institution connecting millions of people across hundreds of countries and cities.
For information on Citi’s commitment to privacy, visit on.citi/privacy.
About the role
We are seeking a highly skilled and experienced Senior Spark Developer to join our team. This role demands a deep understanding of Apache Spark and its ecosystem, along with a strong foundation in Java/J2EE application development. You will be responsible for designing, developing, and optimizing Spark-based solutions to process large volumes of data, collaborating with various teams to deliver high-performance and scalable data applications. This is a challenging and rewarding opportunity to contribute to a cutting-edge data engineering team.
Responsibilities:
- Design, develop, and implement complex Spark applications for processing large datasets, leveraging your expertise in distributed computing and data manipulation techniques.
- Partner with management teams to integrate Spark-based solutions into existing data infrastructure, ensuring seamless data flow and optimal performance.
- Resolve critical data processing challenges by evaluating complex business processes, system processes, and industry standards, applying your deep understanding of Spark's capabilities.
- Provide technical leadership and mentorship to other engineers, guiding them on Spark best practices and promoting efficient development strategies.
- Develop and maintain coding standards, testing procedures, and debugging techniques specifically for Spark applications.
- Contribute to the overall data architecture, ensuring that Spark solutions align with business goals and integrate effectively with other data technologies.
- Appropriately assess risk when making technical decisions, considering factors such as data security, performance, and scalability.
- Drive compliance with applicable laws, rules, and regulations, adhering to company policies and demonstrating sound ethical judgment.
Qualifications:
- 6+ years of application development experience, with a significant focus on Java/J2EE and Apache Spark.
- Strong command of Object-Oriented Programming (OOP) principles and design patterns in Java.
- Excellent understanding of data structures and algorithms, and their application in Spark development.
- Proven experience developing complex Java/J2EE applications, encompassing all areas of the technology stack.
- Thorough knowledge and hands-on experience with Hadoop, MapReduce, YARN, Sqoop, Pig, Hue, Unix, Impala, and Cassandra on Mesos.
- Extensive experience designing, developing, and deploying Spark applications in a production environment, handling large volumes of data.
- Deep understanding of Spark architecture, including Spark SQL, Spark Streaming, RDDs, and DataFrames.
- Experience with ETL and BI tools is a plus.
- Experience working in an agile environment using Scrum methodologies.
- Expertise in designing and optimizing software solutions for performance and stability, particularly in the context of Spark.
- Expertise in troubleshooting and problem-solving complex technical issues.
- Experience with test-driven development (TDD).
- Cloudera certification (CCDH) is an added advantage.
Preferred Qualifications:
- Experience with cloud-based Spark platforms (e.g., Databricks, AWS EMR, Azure HDInsight).
- Experience with other big data technologies (e.g., Kafka, Hive, HBase).
- Experience with data visualization tools and techniques.
Education:
- Bachelor's degree in Computer Science, Engineering, or a related field.
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Job Family Group: Technology
Job Family: Applications Development
Time Type: Full time
Most Relevant Skills Please see the requirements listed above.
Other Relevant Skills For complementary skills, please see above and/or contact the recruiter.
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster.
About Citi
Citi's mission is to serve as a trusted partner to our clients by responsibly providing financial services that enable growth and economic progress. Our core activities are safeguarding assets, lending money, making payments and accessing the capital markets on behalf of our clients. We have 200 years of experience helping our clients meet the world's toughest challenges and embrace its greatest opportunities. We are Citi, the global bank – an institution connecting millions of people across hundreds of countries and cities.
For information on Citi’s commitment to privacy, visit on.citi/privacy.