Lead Data Engineer

Lead, mentor, and guide a team of data engineers, fostering a culture of collaboration, innovation, and continuous learning.,Provide technical guidance and support to team members, helping them solve complex problems and grow their skills.,Perform analysis on structured, unstructured, and semi-structured raw data, utilizing programming skills to standardize and organize data.,Design and implement scalable, reliable, and efficient data pipelines and workflows.,Stay updated on emerging technologies and tools in data engineering, such as cloud platforms (AWS, Azure, GCP), big data frameworks (Hadoop, Spark), and modern data platforms (Snowflake, Databricks).,Evaluate and recommend tools, frameworks, and technologies to improve data engineering processes.,Ensure data systems are optimized for performance, scalability, and security.,Identify, source, staging, and modeling internal process improvements to automate manual processes and optimize data delivery for greater scalability, as part of the end-to-end data lifecycle.,Oversee the integration of data from multiple sources, ensuring data consistency, quality, and accessibility.,Develop ETL/ELT processes to transform raw data into usable formats for analytics and reporting.,Provide consulting services on new initiatives of small to medium complexity.,Support Development for data ingestion and transformation jobs.,Develops and supports Python and SQL code for data ingestion and transformation jobs.,Create IBM JCL and job scheduling documents as needed.,Contribute to successful project completion by identifying risks and developing/recommending mitigation strategies.

7+ years working experience with big data platforms like Cloudera/Snowflake/Hadoop, Databases and SQL language.,5+ years working experience with Scripting languages like Spark, PySpark, Shell, Python, IT Standards, Methodologies, Agile, SDLC, Information Security & audit requirements.,Prior experience leading complex projects and technical team.,Working Knowledge with change control tools like GitHub, Helios etc.,Knowledge on creating formal written deliverables and other documentation and ensure design, code and documents are aligned with enterprise direction, principles and standards.,Working knowledge of JCL/Zeke/OMS (Nice-to-have).,Working knowledge of ServiceNow Incident and Change Management (Nice-to-have).,Working knowledge of Helios/GitHub for Source Code Management (Nice-to-have).,Insurance industry knowledge (Nice-to-have).