Lead Data Engineer – Business & Competitive Intelligence

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers’ evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection. 

Job Description

This role is responsible for the design, development and management of data infrastructure projects & proof of concept business solutions for users in analytics and Data Science. This person works directly with business users, data scientists and analytic engineers on needs and tactical solutions. This role will execute new data engineering work tracks for Data Science and analytics from inception and prototyping to fully developed solutions and transition to Allstate technology.

Key Responsibilities 

• Partner and help execute the development of new concepts, proof of concept designs, and prototypes for business or research data solutions so that business users or predictive modelers may visually understand and explore a new feature or functionality before implementation to expose design assumptions and drive ideation. 
• Execute data exploration, data engineering, testing and implementation. 
• As data engineering expert, guide peers or AOR partners on technical solutions. Understand the changing analytic needs of users and Data Scientists and assists in the development of goals, and tracks of work to meet those needs. 
• Development of enterprise data assets, information platforms or data spaces designed for exploring and understanding the data. Support the data engineering environment, and execute monitoring, quality, and validation processes to ensure data accuracy. 
• Leverage “agile” data analysis with technology fluency in parallel processing/programming, software/programming languages and technologies (i.e., Oracle, MongoDB, SQL, Python, Spark, Kafka, Scala, and Hadoop) paired with a high degree of analytic agility to be able to meet fluid and dynamic business needs in this space. 
• Execute rapid development of new data and analytic work tracks with fast iteration over quick sprints 
• Help develop and deliver the data infrastructure required to support needs of predictive modeling and analytics
• Develop test plans and scripts, preparation of test data, reconciliation of data sources, and execution of functional tests. 
• Participate in requirement development, breaking down business problems into solvable components and assists in documenting requirements 
• Work closely with Data Science and business partners requiring subject matter expertise in the business as well as advanced analytics. 
• Mentor other team …