Senior Data Engineer

Thank you for considering Progyny!

Data is the lifeblood of healthcare, and Progyny as a company depends on the accurate, timely, and secure management of data to achieve our company mission at the highest level. This role will directly support all Machine Learning and AI based solutions at Progyny. This role will be directly responsible for supporting Progyny’s efforts to provide an enhanced member experience and better insights to our team leveraging best in class components for core data persistence, streaming, transformation, and integrated analytics leveraging machine learning. As data systems are constantly evolving, this role will also be expected to help challenge and mature the existing patterns, practices and tool chains for the Data engineering discipline at Progyny; as such, a successful candidates should be motivated by a desire to help optimize and shape their environment, not simply follow the established status quo.

Success in this role is measured by the following:

• Development of data pipelines that meet or exceed our quality goals

• Ability to leverage your knowledge of various data patterns/frameworks/tools to help guide implementation to best possible solutions

• Staying curious and ensuring currency in a fast-evolving technology discipline while also supporting growth/maturity of your peers & stakeholders

• Working “without a net”: navigating past ambiguity through the application of your judgment and experience

• Ability to find pragmatic options to tactical vs. strategic implementation dilemmas

What you’ll do…

• Working with machine learning and AI stakeholders to design and implement numerous data transformation pipelines, with a focus on all functional/non-functional quality goals

• Design and implementation of test, monitoring, and observability strategies for data systems and processes

• Support for outage incident response and return-to-service activities for data systems in your purview

• Supporting evaluation of related tools, frameworks and dev/test/deploy patterns as needed to evolve the platform capabilities for data support

About you…

Must have:

• Strong communication skills

• Experience with well-known data pipeline/transform tools / platforms (DAG-based options like Airflow, Dagster, Prefect, etc.)

• 7+ years of consistent, professional SQL development experience

• 5+ years of Python development experience

• Experience with data access patterns (caching, partitioning, sharding)

• Familiarity with data modeling best practices / data structures

Must have at least 2 of these:

• Experience with Machine Learning and AI training and inference pipeline development.

• Experience working on streaming/event pipelines using Kafka or similar technology.

• Working experience with the FHIR JSON-based storage models.

• Experience working with graph data models and databases.

Please note: Progyny is …