Data Engineer, Affiliate Commerce

Raptive is looking for an experienced Data Engineer to join us as we expand product offerings for our publishers into the affiliate commerce space, helping creators who have authority and expertise in particular subject areas diversify and increase their revenue. 

As a Data Engineer in the Affiliate Commerce Team, you’ll design scalable, reliable, maintainable, and extensible systems to ingest data from a variety of sources, make connections, generate insights, and integrate with our products. You’ll be instrumental in collaboratively delivering a meaningful new product and business opportunity that supports the open web.

What you’ll be doing:

• Maintain existing data pipelines and develop new ones to integrate data from various external and internal sources via APIs, Airbyte, and data lakes.
• Participate in the design and refinement of data systems and infrastructure to ensure scalability, reliability, and efficiency.
• Monitor the performance of data pipelines and databases, optimizing and enhancing as needed to improve efficiency and reduce costs.
• Maintain comprehensive documentation for data systems and pipelines.
• Engage regularly with internal stakeholders to understand their data requirements, update them on pipeline developments, and gather feedback to continuously improve data services.
• Build and optimize data transformation pipelines using dbt, orchestrated via Airflow (using Cosmos), to ensure high-quality, single-source-of-truth reporting.
• Independently manage significant technical tasks while collaborating closely with software engineers and other technical teams on data requirements and integration strategies.
• Develop scalable, efficient, and easily maintainable data pipelines using advanced technologies and frameworks.

The skills and experience you bring to the job:

• Strong hands-on experience with AWS (specifically Lambda, Kinesis Firehose and S3), Snowflake, and dbt.
• Advanced skills in Python and SQL for custom scripting and data manipulation.
• Experience with Snowpark or familiarity with Spark DataFrames.
• Proficient in working with various types of APIs, including REST and GraphQL.
• A keen eye for maintaining high data quality across diverse sources and complex transformations.
• Ability to work independently and drive projects forward while also engaging effectively in collaborative team environments.

Additional Preferred Skills:

• Understanding of DevOps practices and experience with infrastructure as code tools such as Terraform.
• Ability to work with CI/CD pipelines, specifically with GitHub Actions.
• Knowledge of machine learning techniques and their application in data processing or predictive analytics.
• Experience with real-time data processing tools and methodologies.
• Background in building and maintaining data warehouses and data lakes, ensuring robust and scalable data storage solutions.

About Raptive 

Raptive’s mission is to power creator independence. We’re a …