Azure Data Engineer (Azure Databricks & Pyspark)
United States - Remote
12 month contract Azure Data Engineer (Azure Databricks & PySpark)
Job Description
Weβre hiring a talented Azure Data Engineer enthusiast to work in our platform to help
ensure that our data quality is flawless. As a company, we have millions of new data
points every day that come into our system. You will be working with a passionate team
of engineers to solve challenging problems and ensure that we can deliver the best data
to our customers, on-time. You will be using the latest cloud data lake technology to
build robust and reliable data pipelines.
Job Responsibilities
β Develop expertise in the different upstream data stores and systems across the
company.
β Design, develop and maintain data integration pipelines for the organization growing
data sets and product offerings.
β Build unit testing and QA plans for data processes.
β Build data validation testing frameworks to ensure high data quality and integrity.
β Write and maintain documentation on data processes.
β Developing and maintaining data models and schemas.
β Strong analytical experience with database in writing complex queries, query
optimization, debugging, user defined functions, views, indexes etc.
β Write code that adheres to coding standards, procedures, and techniques. Maintain
the integrity of existing program logic according to specifications.
β Actively participate in the code review process to ensure development work adheres
to standards and specifications (including peer review and code review external to
team).
β Respond to all inquiries and issues in a timely manner as developed code/program
moves through the testing process.
β Participate in scrum, sprints, and backlog grooming meetings.
β Evaluate interrelationships between applications to determine whether a change in
one part of a project would impact or cause undesirable results in related applications
and design for effective interfaces between interrelated applications.
β Improve the health of system assets by identifying enhancements to improve
performance through tuning and monitoring, reliability, and resource consumption.
β Evaluate and troubleshoot root-cause analysis for production issues and
system failures; determine corrective action(s) and propose improvements to
prevent their recurrence.
β Maintain up-to-date business domain knowledge and technical skills in software
development technologies and methodologies.
β Provide input in the selection, implementation and use of development tools and best
practices.
Requirements
Technical:
β BS or MS in Computer Science or equivalent experience.
β 4 + years of experience in Databricks/Apache Spark with Azure data storage
solutions handling large datasets.
β Expert in SQL β¦
This job isn't fresh anymore!
Search Fresh JobsJob Profile
12 month contract 40 hours per week Not benefits eligible
Tasks- Build data validation testing frameworks
- Develop data integration pipelines
- Maintain documentation
- Participate in code reviews
- Troubleshoot production issues
Agile Algorithms Apache Spark API Azure Azure DevOps CI/CD Data analysis Databricks Data Factory Data Lake Data Structures Jira MS SQL Object-oriented Programming PostgreSQL PySpark Python Spark SQL SQL SQL DW
Experience4 years
EducationB.S. Computer Science Equivalent experience MS
TimezonesAmerica/Anchorage America/Chicago America/Denver America/Los_Angeles America/New_York Pacific/Honolulu UTC-10 UTC-5 UTC-6 UTC-7 UTC-8 UTC-9