We are looking for a Data Engineer to join our team! In this role, you will work within an engineering team to design and develop data processing platforms and software frameworks for Data Science, Engineering and Analytics organizations.
Most of the things you’ll work on:
- Batch data processing systems
- Batch and semi-real time revenue recognition event processing systems
- Maintenance of the data lake and related access libraries that enable analytics
- Write code to build new or modify existing ETLs (Spark+Scala).
- Build new or modify existing DAGs (Airflow)
- Investigate data discrepancies identified by external stakeholders
- Run UAT activities and make changes as requested.
- Be an active contributor in agile ceremonies that help the team plan and get work done: standup, backlog grooming, sprint planning, retrospectives, etc
You’ll need to have:
- 3+ years of experience as a Data Engineer
- Experience writing ETLs on big data platforms such as Airflow, Spark and Scala
- Knowledge of Apache Spark
- Knowledge of AWS Infrastructure