Data Engineer

The Data team at air up mission is to empower our users with trusted data that is easy to access, understand and use to support evidence-based business decisions. We are the main knowledge and insights powerhouse that supports the business with data and insights that shapes the data-driven decision-making required to constantly scale and grow in profitability. Our data infrastructure mindset is cloud-oriented and based on modern-data stack.

As Data Engineer, you are responsible for managing our tech stack, architecture, data ingestion layer, APIs, orchestration while working in close collaboration with our data scientists, data analysts and web analysts as well as collaborating with our engineering managers within the tech division to provide a self-service platform that delivers value for the business. This could mean doing a number of things, such as, rolling up your sleeves to help out technically, coaching, advising on design, process, organisation or best practices etc. You’ll work closely with the head of data to set the overall vision and direction for data architecture and engineering that create the much-needed impact in-line with company objectives.

• Collaboration: You work with analysts and scientists and software development engineers to develop, implement, and validate business logics, KPIs, statistical analyses, data profiling, prediction, forecasting, clustering, and machine learning algorithms.
• Monitoring: Leverage our monitoring tool (metaplane) to proactively detect data quality problems in our data lake and business metrics based on standard best practices.
• Engineering: You produce scalable, reliable code and engineering solutions that help automate repetitive data management tasks.
• Software Development: The development and improvement of the current software architecture, emphasizing data quality and timeliness, scalability, and extensibility is also part of your position.
• Architecture: Knowledge of data architectures and evolution to the expectations of a scaling organization.

About you

• Experience in data engineering, software architecture, and analytics with a commercial software product (2.5+ years)
• Experience with complex data modeling, ETL design, and using large databases in a business environment, as well as profound understanding and experience with Big Data technologies like Apache Spark and PySpark and corresponding AWS technologies.
• Advanced experience with languages like Python and SQL and advanced knowledge of relational databases (preferably AWS databases) and data management (Airflow)
• Experience with Cloud Services (e.g. EC2, S3, EMR, Redshift), modern cloud architecture (e.g. Docker), and platforms.
• Excellent verbal and written communication in English
• Proven interpersonal skills and ability