As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines, data integration processes, and data infrastructure using Dataiku
You will collaborate closely with data scientists, analysts, and other stakeholders to ensure efficient data flow and support data-driven decision making across the organization
Implement data processing and transformation workflows using Databricks, Apache Spark, and SQL to support analytics and reporting requirements
Build and maintain orchestration workflows using Apache Airflow to automate data pipeline execution, scheduling, and monitoring
Lead the migration of legacy data systems to modern cloud-based architectures
Develop and maintain CI/CD pipelines for data workflows
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver scalable data solutions
Требования
We are seeking an experienced Data Engineer to join our data team
Optimize data pipelines for performance, reliability, and cost-effectiveness, leveraging AWS best practices and cloud-native technologies
Implement data processing and transformation workflows using Databricks, Apache Spark, and SQL to support analytics and reporting requirements
Build and maintain orchestration workflows using Apache Airflow to automate data pipeline execution, scheduling, and monitoring
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver scalable data solutions
Навыки
Design, develop, and deploy end-to-end data pipelines on AWS cloud infrastructure using services such as Amazon S3, AWS Glue, AWS Lambda, Amazon Redshift, etc
Условия
Significant career development opportunities exist as the company grows
