Data Engineer
Hong Kong
Full time
Hybrid
Compensation is not specified
Role
Data Engineer
Description
As a Data Engineer, you will be an integral part of a skilled data engineering team, contributing to the design, construction, and maintenance of our robust data infrastructure. Working closely with data analysts, scientists, and business stakeholders, your role will involve creating and deploying data solutions to support data-driven decision-making. Through adherence to best practices, you will ensure the smooth operation of our big data platform, enabling organizational insights and innovation.
Responsibilities
- Develop scalable data pipelines and ETL processes to efficiently manage large volumes of data from diverse sources into our data lake.
- Establish and execute data quality checks and monitoring procedures to uphold data accuracy and integrity.
- Utilize AWS services (such as EKS, EMR, S3, Glue, Lambda) to construct and manage cloud-based data infrastructure.
- Build CI/CD pipelines for data engineering operations.
- Leverage Infrastructure as Code (IaC) tools like CloudFormation and Terraform for AWS resource management.
- Stay informed about the latest trends and technologies in data engineering.
Qualifications
- Hold a Bachelor's degree in Computer Science, Engineering, or related field.
- Possess a minimum of 3 years of experience in Data Engineering.
- Proficient in SQL, Python, and Spark.
- Familiarity with relational databases like PostgreSQL.
- Hands-on experience with AWS cloud services (e.g., EKS, EMR, S3, Athena, Lambda).
- Proficiency in Kafka and real-time data processing.
- Experience with Tableau for setup and maintenance.
- Familiarity with data lake and lake house architecture.
- Experience with Databricks and Delta Lake.
- Proficient in Apache Airflow for workflow orchestration.
- Proficient in Kubernetes and Docker for containerization and orchestration.
- Experience with CI/CD pipelines and related tools.
- Experience with Infrastructure as Code (IaC) tools like CloudFormation and Terraform.
- Strong problem-solving and analytical capabilities.
- Attention to detail and ability to work autonomously.
- Excellent communication and collaborative skills.
Skills Required
Сrypto.com
Website
Сrypto.comCompany size
Not specified
Location
United States
Description
Not specified