Data Engineer
Taipei, Taiwan
Full time
Hybrid
Compensation is not specified
Role
Data Engineer
Description
As a Data Engineer at Crypto.com, you have a crucial role within our skilled data engineering team. You will be instrumental in creating, constructing, and managing our solid data infrastructure. Collaboration with data analysts, scientists, and business stakeholders is key to developing and executing data solutions that foster informed decision-making. Adherence to top industry practices is vital to ensuring the effective and efficient operation of our large data platform, enabling data-driven insights and driving innovation throughout the organization.
Responsibilities
- Designs, creates, and maintains scalable data pipelines and ETL processes for extracting, transforming, and loading significant volumes of data from diverse sources into our data lake.
- Develops and deploys data quality checks and monitoring mechanisms to uphold data integrity and accuracy.
- Utilizes various AWS services like EKS, EMR, S3, Glue, and Lambda for constructing and managing cloud-based data infrastructure.
- Implements CI/CD pipelines for streamlining data engineering processes.
- Manages AWS resources using Infrastructure as Code (IaC) tools such as CloudFormation and Terraform.
- Stays informed about the latest data engineering technologies and evolving trends.
Qualifications
- Holds a Bachelor's degree in Computer Science, Engineering, or a directly related field.
- Has at least 3 years of experience in a Data Engineer role.
- Demonstrates strong skills in SQL, Python, and Spark.
- Shows familiarity with relational databases like PostgreSQL.
- Has hands-on experience with AWS cloud services including EKS, EMR, S3, Athena, and Lambda among others.
- Possesses experience with Kafka and real-time data processing.
- Experience with Tableau, specifically in setup and maintenance.
- In-depth knowledge of data lake and lake house architecture.
- Familiarity with Databricks and Delta Lake.
- Proficient in Apache Airflow for orchestrating workflows.
- Proficient in Kubernetes and Docker for containerization and orchestration.
- Experience with CI/CD pipelines and related tools.
- Familiar with Infrastructure as Code (IaC) tools like CloudFormation and Terraform.
- Exhibits excellent problem-solving and analytical capabilities.
- Demonstrates strong attention to detail and can work autonomously when needed.
- Excellent communication and collaboration skills.
Skills Required
Сrypto.com
Website
Сrypto.comCompany size
Not specified
Location
United States
Description
Not specified