Data Engineer
8.0/10
Hibachi
$140,000 – $200,000
Remote
mid
10 days ago
analyticsdevPySparkAWS GlueAirflowAWS DMSSQLIcebergQuickSightTableauPower BIKafka
AI Summary
The vacancy is well-defined with clear responsibilities and compensation, but lacks detailed company information and explicit KPIs.
Description
Join Hibachi as a Data Engineer to develop and maintain data pipelines, design data models, and create dashboards using PySpark, AWS Glue, and Airflow.
## What you'll do
- •Architect, build, and maintain batch and streaming data pipelines using PySpark, AWS Glue, and Airflow.
- •Implement Change Data Capture (CDC) with AWS DMS to capture incremental updates.
- •Design modular, reusable, and scalable data models.
- •Manage ETL/ELT pipelines ensuring efficient data ingestion, cleansing, and aggregation.
- •Develop QuickSight dashboards to surface actionable insights.
Requirements
- •Bachelor’s or Master’s Degree in Computer Science, Engineering, or a related field.
- •2+ years of experience with PySpark for batch and streaming pipelines.
- •Proficiency in AWS Glue, Apache Airflow, and Iceberg.
- •Experience with AWS DMS or other CDC tools.
- •Advanced SQL knowledge.
- •Experience with BI platforms like QuickSight, Tableau, or Power BI.
- •Understanding of testing frameworks like Pytest.
- •Excellent communication skills.
Loading similar jobs...