JOB DESCRIPTION
- Design, develop, and maintain scalable data pipelines using Apache Airflow, DBT and other related tools.
- Implement and manage ETL processes to ingest data from various sources, including Google DataStream, Redpanda and Kafka, into Google BigQuery.
- Collaborate with Data Analysts and Data Scientists to ensure that the data architecture supports their requirements for reporting and analysis.
- Build and maintain CI/CD pipelines for data-related projects using Git and other related tools.
- Monitor and troubleshoot data pipelines to ensure reliability and performance.
- Implement data governance and best practices to ensure data quality and security.
- Work with stakeholders across the organization to gather requirements and deliver data solutions that drive business impact.
JOB REQUIREMENTS
MUST HAVE SKILLS:
- 4+ years of experience in data engineering or a related field.
- Bachelor’s degree in Computer Science, Data Engineering, or a related technical field.
- Hands-on experience with Apache Airflow, DBT or other related tools.
- Strong proficiency in Python for data manipulation, transformation, and automation.
- Demonstrated proficiency in one or more implementations of SQL (e.g. MySQL, BigQuery)
- Familiarity with CI/CD, Kubernetes practices and tools, particularly Git.
- Experience with streaming data platforms such as Google DataStream, Redpanda and Kafka.
- Strong problem-solving skills and the ability to work with large, complex data sets.
- Understanding of data governance, data security, and best practices in data management.
NICE TO HAVE SKILLS:
- Experience with cloud platforms, particularly Google Cloud Platform (GCP).
- Familiarity with data visualization tools like Tableau, PowerBI, or Apache Superset.
- Experience in a fast-paced, high-growth environment, such as a startup or tech company.
- Knowledge of other data storage and processing technologies (e.g., Snowflake, Kafka).
- Strong communication skills with the ability to collaborate with cross-functional teams.
Skills Tags: Python, Data Engineer, SQL, DB, Tableau, PowerBI
BENEFITS
Competitive Salary and Brilliant Health Benefits
Progressive and Fun Working Environment
Valuable Training
