Big Data Engineer
Description: Responsibilities: • Development and Maintain Data Pipelines: Design, implement, and optimize end-to-end ETL/ELT pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. • Utilize Python and Pyspark: Write efficient, scalable and maintainable code in Python and leverage Pyspark for large-scale data processing in distributed computing environments. Also be able to review existing code and identify areas of...
- Amazon Redshift
- Apache Spark
- Azure Data Factory
- BigData -Hadoop
- BigData -Hive
- Databricks
- Google Cloud Platform
- Scala
- Snowflake