We are seeking an experienced Data Engineer with a strong background in AWS toolsets to advise, develop, and maintain data engineering solutions within the AWS Cloud ecosystem. This role involves designing, building, and managing both batch and real-time data pipelines using various AWS services.

Key Responsibilities:

• Design, build, and operate batch and real-time data pipelines using AWS services like AWS EMR, AWS Glue, Glue Catalog, and Kinesis.
• Develop data layers on AWS RedShift, Aurora, and DynamoDB.
• Migrate data using AWS DMS and manage data integration using AWS Data Platform components such as S3, RedShift, RedShift Spectrum, AWS Glue with Spark, AWS Glue with Python, Lambda functions with Python, AWS Glue Catalog, and AWS Glue Databrew.
• Build data pipelines for Data Warehouses and Data Lakes using AWS Kinesis and Managed Streaming for Apache Kafka.
• Utilize open-source technologies, including Apache Airflow and dbt, as well as Spark/Python or Spark/Scala on the AWS Platform.
• Schedule and manage data services on AWS, ensuring efficient operation and integration.

Mandatory Requirements:

• Experience working on traditional Data Warehousing with any database (Oracle, DB2, or SQL Server; Redshift experience is a plus).
• Strong SQL skills with the ability to write complex queries using analytical functions.
• Prior experience working on the AWS platform.
• Proficient in Python programming for data engineering.
• Experience with PySpark/Spark.
• Working knowledge of data pipeline tools like Apache Airflow.

Preferred Skills and Expertise

• Experience with DBT.
• Familiarity with Agile working environments.
• Proven ability to troubleshoot and resolve production issues under a DevOps model.
• A consistent track record of identifying opportunities to improve performance and quality within data ecosystems.
• Experience monitoring system performance and ensuring high-quality data operations.

if the above looks of intererest, please make an applicatoin and Experis will be in contact with you.

Rowan Mollison
Senior Consultant, Experis

ManpowerGroup is committed to being a Diversity Confident Recruiter and encourages applications from people from a diverse range of backgrounds, including people with a disability. Please indicate your preferred method of communication in your resume and please let us know if you require any reasonable adjustments should you be contacted for an interview.

Aboriginal and Torres Strait Islander people are encouraged to apply.

By submitting your resume and other personal information with this application you are consenting to this information being collected in line with our privacy policy. Follow the link to learn more - www.manpowergroup.com.au/privacy-policy

Experis Pty Ltd is a wholly owned subsidiary of ManpowerGroup

State: QLD, licensee/s Manpower Services (Australia) Pty Ltd, LHL-02026-D5L4Q. State: QLD, licensee/s Experis Pty Ltd, LHL-02014-Y5F6D. State: SA, licensee/s Manpower Services (Australia) Pty Ltd, LHS 288856