Back to Job Search

IS Data Engineer III

  • Location: Philadelphia, Philadelphia, Pennsylvania
  • Salary: 75-78 per hour
  • Job Type:Contract

Posted about 1 month ago

  • Sector: Information Technology
  • Contact: Shannon Block
  • Contact Email: Shannon@staffing-the-universe.com
  • Contact Phone: (646) 652-0976
  • Start Date: 2021-05-24
  • Job Ref: 21-11755
Design and architect Data Lake on AWS/Snowflake
Design and implement data ingestion pipelines to load data into AWS platform.
Perform data migration from Hadoop on-premises platform to AWS platform.
Build automated, scalable, test-driven ELT pipelines by utilizing software development
practices such as version control via Git, CI/CD, and release management
Build automated data validation processes to ensure the quality and integrity of the datasets.
Build automation scripts to convert Hive/IBM SQL and Netezza SQL into Redshift/Snow SQL.
Orchestrate jobs through scheduling tools such as Airflow.
Help setup the AWS environments

Experience working with AWS Cloud platform
Experience with big data toolset: Hadoop, Spark, Kafka, Hive, sqoop etc.
Strong SQL, Data Modeling and Data Warehousing & Data Lake fundamentals.
Experience working with Snowflake data platform.
Ability to code with scripting languages such as Python, Bash etc.,
Good hands-on experience with Linux (RHEL/Debian) operating system
Knowledge of version control systems such as Git.
Experience consuming and building APIs
Experience utilizing Agile methodology for development

Required Education:
Bachelor's Degree in Computer Science, Computer/Software Engineering, Information Technology or related fields.

Preferred Education:
Advanced Degree in Computer Science, Informatics, Information Systems or another quantitative field.

Experience:
Minimum of ten (10) years of experience in a Data Engineering/Data Architecture/Data Warehousing