Our downtown Chicago client is adding to their data engineering team in order to expand their cutting-edge, proprietary big data solutions for the retail industry. They seek intelligent Data Engineers who are critical thinkers, solutioneers and collaborators who can jump in, understand the big picture and craft solutions without any micromanagement.

Your expertise is needed to design a new architecture and development process to convert data warehouse and database models into Hadoop systems.

The team will rely on your knowledge of Spark SQL and Spark Streaming to support real-time processing of data and convert SQL and Oracle stored procedures into Hadoop using Spark SQL, Hive, Scala and Java.

If you are looking for a Data Engineer role where you can influence your direct circle and make a large impact on engineering solutions you will find this opportunity refreshing!

Some of your other responsibilities will include-

• Develop large scale data pipelines to handle petabytes of transaction data to analytical sources
• Build the supporting utilities and UDFs for the standardization of data pipelines with Scala and Java 7

• Extract, parse, clean and ingest the incoming data feeds, data logs and transaction logs
• Automate the data lake systems by developing batch and streaming workflows with Stonebranch scheduler and bash scripts
• Tap your Spark expertise to improve resource tuning, memory management and partitioning
• Resolve Spark and Yarn open issues

This is a six month contract to hire opportunity. Immerse yourself in the role for six months and then decide if you want to call it home.

If you find yourself excited about what you ve just read and have the qualifications shown below we want to talk to you. Apply now to have a confidential conversation where you will learn more details about this special opportunity.