Data Engineer

Job Title: Hadoop data engineer Location: Chicago, IL Work Arrangement: Onsite Client Industry: Financial Services Duration: 12 Months Contract Schedule: Standard Hours( Monday - Friday, 8 AM to 5 PM) Day-to-Day: Design, develop, and maintain batch and near-real-time data pipelines using Spark Structured Streaming, MapReduce, and other Big Data frameworks. Ingest data from multiple sources such as message queues (Kafka), file shares, REST APIs, and relational databases. Transform, clean, and validate data in HDFS, Hive, Impala, or Spark SQL. Convert and manage data in formats like JSON, CSV, XML to support downstream analytics. Perform data validation, profiling, and analysis to identify anomalies and ensure data integrity. Troubleshoot issues in data pipelines, SQL jobs, or Spark applications, including slow-running jobs or failures. Must Haves: Strong SQL Skills – one or more of MySQL, HIVE, Impala, SPARK SQL Data ingestion experience from message queue, file share, REST API, relatio...