Data Engineer - Mid Level

Computer Scientist - II ( Data Engineering ) Key Responsibilities Data Architecture: Design scalable distributed data processing systems. Pipeline Development: Build ETL/ELT pipelines for complex datasets. AI Integration: Implement and fine-tune generative AI capabilities. App Engineering: Develop backend services to expose data products. System Optimization: Improve code performance, data quality, and storage efficiency. Mentorship: Guide junior engineers and advocate for technical best practices. Required Technical Skills Primary Expertise (Expert Level) Data Engineering: Apache Spark, Hadoop, Kafka, and Databricks/Delta Lake. Python Ecosystem: PySpark, Pandas, NumPy, and performance tuning. Cloud Platforms: Advanced deployment experience within AWS or Azure. Data Modeling: Expert knowledge of relational and non-relational databases. Secondary Expertise (Intermediate Level) GenAI / LLMs: Experience with LangChain, LlamaIndex, and vector databases (e.g., Pinecone, Milvus). Application...