Lead Data Engineer
Location : Fort Mill SC
Skills : Snowflake, AWS glue, Hive, Python and Pyspark.
Experience: +10 years
We at Coforge are hiring Lead Data Engineer with the following skill sets.
Design, develop, and implement data pipelines using AWS Glue and Lambda to support data ingestion and transformation.
Utilize PySpark for data processing and analysis, ensuring efficient data handling and performance optimization.
Implement Infrastructure as Code (IAC) using Terraform (TF) to automate deployment and management of cloud resources.
Develop and maintain SQL queries for data extraction, transformation, and loading (ETL) processes.
Manage containerized applications using Docker and orchestrate them with Amazon EKS.
Design and optimize data warehouse solutions using Snowflake (SF) or Redshift (RS) to support analytics and reporting.
Work with data lake technologies such as Iceberg, Lake Formation, and Delta tables to enhance data accessibility and governance.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Ensure data quality, integrity, and security throughout the data lifecycle.
Stay updated with the latest industry trends and technologies to continuously improve data engineering practices.
Mandatory Skills
Strong experience with AWS services, particularly Glue and Lambda.
Proficiency in PySpark for data processing.
Expertise in Infrastructure as Code (IAC) using Terraform.
Solid SQL skills for data manipulation and querying.
Experience with containerization using Docker and orchestration with EKS.
Hands-on experience with data warehousing solutions like Snowflake or Redshift.
Familiarity with data lake technologies such as Iceberg, Lake Formation, and Delta tables.
Proven ability to work in a cloud environment.
Preferred Skills
Experience with data governance and data quality frameworks.
Knowledge of data modeling and database design principles.
Familiarity with Agile methodologies and project management tools.
Strong analytical and problem-solving skills.
Excellent communication and collaboration abilities.
Qualifications
Bachelor's degree in Computer Science, Information Technology, or a related field.
8-12 years of experience in data engineering or a related role.
Proven track record of delivering data solutions in a cloud environment.
Strong understanding of data architecture and data management best practices.
...Title: Aerospace Engineer Level IV Belong. Connect. Grow. with KBR! KBRs National Security Solutions team provides high-end engineering and advanced technology solutions to our customers in the intelligence and national security communities. In this position, your...
...who are great at data entry and typing. We offer a flexible work from home remote position that allows you to stay home with the family! The... ...assistant, warehouse, inventory, receptionist, call center, parttime, retail fields & more. Must be 16 years of age or older...
...Middle School Computer Science TeacherSalary Range: $72,943-$137,945/annuallyLength of Appointment: 2026-2027 School YearCalendar: Full Time / Full YearWho We AreThe San Francisco Unified School District (SFUSD) is the seventh largest school district in California...
...Education Minimum Required Registered by American Registry of Radiologic Technologists ARRT(R), with Registry in Mammography ARRT(M) preferred. Must complete certification for ARRT(M) within 12 months of hire date. Basic Life Support (BLS) certification...
...Job Description Job Description Assist our TEAM with shampoo rituals and general upkeep of salon. In this role the ideal candidate would be a quick learner, able... ...to multi task to keep salon floor clean (sweeping hair) and supplies available (laundry). This is a fun environment...