Hadoop PySpark Technical Lead Job at Diverse Lynx, Pittsburgh, PA

dzhDMkJlNW9IWStxSGpzbmpJdWQwTWho
  • Diverse Lynx
  • Pittsburgh, PA

Job Description

Position: Hadoop PySpark Technical Lead (10+ Years) Location : Pittsburgh, PA (Onsite) Hire Type: Contract Competencies: BigData and Hadoop Ecosystem - MapR, PySpark, Unix Shell Scripting and text processing tools Skill set- Cloudera Data Platform , PySpark, python ,Hive-Map Reduce, Linux /Unix, Impala , Big Data Technologies, Cloud Technologies Roles and responsibilities: Understand requirements/use cases and build efficient ETL solutions using Apache Spark, python , Kafka, Hive targeting Cloudera Data Platform Requirement/use case analysis and convert functional requirements into concrete technical tasks and able to provide reasonable effort estimates. Work closely with Data analyst/modeler, Business User to understand the data requirement. Convert requirements to high-level , low-level design and, source-to- target documents. Responsible to design , develop and schedule data pipelines which handle large volume of data within SLA. Work with solution architect, Technical Managers, Admins to understand SLAs , limitations of systems and provide efficient solutions. Expertise in processing large volume of data aggregation using spark , must know different performance improvement technique and should lead teams on optimization. Responsible to develop efficient data ingestion and data governance framework as per specification. Performance improvement of existing spark-based data ingestion, aggregation pipelines to meet SLA. Work proactively, independently with global teams to address project requirements, articulate issues/challenges with enough lead time to address project delivery risks. Plan production implementation activities , execute change requests and resolve issues in production implementation. Plan and execute large data migration, history data rebuild activities Code reviews/optimization, test case reviews . Demonstrate troubleshooting skill in resolving technical issues, bugs. Demonstrate ownership and initiative. Ability to bring-in best practices /solutions which best fit for client problem and environment. Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company. Diverse Lynx

Job Tags

Contract work,

Similar Jobs

Experior Financial Group

Work From Home Insurance Sales Job at Experior Financial Group

 ...communication skills.High energy in order to maintain high outbound sales activities.Ability to stay positive, engaged and unphased by rejection.Provide excellent customer service.Life/Health Insurance LicenseSales experience with successful track record of meeting... 

US Foods, Inc.

CDL A Truck Driver Job at US Foods, Inc.

 ...products with a hand truck and place items in the customer storage area. Able to perform repeated, manual heavy lifting of items up to 80 pounds. Make frequent stops during a work shift. Deliver product in inclement weather conditions. Verify delivery of items... 

Beacon Hill

Staffing Consultant-Agency Recruiter Job at Beacon Hill

 ...applications Salary: USD45000 - USD70000 per year + uncapped commissionSTAFFING CONSULTANTFINDING PEOPLE FOR JOBS, AND JOBS FOR PEOPLE.As a...  ..., post advertisements for open jobs, and learn to utilize online sourcing tools and resources.To be a successful Staffing... 

Intuit

Tax Associate - Work From Home - 2+ Yrs Paid Tax Experience Required Job at Intuit

Overview Intuit is seeking Tax Associates for the upcoming tax season. If you have an active PTIN, we would love to connect with you! What You Get: Company Benefits: 401(k) savings plan with Intuit matching, paid sick leave, Intuit provided E&O insurance, employee...

Allied Universal® Janitorial Services

Janitor Job at Allied Universal® Janitorial Services

 ...employee assistance programs, and exclusive perks. Job Description Allied Universal Janitorial Services is seeking the position of Janitor. $ 15.00 / Hour As a Janitor for Allied Universal Janitorial Services, you will be responsible for cleaning and servicing...