Job Description

Apply - Click Here

Essential Functions

• Execute and manage large scale ETL processes to support development and publishing of Datamarts, Dashboard and reports and Predictive models
• Proficient in building, scheduling and managing DAGs in Apache Airflow
• Monitor data processing tasks in the data pipelines using in-house monitoring tools
• Quality control of data assets to ensure the data loaded across different stages in the data pipeline are reconciled
• Collaborate with cross-functional teams across geographies to understand data flows and processes to enable design and creation of the best possible solutions to each engineering challenge
• Strong analytical and problem resolution skills and build self-service tools and utilities to improve operational tasks
• Identify, design and develop areas for automation in the data monitoring processes
• Build and develop ETL pipelines in Spark, Python, HIVE that process transaction and account level data and standardize data fields across various data sources
• Provide quality data solutions in a timely manner and be responsible for data governance and integrity while meeting objectives and maintaining SLAs
• Desire to develop a working knowledge of change management, corporate IT audit processes, technical problem resolution, operations systems and data sources knowledge
• Flexible approach to analyzing technical issues and clearly communicating recommendations/solutions with stakeholders in an ever-changing environment
  
  Qualifications
• 3+ yrs. work experience with a Bachelor s Degree or 2+ years of work experience with a Masters or Advanced Degree in an analytical field such as computer science, statistics, finance, economics or relevant area.
• Working knowledge of Hadoop ecosystem, Apache Spark and Python

Business skills and general qualities:

• Ability to understand the problem statement or issue at hand raised by tech and business teams on data availability
• Good understanding of agile working practices and related programmanagement skills.
• Good communication skills with ability to interact with different cross-functional team members at varying levels
• Ability to learn new tools and paradigms as data science continues to evolve at Visa and elsewhere.
• Demonstrate intellectual and analytical rigor, team oriented, energetic, collaborative, diplomatic, and flexible style.
• Intellectually curious and continuously striving to learn.
   

Technical skills:

• 3+ years professional experience in building and managing data pipelines and writing ETL/ELT code (Python, Hive, Spark, SQL)
• 2+ years of experience working with scheduling tools (Airflow, Oozie) or building data processing orchestration workflows
• 3+ years of hands-on experience working with large scale data ingestion, processing, and storage in the Hadoop ecosyste
• Experience in writing and optimizing SQL queries in Big data environment.
• Experience working in Linux/Unix environment and exposure to command line utilities.
• Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.
• Exposure to Visualization Tools like Tableau, Power BI and D3
• Exposure to machine learning models based on unstructured, structured, and streaming datasets.
• Strong communication skills

Skills Required: Scheduling