fbpx

Senior Data Engineer
Wiley
Colombo

Location: 
Contract Type: 
Industry: 

Senior Data Engineer at Wiley

 

Applications are invited for all candidates to fill the Senior Data Engineer Job Opening at Wiley

Closing Date : 21/11/2021

 

As a Senior Data Engineer, you will work in the Wiley Education Services Analytics Engineering team who is responsible for developing robust and secure data pipelines to support Education Services initiatives. The role is responsible for design and development activities while working very closely with the Wiley Technology teams, and is expected to follow a test-driven development methodology, using CI/CD to deliver Data Pipelines. Good working knowledge of Python, Kubernetes, DevOps, DataOps, API’s, AWS, Databases, Data Warehouses and Data Lakes are essential. This role will require an individual who can work with minimal supervision on a wide range of projects interfacing with development teams, business analysts, management and business community while constantly ensuring that their work is aligned with Wiley Education Services’ strategic direction. They will be expected to stay current on technology and technique developments in cloud computing space while also continuing to grow their knowledge of Wiley’s business.
 
Responsibilities

  • Design and implement robust & secure data pipelines into a Snowflake data warehouse from on premise and cloud data sources.
  • Design and implement containerized solutions to facilitate fault-tolerant and scalable workflows using Kubernetes/Argo.
  • Design and implement automated CI/CD data pipelines.
  • Follow a Test-Driven Development methodology, working closely with QE to ensure robust pipeline development.
  • Mentor other team members in data engineering, guide and review development work providing coaching and coding feedback aligning to best practices.
  • Collaborate with Quality engineering and Solution management teams for capacity planning.
  • Work with Business Analysts and Users to translate functional specifications into technical requirements and designs.
  • Define best practices and standards for data pipelines and integration with Snowflake data lake and warehouses in collaboration with Data Architects and other Data Leads.
  • Ensure enterprise security and access control policies are adhered to in the solutions.
  • Creation of architecture and design artifacts and documents.

 
Requirements:

  • BSc in Computer Science, Engineering or equivalent. MSc in Big Data Engineering or equivalent will be beneficial.
  • 5+ years of hands-on experience in developing data applications using Python.
  • Good working knowledge of database technologies including relational databases, data warehouses, data lakes and cloud technologies.
  • Hands on experience with data workflow tools such Apache Airflow / LinkedIn Azkaban, and in-depth understanding of designing DAG based workflows, including workflow trees scheduling & triggering aspects.
  • Solid understanding of ELT and ETL approaches and knowledge of using either or both in large scale ingestion pipeline system.
  • Solid understanding of scalable and fault-tolerant pipeline deployment mechanisms and practices including automated ETL testing mechanisms, monitoring & alerting mechanisms, ingestion audit reports etc.
  • Strong understanding of various data formats such as CSV, XML, JSON, etc.
  • Hands on experience in developing/designing/integrating using Kubernetes.
  • Hands on experience implementing CI/CD pipelines, build and release management processes in AWS cloud environment.
  • Experience in producing architectural diagrams and documentation in an analytics environment.
  • Strong verbal and written communication skills including making presentations to senior leaders (business and technology).
  • Understanding of technology methodologies including Agile (SCRUM), continuous integration/ continuous development, and automation.
  • Familiarity with JIRA and writing user stories.

 
Please attach your CV in order to be considered for this position.

 

Apply now : https://bit.ly/3C18PAv

Contact Employer

If you are interested in this job, feel free to submit your info to the employer.

Find More Computer Science Jobs
Find More Wiley Jobs

+ Other Jobs by Wiley