This position is with the Enterprise System Management (ESM) department. The position is with a team responsible to manage our company' s IT inventory and assets data. This data is critical for regulatory, operational, and efficiency purposes. 

Our goals are: Sourcing IT inventory and machine data (e.g. logs, alerts) in a timely manner. Storing the data in scalable repositories

Providing analytics tools to analyze and derive insights from the data.

Providing data access APIs so that other applications can consume the data and insights.

Project Description: We are launching a project to collect, correlate, summarize, and analyze large amounts of network and application monitoring data to be able to determine run-time dependencies and data flows, to be used for risk controls, operational efficiencies, and regulatory compliance.

Position Description: 

We are looking for an experienced Data Engineer with deep understanding of large-scale data handling and processing best practices in a mixed on-prem and cloud environment to help us build scalable systems. It is vital that we deliver robust solutions that ensure reliable access to data with a focus on quality and availability to facilitate analytics and user understanding. Those solutions will help bring to light actionable insights to improve business decisions and processes. Responsibilities include, but are not limited to: Collaborate with other data engineers to implement a shared technical vision

Participate in the entire software development lifecycle, from concept to release

Designing and implementing batch and streaming data pipelines capable of processing large volumes of data

Build efficient code to transform raw data into datasets for analysis

Designing and implementing a data warehouse for reporting and analytics

Designing and deploying data models and views with large datasets that meet functional / non-functional business requirements

Identifying, designing, and implementing internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability

Delivering quality production-ready code in an agile environment

Delivering test plans, monitoring, debugging and technical documents as a part of development cycle

Creating data tools for analytics and working with stakeholders across all departments to assist with data-related technical issues and supporting their data infrastructure needs

We are looking for engineers who can not only tackle technical problems, but also collaborate and evangelize best practices across teams. We value engineers who can bring fresh ideas and are willing to own solving problems all the way to a solution.

Qualifications (required):

4+ years of data engineer experience with experience with data warehouse architecture and hands-on experience of data modeling design, ETL pipeline implementation

Deep understanding of data architecture data engineering, data analysis, data modelling, and reporting

Experience with cloud-based data warehouses such as Snowflake or similar

Advanced working SQL knowledge and experience working with relational databases

Kafka or equivalent streaming platform experience

Strong experience with Python

Strong general programming proficiency

Can learn anything you don't already know to get the job done

Excellent communication skills and ability to understand complex technology systems

Good active listening skills, the ability to empathize with stakeholders, and other engineers

Comfortable with ambiguity; you enjoy figuring out what needs to be done

Ability to quickly learn new technologies

Experience with GIT

Experience with Agile methodology

Comfortable with Linux operating system and command-line tools

Qualifications (nice-to-have):Strong experience with Snowflake

Experience working in a hybrid cloud and on-prem environment and/or with migrations from on-prem to cloud.

Experience with continuous integration and automated testing tools and methodologies

Experience with server less architecture and microservices.

Experience JIRA

Experience with SCRUM

Experience with of Tableau, QlikView, PowerBI or similar tools

Experience with Machine Learning