Log In

Don't have an account? Sign up now

Lost Password?

Sign Up

Software

The Joint Commission Hiring for Lead Azure Databricks Engineer – Hybrid Opportunity at Oakbrook Terrace, IL Full Time

Overview:

The Lead Azure Databricks Engineer is responsible for designing and implementing Big Data solutions using the Microsoft Azure platform.

You must have at least five years of hands-on Azure expertise in designing and developing scalable, multi-tenant Big Data and machine learning products using Azure Data Lake Gen2, Azure Databricks, Setting up containers; data pipelines for near real-time data analytics and machine learning; and near real-time Azure SQL cloud-based data warehouses.
You must also be an expert with 10 years of hands-on experience in designing, implementing and delivering large-scale, near real-time data warehouses into production.
The Senior Azure Databricks engineer is expected to evangelize and educate others on engineering design and development standards. Heshe is expected to function as a productive member of a team, working and communicating proactively with engineering peers, technical leads, project managers, product owners and resource managers.
The ideal candidate is an exceptional critical thinker who is adept at solving complex problems and designing and developing elegant business solutions.
TJCCW

Responsibilities:

    • Design and develop data pipelines
    • Design and develop Azure ML model execution architecture using Databricks
    • Design and develop methods for handling third-party data for use within ML models
    • Evangelize engineering design and development standards
    • Act as a key contributor to the design and development lifecycle of analytic applications utilizing Microsoft Azure and BI technology platforms
    • Participate including daily stand-ups, sprint planning, retrospectives and product demonstrations
    • Produce efficient and elegant code that meets business requirements
    • Author unit tests that adhere to code coverage guidelines
    • Proactively communicate progress, issues and risks to stakeholders
    • Accurately estimate assignments
    • Create and maintain technical documentation
    • Mentor less experienced engineers
    • Contribute to the growth and maturity of the Software Engineering group
    • Perform other related duties as directed.

Qualifications:

    • 5 years of hands-on experience designing and implementing multi-tenant solutions using Azure Databricks, data pipelines for near real-time data warehouse and machine learning solutions
    • 10 years of extensive, hands-on experience designing and implementing large-scale, multi-tenant distributed data architecture for BI and OLTP systems
    • 10 years of advanced hands-on experience designing and implementing large-scale, multi-tenant data pipelines (TB or PB in size), including Change Data Capture (CDC) solutions for structured, semi-structured and unstructured data in batch and real-time environments
    • 5 years of hands-on experience in Azure services including Azure SQL Database, Azure Databricks, Azure Analysis Services, Azure ML Services, and Azure Synapse Analytics
    • 5 years of hands-on experience with data integration using ETL/ELT tools (e.g., SSIS, Azure Data Factory, Airflow)
    • Advanced knowledge of Python programming
    • Expertise in data modeling
    • Broad experience with multi-tenant data architecture and implementation across different data stores (e. g., Azure Data Lake GEN2, Azure SQL Data Warehouse, Azure Blob Storage, HDFS), messaging systems (e. g., Azure Event Hubs, Apache Kafka), and data processing engines (e. g., Azure Data Lake Analytics, Apache Hadoop, Apache Spark, Apache Storm, Azure HDInsight)
    • Experience with data integration through APIs, web services, SOAP, and/or REST services
    • Experience using Azure DevOps and CI/CD as well as Agile tools and processes, including Git, Jenkins, Jira and Confluence
    • Additional programming experience is a plus, including experience with NET, Scala, R, Java
    • Demonstrated experience designing and implementing data pipelines for consumption by machine learning and BI solutions
    • Knowledge of SOA and Micros Services Application Architecture
    • Ability to work in a fast-paced, collaborative team environment
    • Excellent written and verbal communication skills and ability to express ideas clearly and concisely.
This job description is intended to describe the general nature and level of work performed by an employee assigned to this position. The description is not an exhaustive list of all duties, responsibilities, knowledge, skills, and abilities, and working conditions associated with this position. All requirements are subject to possible modification and reasonably accommodate individuals with disabilities.

Job Overview

  • ID: 313035
  • Date Posted:
  • Job Title: The Joint Commission Hiring for Lead Azure Databricks Engineer – Hybrid Opportunity at Oakbrook Terrace, IL