Lead Azure Data Engineer Job at Orion Innovation, Montvale, NJ

L0JmOEtxbFRiNDloRkFsakorUnRJbUtqQUE9PQ==
  • Orion Innovation
  • Montvale, NJ

Job Description

Title: Lead Data Engineer

A large-scale project that would process 100s TB of ERP data and other relevant data. Auditors would use this processed information to opine on the client engagements and provide value-added services to their clients. The project currently uses Azure data technologies and Databricks.

What Candidate Will Do:

  • Design, build, and manage scalable data architectures using Databricks and Azure data technologies.
  • System Design and implement modern data lake house solutions, ensuring seamless data integration, storage, and processing.
  • Collaborate with cross-functional teams to gather and translate business requirements into effective technical designs.
  • Ensure data quality and governance practices are implemented throughout the data lifecycle.
  • Optimize data workflows for performance, reliability, and security using Azure one lake, Databricks Workflow and Databricks Compute, Unity catalog, Azure Synapse and PowerBI.
  • Develop and enforce best practices in data modeling, pipeline design, and storage architecture.
  • Conduct regular assessments of data systems to identify and address performance bottlenecks or areas for improvement.

Must-Have:

  • 12+ Years of total IT experience in Data engineering and Data warehouse project development.
  • 6+ years of hands-on experience with Azure Databricks and expertise in PySpark & Python development .
  • Proven expertise in designing and managing scalable data architectures. Experience in Databricks Serverless is plus.
  • 6+ years’ experience with Azure Synapse, Data Factory, and other Azure data technologies.
  • 8+ years’ experience in data modeling,
  • 6+ Years experience with Data pipelines design and implementation, and cloud storage architecture.
  • Deep understanding of data quality and governance practices, and hands-on with Data quality and governance using Unity catalog or azure purview.
  • Ability to collaborate with cross-functional teams and translate business requirements into technical designs.

Nice-to-Have:

  • Familiarity with other modern data processing tools and technologies.
  • Previous experience in a similar role working with large-scale data systems.

Job Tags

Similar Jobs

Guthrie

Care Companion - Virtual Onsite - Pulse Center - Full Time Job at Guthrie

 ...High School education or GED required. Must be comfortable using computers and learning new technology. Experience: Care Companion or Sitter experience preferred, but not required. Professional communication and teamwork required. Essential Functions: 1.... 

Resource Informatics Group Inc

EDI Lead Job at Resource Informatics Group Inc

 ...Job Description Job Description Role: - EDI Lead Location: - Hartford, CT Duration: - 12 Months Visa: - Any Interview: - Phone/Skype Job Description:- Application Integration - 10 years EDI standards (X12, EDIFACT etc.) B2B Integration... 

PACE Southeast Michigan

Nursing Assistant - Community Living Associate - Windemere 7 am - 7:30 pm Job at PACE Southeast Michigan

 ...health center while under the supervision of the manager, registered nurse, or lead CLA. The CLAs responsibilities for each participant...  ...manager, assigned RNCM/clinical team, and lead CLA. ~ The CLA assists in providing and/or maintaining optimal physical and emotional... 

Compeer Financial

Lending Client Coordinator Job at Compeer Financial

 ...Empowered to live. Inspired to work. Compeer Financial is a...  ...model - up to 50% work from home Flexible schedules including...  ...servicing. Provides coordination for office operations, including...  ...renewals, ensuring accuracy in all transactions. Coordinates and tracks the... 

Clark Transfer

Owner Operators Job at Clark Transfer

 ...four weeks, including orientation (Singles/Teams)~$1.60/gallon or lower fuel price guarantee (average, after fuel surcharge)~ Company Supported Owner Operator health insurance program ~ All miles paid, plus detention pay ~ Up to $5,000 referral bonus ~ Paid fuel...