Job Details

Client is seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of our enterprise-wide data platform. The ideal candidate will be responsible for creating a scalable, secure, and high-performance data ecosystem that enables advanced analytics, machine learning, and business intelligence capabilities. The role involves aligning solutions with Client operating procedures and Architecture principles, working closely with internal technical teams, and performing proof of concepts to evaluate non-functional requirements. The candidate will provide mentorship and technical guidance, transitioning work to other Client operational teams.

Key Responsibilities:

1. Design and architect comprehensive data solutions using Azure Databricks, Azure Data Lake, Azure Event Hub, and related Azure data services.
2. Develop and implement data Lakehouse architecture patterns supporting both batch and real-time data processing.
3. Create robust data ingestion pipelines leveraging Azure Databricks, Azure Data Factory, and other ETL technologies.
4. Establish best practices for data governance, security, and compliance across the data platform.
5. Design and implement advanced analytics and machine learning workflows using Databricks.
6. Create and maintain infrastructure-as-code (IaC) templates for reproducible and scalable data platform deployments.
7. Provide architectural guidance and mentorship to data engineering and analytics teams.
8. Ensure alignment of data platform architecture with overall enterprise architecture and business objectives.
9. Develop prototypes to prove out designs and delivery patterns.
10. Foster collaboration with cross-functional teams including Development, Operations, Compliance, Security, etc.
11. Understand and apply principles, theories, and concepts related to the profession and Client's culture.
12. Provide mentoring and guidance to other team members.

Additional Job Duties:

1. Convey team goals effectively through strong written and verbal communication skills.
2. Execute end-to-end testing for various non-functional requirements.
3. Resolve design and development issues in creative and effective ways.
4. Research, evaluate, and incorporate new technology and tools to enhance product development and platform operations.
5. Provide advice and recommendations to management based on experience.
6. Expand technical proficiency in key technical areas.

Experience:

Minimum of 5 years related work experience beyond degree requirements.

Desired Skills:

1. Advanced proficiency with Azure Databricks and the ability to provide best practices for a large organization.
2. Expert-level knowledge of:

• Python
• SQL
• Spark (PySpark) and Delta Lake

Education:

1. Bachelor's Degree in Computer Science or a related field (Required).
2. Alternatively, a combination of education and experience that provides equivalent knowledge is acceptable.

Work Location:

1. This position can be performed remotely from anywhere in the US.
2. May be required to be on-site at the Taylor facilities occasionally based on business needs as determined by the Manager.
3. Work arrangements are subject to business needs and may change accordingly.