Job Details

This role is categorized as hybrid. This means the successful candidate is expected to report to Austin, TX, Warren, MI, or Roswell, GA three times per week, at minimum.

The Role

The Data Delivery team is seeking a Senior Software Engineer with a proven track record building large scale data streaming platforms. The ideal candidate will play a crucial role in developing and maintaining data-related systems comprising high throughput data streams, data curation and data storage / file system architectures, data security and governance, as well as inline inferencing and AI algorithms.

What You'll Do

• Develop a scalable data platform to support continuing increases in data volume and complexity.
• Develop data pipelines to ingest, process, and curate large volumes of data using Spark Structured Streaming or similar technologies.
• Support the infrastructure required for optimal ingestion, transformation, and delivery of data.
• Design and implement rigorous data validation, governance, and quality checks to uphold data integrity.
• Mentor & Support junior engineers by providing guidance, coaching and educational opportunities.

Your Skills & Abilities (Required Qualifications)

• Bachelor's degree in computer science or a related field
• At least 5 years of hands on experience with Big Data Tools: Hadoop, Spark, Kafka, etc.
• You have mastery with databases - Advanced SQL and NoSQL databases, including Postgres and Cassandra
• Proven cloud experience and strong familiarity with at least one cloud platform (Microsoft Azure - preferred , AWS, GCP).
• Experience in building distributed data processing micro-services and high-quality, scalable data products.
• Stream-processing systems: Storm, Spark-Streaming, etc.
• 5 or more years with object-oriented/object function scripting languages: Python, Java, Scala, etc.
• Ability to tackle problems quickly and completely
• Ability to identify tasks which require automation and automate them
• A demonstrable understanding of networking/distributed computing environment concepts
• Ability to multi-task and stay organized in a dynamic work environment
• Good understanding and experience with CI/CD practices.

What Can Give You a Competitive Advantage (Preferred Qualifications)

• Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Azure cloud services