CrawlJobs Logo

Senior Data Engineer

https://www.atlassian.com Logo

Atlassian

Location Icon

Location:
India, Bengaluru

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided

Job Description:

Collaborate with engineering and TPM leaders, developers, and process engineers to create data solutions that extract actionable insights from incident and post-incident management data, supporting objectives of incident prevention and reducing detection, mitigation, and communication times. Work with diverse stakeholders to understand their needs and design data models, acquisition processes, and applications that meet those requirements. Add new sources, implement business rules, and generate metrics to empower product analysts and data scientists. Serve as the data domain expert, mastering the details of our incident management infrastructure. Take full ownership of problems from ambiguous requirements through rapid iterations. Enhance data quality by leveraging and refining internal tools and frameworks to automatically detect issues. Cultivate strong relationships between teams that produce data and those that build insights.

Job Responsibility:

  • Collaborate with engineering and TPM leaders, developers, and process engineers to create data solutions that extract actionable insights from incident and post-incident management data, supporting objectives of incident prevention and reducing detection, mitigation, and communication times.
  • Work with diverse stakeholders to understand their needs and design data models, acquisition processes, and applications that meet those requirements.
  • Add new sources, implement business rules, and generate metrics to empower product analysts and data scientists.
  • Serve as the data domain expert, mastering the details of our incident management infrastructure.
  • Take full ownership of problems from ambiguous requirements through rapid iterations.
  • Enhance data quality by leveraging and refining internal tools and frameworks to automatically detect issues.
  • Cultivate strong relationships between teams that produce data and those that build insights.

Requirements:

  • BS in Computer Science or equivalent experience with 8+ years as a Senior Data Engineer or similar role
  • 10+ Years of progressive experience in building scalable datasets and reliable data engineering practices.
  • Proficiency in Python, SQL, and data platforms like DataBricks
  • Proficiency in relational databases and query authoring (SQL).
  • Demonstrable expertise designing data models for optimal storage and retrieval to meet product and business requirements.
  • Experience building and scaling experimentation practices, statistical methods, and tools in a large scale organization
  • Excellence in building scalable data pipelines using Spark (SparkSQL) with Airflow scheduler/executor framework or similar scheduling tools.
  • Expert experience working with AWS data services or similar Apache projects (Spark, Flink, Hive, and Kafka).
  • Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team.
  • Well versed in modern software development practices (Agile, TDD, CICD)

Nice to have:

  • Demonstrated ability to design and operate data infrastructure that deliver high reliability for our customers.
  • Familiarity working with datasets like Monitoring, Observability, Performance, etc.

Additional Information:

Job Posted:
March 19, 2025

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:
Welcome to CrawlJobs.com
Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.