Senior Data Engineer

How you'll help us Keep Climbing (overview & key responsibilities)

At Delta, we connect people and cultures, power extraordinary experiences, and shape the future of travel. Here, you'll work on complex challenges on a global scale, using augmented intelligence to build emerging technologies, amplify human potential and continuously grow the skills that keep you at the forefront of what's next. It is an exciting time to be part of Information Technology as Delta continues to modernize how operational data is accessed, trusted, and used to improve reliability, efficiency, and decision-making.

Our team is focused on creating the data foundation, access patterns, and intelligence capabilities needed to connect operational signals across ITSM, telemetry, application, and infrastructure platforms The Senior Data Engineer role supports IT Service Assurance by leading the design and delivery of secure, scalable, and AI-ready data solutions that strengthen operational intelligence, service reliability, automation, and decision-making.

Operating with a high degree of autonomy, this position translates IT operational priorities into trusted data products, reusable integration patterns, and modern engineering capabilities that connect signals across ITSM, CMDB, observability, application, and infrastructure platforms.

This role partners across technology, operations, analytics, security, and architecture teams to improve data quality, enable self-service access, advance automation, and create the foundation for reporting, predictive insights, semantic search, and intelligent enterprise solutions

Lead independently across complex data initiatives, proactively identifying improvement opportunities, shaping solutions, and driving delivery from concept through production.

Design and modernize secure, scalable AWS data platforms, pipelines, Lakehouse patterns, and reusable data products aligned to enterprise architecture and governance standards.

Provide senior technical leadership for ingestion, transformation, modeling, metadata, quality, lineage, performance optimization, and production reliability.

Build and optimize batch, incremental, event-driven, and near-real-time integrations across APIs, databases, files, ServiceNow, CMDB, observability, telemetry, and enterprise platforms.

Develop efficient, maintainable Python and SQL for data pipelines, orchestration, transformations, analytics layers, APIs, automation, and reusable engineering frameworks.

Establish trusted data products through automated controls for completeness, accuracy, freshness, reconciliation, monitoring, alerting, and exception handling.

Advance AI-ready architectures that prepare structured and unstructured data for Amazon SageMaker, Amazon Bedrock, LLMs, RAG, vector databases, semantic search, agents, and MCP-enabled solutions.

Design retrieval, embedding, indexing, enrichment, and context-management patterns that improve data usability, discoverability, and responsible AI outcomes.

Drive operational excellence through CI/CD, Infrastructure as Code, automated testing, source control, code reviews, runbooks, production monitoring, and resilient deployment practices.

Apply security, privacy, identity, access control, encryption, retention, data classification, and responsible AI requirements across the data lifecycle.

Influence stakeholders, mentor engineers, promote reusable standards, evaluate emerging technologies, and guide modernization efforts that improve scalability, automation, cost, and business value.

What you need to succeed (minimum qualifications)

Is at least 18 years of age and has authorization to work in the United States.

Possesses a high school diploma, GED, or high school equivalency.

Embraces diverse people, thinking, and styles.

Consistently prioritizes safety and security of self, others, and personal data.

Strong analytical and interpersonal skills, with the ability to develop effective relationships and influence without direct authority.

Superior written and verbal communication skills, as well as executive presence to be an effective representative of the business across the organization.

7+ years of IT experience, including 4+ years of hands-on experience designing, developing, and supporting enterprise data pipelines, data platforms, lakehouses, or analytical data solutions.

Demonstrated experience designing and implementing cloud-based data lake, data warehouse, or lakehouse architectures using AWS services.

Hands-on experience with Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift, AWS Lambda, Amazon EventBridge, AWS Lake Formation, AWS IAM, and related data services.

Advanced proficiency in Python and SQL for ingestion, transformation, automation, analysis, data services, and performance optimization.

Experience designing scalable ETL and ELT pipelines integrating APIs, databases, structured files, semi-structured content, and cloud services.

Strong experience with dimensional, relational, normalized, denormalized, and analytics-oriented data modeling, including layered raw, curated, published, and analytics architectures.

Strong understanding of incremental ingestion, change-data capture, watermarking, deduplication, record versioning, historical tracking, late-arriving data, schema evolution, and reconciliation

Experience implementing metadata, lineage, cataloging, source-to-target mapping, data-quality testing, freshness monitoring, logging, alerting, and exception handling.

Working knowledge of Amazon SageMaker, Amazon Bedrock, foundation models, LLM application architectures, RAG, embeddings, vector search, semantic retrieval, and AI-ready data preparation.

Experience preparing structured and unstructured data through parsing, chunking, enrichment, embedding generation, indexing, access filtering, and context management.

Strong understanding of data governance, data classification, privacy, encryption, retention, identity, authorization, and secure handling of personal and sensitive information.

Experience with Amazon Redshift development concepts, including SQL optimization, workload performance, distribution and sort strategies, materialized views, stored procedures, access controls, and columnar formats such as Parquet.

Experience with Git, automated testing, CI/CD, Infrastructure as Code such as AWS CDK, or CloudFormation.

Ability to communicate complex technical concepts, data limitations, architecture decisions, AI data requirements, and implementation plans to technical and business audiences.

Demonstrated ability to lead technical design, mentor engineers, conduct code and architecture reviews, establish reusable standards, and manage work across concurrent Agile initiatives.

Consistent commitment to safety, security, responsible AI, operational reliability, and protection of personal and sensitive data.

What will give you a competitive edge (preferred qualifications)

AWS Certified Data Engineer, AWS Certified Solutions Architect, AWS Certified Machine Learning Engineer, or another relevant AWS certification.

Hands-on experience with Amazon SageMaker, Amazon Bedrock, Bedrock Knowledge Bases, Bedrock Agents or AgentCore, Amazon OpenSearch Service, foundation models, embedding models, and production generative AI architectures.

Experience implementing enterprise RAG solutions, semantic search, hybrid retrieval, reranking, retrieval evaluation, grounding, citations, and AI-powered knowledge repositories.

Experience designing or integrating vector databases and vector-capable platforms such as Amazon OpenSearch Service, PostgreSQL with pgvector, or comparable technologies.

Experience with prompt engineering, model evaluation, guardrails, hallucination reduction, responsible AI controls, observability, and secure AI deployment patterns.

Deep knowledge of ServiceNow data, APIs, table structures, reference fields, ITSM processes, configuration management data, and CMDB relationships.

• Experience integrating observability and operational platforms such as Dynatrace, PagerDuty, Amazon CloudWatch, ServiceNow, Sumo Logic, or similar enterprise services
• Experience implementing OAuth 2.0, OpenID Connect, identity federation, role-based access controls, and document- or row-level authorization for data and AI retrieval
• Experience with automated data testing and quality frameworks such as AWS Glue Data Quality, Great Expectations, dbt tests, or comparable technologies
• Demonstrated history of leading data modernization, cloud migration, operational intelligence, platform engineering, generative AI, or AI-enablement initiatives