Principal Engineer, Tableflow

We’re not just building better tech. We’re rewriting how data moves and what the world can do with it. With Confluent, data doesn’t sit still. Our platform puts information in motion, streaming in near real-time so companies can react faster, build smarter, and deliver experiences as dynamic as the world around them. It takes a certain kind of person to join this team. Those who ask hard questions, give honest feedback, and show up for each other. No egos, no solo acts. Just smart, curious humans pushing toward something bigger, together. One Confluent. One Team. One Data Streaming Platform. ABOUT THE ROLE: As a Principal Engineer, you will be a key technical leader shaping the long-term architecture, scalability, and reliability of the Tableflow project and the broader Confluent platform. You will focus on advancing key layers of Tableflow, including its multi-tenant, highly available compute infrastructure and core components responsible for table materialization and background maintenance. You will own company-level technical domains, set direction for critical initiatives, and influence both engineering practices and product strategy. This role requires operating with autonomy in highly ambiguous spaces, making decisions that balance technical depth with business impact to ensure high availability and scalability for background storage tasks across all of Confluent Cloud. WHAT YOU WILL DO: - Technical Strategy & Leadership: Define and drive company-level technical strategy for critical systems, ensuring long-term innovation for Tableflow’s storage and metadata engines. - Metadata Architecture & Lifecycle Management: Architect and evolve a high-performance, metadata management system that supports low latency updates and efficient query planning. - Storage Engine Innovation: Design and implement innovative strategies for table schematization, materialization, and compaction at a massive scale. - Platform Engineering: Build and maintain multi-tenant, highly available infrastructure for background computational tasks, ensuring the platform is optimized for scalability, reliability, and resilience. - Architectural Excellence: Establish and evangelize engineering standards, architectural patterns, and technical frameworks adopted across the organization. - Cross-Functional Collaboration: Partner with executives, product management, and engineering leaders to align technical vision with business objectives. - Mentorship & Culture: Mentor senior and staff-level engineers, growing future technical leaders and shaping a culture of engineering excellence and open-source contribution. - Open-Source & External Representation: Lead contributions to relevant open-source projects like Apache Iceberg and represent Confluent externally through technical talks and thought leadership. WHAT YOU WILL BRING: - Deep Expertise: A proven track record of leading the architecture and delivery of large-scale distributed storage systems with company-level impact. - Distributed Systems Mastery: Comprehensive knowledge of fault tolerance, consistency, scalability, and cloud-native platforms. - Strategic Influence: Ability to navigate extreme ambiguity, reduce systemic complexity, and anticipate industry trends to design systems for the next 3–5 years. - Leadership Presence: Strong ability to influence executives and stakeholders while fostering a collaborative environment across multiple teams. - Execution: Demonstrated ability to move from high-level vision to detailed technical implementation and delivery. WHAT GIVES YOU AN EDGE: - Industry Leadership: Recognized industry thought leader through open-source contributions (e.g., Apache Kafka, Apache Iceberg, Apache Flink) or published research. - Advanced Domain Knowledge: Expertise in storage internals, runtime systems, or large-scale data infrastructure supporting mission-critical workloads. - Cloud Infrastructure: Deep experience with public clouds (AWS, Azure, or GCP) and expertise in resource management and multi-tenancy in complex cloud environments. READY TO BUILD WHAT'S NEXT? LET’S GET IN MOTION. COME AS YOU ARE Belonging isn’t a perk here. It’s the baseline. We work across time zones and backgrounds, knowing the best ideas come from different perspectives. And we make space for everyone to lead, grow, and challenge what’s possible. We’re proud to be an equal opportunity workplace. Employment decisions are based on job-related criteria, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other classification protected by law.