Databases, pipelines and data platforms.
Peter and the team utilized B-trees and inverted search indices to handle message threading and unread counts right from the start. This architecture supported fast lookups despite the constraints of early distributed systems.
▶ 12:18
Distributed databases with Peter Mattis
You can use Zerobus, which is an API that handles high-rate, bursty, or tiny-row data ingestion directly into your open lake without requiring a traditional message bus or buffering. Additionally, Spark Real-time Mode enables low-latency streaming operations down to tens of milliseconds.
▶ 17:21
Data + AI Summit Keynote 2026 | Day 1
Memory enables agents to be reflective, interactive, proactive, and autonomous, mimicking human intelligence where recall determines capability. Without it, applications remain stateless and heavily reliant on repetitive prompt engineering.
▶ 3:15
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
Superbase transitioned from dashboard and API models to a code-first approach emphasizing CLI tools and infrastructure as code. This ensures that features available in dashboards also work natively in sandboxes where agents operate.
▶ 3:00
What Happens When Everyone Can Build Software
Traditional replication required storing three identical copies of data, which consumed excessive storage space. Reed-Solomon erasure coding allowed the system to store data much more efficiently while still allowing data reconstruction if multiple chunks were lost.
▶ 19:24
Distributed databases with Peter Mattis
Apache Iceberg v3 support allows managed Iceberg tables to share data files with Delta tables without requiring any file rewrites. This establishes a unified data layer where data laid out on disk looks identical regardless of the format.
▶ 20:56
Data + AI Summit Keynote 2026 | Day 1
Memory management involves generation, storage, retrieval, integration, updating, and specialized deletion mechanisms like forgetting. The retrieval phase is particularly crucial for pulling relevant structured memories into the context window rather than stuffing all data in.
▶ 6:37
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
Open-source stacks allow agents to inspect source code directly when encountering errors, avoiding frustrating brick walls. This transparency also appeals to enterprises that need to audit security before adopting a vendor.
▶ 3:56
What Happens When Everyone Can Build Software
Standard balanced binary trees cause frequent cache misses because every node traversal jumps to a different cache line. B-trees improve spatial locality and reduce pointer overhead by packing multiple sorted items into a single node. This structure allows more compact storage and faster linear scans or binary searches over items.
▶ 31:08
Distributed databases with Peter Mattis
Lakebase Postgres allows you to instantly branch petabyte-scale databases in less than a second using a copy-on-write mechanism on lake storage. This allows AI agents to safely experiment, test, and roll back changes without duplicating underlying storage.
▶ 26:00
Data + AI Summit Keynote 2026 | Day 1
MongoDB provides a flexible document data model that adapts to various data structures alongside multiple retrieval capabilities like graph, vector, text, and geospatial queries in one database. This allows developers to store diverse memory types and build custom memory management solutions.
▶ 8:36
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
Domain experts and non-technical builders leverage AI models to create bespoke personal software and niche solutions without needing software engineers. This surge in builder activity has accelerated signups and database creation rates significantly.
▶ 2:26
What Happens When Everyone Can Build Software
Distributed storage systems like Colossus target large, immutable, append-only files such as 64-megabyte or gigabyte chunks. In contrast, distributed databases like Spanner manage relational tables with billions of typed rows and columns requiring frequent updates. Databases are typically built on top of distributed storage systems, using immutable files and log-structured merge trees to handle updates.
▶ 44:53
Distributed databases with Peter Mattis
The Unity AI Gateway serves as a single pane of glass to manage all AI spend, models, and MCP tools across an organization. It provides granular observability dashboards, budget limits and rate-limiting down to the individual level, and enforces security and compliance guardrails.
▶ 34:09
Data + AI Summit Keynote 2026 | Day 1
Persona memory helps assign a consistent personality to AI systems to make them more believable and human-like. Developers can model this personality data inside MongoDB to foster better relationships with users.
▶ 10:09
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
Supabase shifted to a code-first and Git-native architecture so that database capabilities match infrastructure as code. This allows AI agents to interact with the database entirely through sandboxes rather than relying on traditional dashboards.
▶ 21:37
What Happens When Everyone Can Build Software
Basic hash-based sharding maps keys to a fixed number of shards using a hash function, which makes resharding and growing the table an onerous process. In contrast, systems like Cockroach DB use contiguous key spans across a single big key space, partitioning them with a B-tree-like index structure. This avoids full table remappings when adding nodes and places the sharding burden on the database rather than the developer.
▶ 52:40
Distributed databases with Peter Mattis
Live retrieval loops force agents to perform random walks through massive enterprise data corpora, leading to high token costs and runtimes of 10 to 15 minutes. Furthermore, this ad-hoc search often results in poor accuracy because the agent lacks proper upfront context.
▶ 37:52
Data + AI Summit Keynote 2026 | Day 1
Instead of stuffing all tool schemas into the limited context window, developers can store JSON schemas of tools inside MongoDB. Just before hitting the LLM, the system queries the database to retrieve only the relevant tool via search.
▶ 10:45
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
You should re-engineer error messages to include explicit instructions or documentation links on how to fix the error. This enables coding agents to autonomously remedy issues and complete self-healing loops without human intervention.
▶ 30:22
What Happens When Everyone Can Build Software
Consensus protocols like Raft cannot function with only two replicas because a crash leaves it impossible to determine if a write successfully occurred on the other node without data loss or rollback ambiguities. Having at least three replicas allows the system to perform consensus checks during recovery by reading from a quorum. Normal reads can still hit a single replica, while writes go to all three to ensure durability.
▶ 1:00:44
Distributed databases with Peter Mattis
Genie Ontology automatically connects to structured and unstructured sources like the lakehouse, Google Drive, and SharePoint in the background to build a comprehensive knowledge graph. It uses an algorithm called OntoRank, inspired by PageRank, to index snippets and incorporate human-curated semantics.
▶ 39:10
Data + AI Summit Keynote 2026 | Day 1
When an agentic system executes steps that fail, that failure can be stored in the database as a learning experience. This historical failure data is then pulled into subsequent execution steps to prevent the LLM from repeating the same mistake.
▶ 12:15
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
As agents accumulate massive working context, such as tens of gigabytes of markdown files, standard context windows fall short. Using tools like PG vector and Postgres allows agents to perform efficient vector searches and traverse graph data structures to pull relevant context.
▶ 34:08
What Happens When Everyone Can Build Software
Managing multiple agent sessions involves handling a cognitive overhead of about five to ten concurrent sessions. Depending on the complexity of the task, these sessions can orchestrate anywhere from three to over a hundred sub-agents running in parallel to perform research or implementation. This allows developers to act more like hands-on architects scaling their throughput.
▶ 1:20:42
Distributed databases with Peter Mattis
Genie 1 connects to multiple enterprise data sources, including email, calendar, Salesforce, and Databricks, to deliver comprehensive customer profiles. In benchmarks on novel enterprise questions, coding agents only succeeded half the time and hit time limits, whereas Genie 1 provided grounded and accurate results.
▶ 1:02:45
Data + AI Summit Keynote 2026 | Day 1
MongoDB acquired Voyage AI to integrate their embedding models and re-rankers directly into MongoDB Atlas. This integration aims to reduce AI hallucinations and eliminate the need for developers to write custom chunking strategies.
▶ 14:09
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
The most effective approach is rigorous dogfooding where you use coding tools and AI loops directly on your own products. Encountering your own friction and gaps provides the best feedback for improving agent workflows.
▶ 47:34
What Happens When Everyone Can Build Software
To prevent regressions, developers must provide firm guidance and enforce rigorous testing techniques like property-based testing and deterministic simulation testing. Agents can sometimes be lazy with tests, so humans need to instill discipline and use multiple agents for adversarial security reviews on every commit. This ensures that the high volume of generated code still meets production standards.
▶ 1:25:00
Distributed databases with Peter Mattis
Genie Ontology extracts expressions, relationships, and expertise from underlying data assets, running them through an onto-rank algorithm to inject authoritative context. This approach yields a consistent 30 percentage point improvement in accuracy while cutting runtime in half compared to leading agents.
▶ 1:07:58
Data + AI Summit Keynote 2026 | Day 1
Historical neuroscience research on the visual cortex of cats inspired convolutional neural networks for object and face detection. Today, agent architects collaborate directly with neuroscientists studying human memory to apply nature's blueprint to advanced artificial intelligence systems.
▶ 15:59
Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB
Code review is shifting because developers spend less time scrutinizing code as agents improve and guardrails are put in place. While junior-level code still requires high scrutiny, senior developers rely on automated verification and guardrails—such as checking decompiled code for zero-overhead abstractions—rather than manual line-by-line review. Eventually, this leads to trusting compilers and automated tools to verify low-level output.
▶ 1:27:44
Distributed databases with Peter Mattis
PepsiCo deployed Genie within its supply chain AI brain platform called Spend Wise to handle direct and indirect procurement insights. Within the first few weeks of launch, users drove roughly 30,000 queries, shifting away from traditional dashboards toward conversational data engagement.
▶ 1:28:54
Data + AI Summit Keynote 2026 | Day 1
Domain experts act like amplified sorcerers when using AI, leveraging their deep knowledge of system architecture to steer models effectively. By knowing the right architectural patterns—such as storage and networking layers for distributed databases—experts can extract high-quality, complex implementations rapidly. Without that underlying domain expertise, the generated output tends to be hollow and lacking substance.
▶ 1:33:47
Distributed databases with Peter Mattis
LakeFlow replaces legacy Spark with open declarative pipelines that unify batch and streaming workloads using both SQL and Python. Because the frameworks are open and declarative, developers can run them anywhere, while enabling AI agents to easily write the code.
▶ 1:34:38
Data + AI Summit Keynote 2026 | Day 1
Zerobus Ingest is a fully managed service that is 100% wire compatible with Kafka, allowing existing producers to land data directly into the lakehouse in open formats. It eliminates small file issues and processes millions of small files at high speeds without requiring complex cluster management.
▶ 1:37:16
Data + AI Summit Keynote 2026 | Day 1
Traditional lake houses hit a latency wall around one second for stringent workloads, forcing companies to maintain separate serving stacks. Raiden flips traditional database engineering by training a machine learning model on exabytes of execution traces to dynamically dispatch the right algorithms at runtime. This enables Lakehouse RT to deliver millisecond performance and massive concurrency directly on governed lake data.
▶ 1:54:49
Data + AI Summit Keynote 2026 | Day 1
Lakebase decouples storage and compute by moving data storage into the lake while introducing specialized services to maintain performance. Safekeepers handle writes using the Paxos consensus protocol for low-latency transactions, while page servers deliver low-latency reads. This architecture enables fully managed serverless Postgres instances with fast provisioning and scaling.
▶ 2:11:53
Data + AI Summit Keynote 2026 | Day 1
Lakebase offers rapid provisioning in under 500 milliseconds and supports instant branching to create isolated dev, test, and staging environments. Developers and agents can also use snapshot restore via a simple click to test code changes or schema modifications. If an agent's work is unsatisfactory, the database can be instantly rolled back to the previous snapshot.
▶ 2:13:11
Data + AI Summit Keynote 2026 | Day 1
Lakebase provides fully managed cross-cloud disaster recovery to ensure continuous business operations during major cloud outages. Users can configure replication across different cloud providers, such as AWS and Azure. In the event of an outage, the system can instantly fail over to the secondary cloud.
▶ 2:17:32
Data + AI Summit Keynote 2026 | Day 1
Mastercard built a shared foundation using Lakebase to support thousands of issuing banks on a single platform. This architecture ensures strict data isolation and complies with data residency requirements to maintain client trust. Simultaneously, it allows insights to be consolidated securely so agents can reason across data without leaking sensitive multi-tenant details.
▶ 2:19:37
Data + AI Summit Keynote 2026 | Day 1
Mastercard leverages performance pools within Lakebase to maintain strict data separation for thousands of issuing banks while enabling continuous insights. This architecture allows them to scale horizontally while ensuring secure, anonymized data consolidation.
▶ 2:21:39
Data + AI Summit Keynote 2026 | Day 1
Traditional change data capture pipelines read database binary logs to replicate changes, making them fragile and prone to data corruption or maintenance overhead. They often require external infrastructure and can fail under heavy production workloads.
▶ 2:43:32
Data + AI Summit Keynote 2026 | Day 1
LTAP stands for lake transactional analytical processing, which unifies storage by transcoding row-oriented operational data into columnar formats directly on storage services. Unlike traditional failing HTAP systems, it avoids query engine compromises while providing real-time analytics without hurting OLTP performance.
▶ 2:47:55
Data + AI Summit Keynote 2026 | Day 1
Storage services use their underutilized CPUs to transcode row data into columnar Delta and Iceberg formats during local disk writes. Because columnar formats offer high compression ratios, this conversion minimizes data volume and leverages I/O bound resources efficiently.
▶ 2:46:02
Data + AI Summit Keynote 2026 | Day 1
Developers do not need to set up custom pipelines or change their operational workflows to use LTAP. Once enabled, Lakebase tables automatically appear in Unity Catalog in Delta and Iceberg formats for real-time querying.
▶ 2:49:25
Data + AI Summit Keynote 2026 | Day 1
No answer mentions that.