This year, Databricks introduced yet another conceptual framework: LTAP (Lake Transactional/Analytical Processing). Tracing the narrative progression from Lakebase to LTAP, let us examine precisely what architectural challenges Databricks seeks to address—and what even thornier problems this approach brings to the fore.
The foundational technology underpinning Lakebase originates from Neon. In May 2025, Databricks announced its acquisition of Neon in a transaction reported by media outlets to be worth approximately $1 billion. Neon's core architectural innovation lies in decoupling the tightly coupled compute, write-ahead logs (WAL), and data files of traditional PostgreSQL:
According to Gartner predictions, by 2028, 33% of enterprise software applications will incorporate Agentic AI, and at least 15% of routine operational decisions will be made autonomously by agents. Agentic workloads exhibit distinct operational characteristics compared to traditional core enterprise systems: high database instance counts, uncertain lifecycles, frequent development branching, strict state isolation requirements, and transient, bursty traffic patterns. Serverless execution, scale-to-zero capabilities, rapid branching, and instant point-in-time recovery align naturally with this development paradigm.
From the perspective of Databricks' product portfolio, this acquisition was exceptionally timely. Prior to Neon, Databricks encompassed data lakes, data warehousing, Spark, Databricks SQL, streaming, BI, machine learning, and AI—yet lacked an operational transactional entry point adjacent to the application layer. Neon plugged this critical capability gap, serving as a vital strategic building block as Databricks transitions from an "Analytics and AI Platform" toward an "Application Data Platform."
At the Data + AI Summit in June 2026, Databricks officially unveiled LTAP (Lake Transactional/Analytical Processing). The core value proposition of LTAP is to unify OLTP, OLAP, AI applications, and AI Agents on top of a single, governed body of data residing in the data lake. This aims to eliminate independent dual-data representations maintained via Change Data Capture (CDC), ETL pipelines, database mirroring, or sync jobs. Databricks encapsulated this vision under the slogans "One data, zero compromises, zero copies" and zero required data duplication.
Databricks' core argument unfolds in three sequential steps:
The architectural foundation enabling this conceptual pivot remains Lakebase. Because Lakebase decoupled WAL and page storage from the PostgreSQL compute instance—delegating WAL management to Safekeepers, page reconstruction to PageServers, and long-term persistence to object storage—the PageServer acts as a storage microservice that materializes and persists data into object storage. This naturally raises an architectural question:
If data ultimately resides in object storage anyway, why continue persisting it in PostgreSQL's native page format? Why not write directly to open formats like Parquet, Delta Lake, or Apache Iceberg, allowing analytical query engines to scan the storage directly?
This represents the pivotal transition from Lakebase to LTAP: packaging Lakebase (OLTP), Lakehouse (OLAP), and Unity Catalog (unified governance) into a next-generation unified data architecture tailored for the AI era.
Fundamentally, HTAP and LTAP tackle the exact same underlying problem: the historical dichotomy between transactional and analytical systems, which forces constant, expensive data movement.
However, HTAP emphasizes unification inside the database system itself. A single logical table can simultaneously maintain row-store, column-store, or hybrid replicas coordinated by a unified transaction coordinator, SQL parser, and consistency engine. YMatrix is a pioneer of this unified engine approach. Built upon the PostgreSQL ecosystem with a single SQL interface, YMatrix utilizes row storage for low-latency transactional access alongside column storage for high compression ratios, fast scans, and optimized data layouts for analytics. Paired with the mxvector vectorized execution engine, it accelerates complex analytical processing within a unified transaction, consistency, and governance framework, providing tailored physical execution paths for diverse workloads.
Conversely, LTAP emphasizes unification at the storage and governance layers. PostgreSQL continues handling OLTP transactions, while engines like Spark and Databricks SQL handle OLAP queries. The compute engines do not need to be uniform; instead, they share a single, open, and governed authoritative dataset at the storage tier.
This architectural split does not imply that LTAP is inherently superior to HTAP. Many mature HTAP platforms already offer robust compute isolation, row/column replication, and independent resource scaling. Databricks' critique of HTAP highlights real operational pain points, but intentionally oversimplifies HTAP architectures to accentuate the advantages of its own storage-centric model.
Upon closer inspection, an inherent architectural tension emerges: OLTP and Lakehouse storage profiles occupy opposite ends of the operational spectrum.
OLTP engines demand small, random reads, low-latency commits, and fast point lookups via B-tree indexes. Lakehouse formats, in contrast, are optimized for large, sequential file scans and batch writes. Merely placing data into the same object storage bucket does not magically resolve fundamental disparities in access granularity, transactional semantics, and physical memory/disk layouts. Even when Lakebase persists data to object storage, the underlying bytes remain predominantly PostgreSQL pages or Neon internal structures. Engines like Spark, Trino, and Databricks SQL cannot efficiently query this data simply because it physically resides in object storage.
Consequently, a mechanism is required to convert traditional PostgreSQL row-oriented formats into Parquet files, ensuring the data can both be losslessly reconstructed by PostgreSQL and directly scanned in a columnar layout by analytical engines.
In the official Databricks publication titled From monolith to Lakebase to LTAP: rethinking the database from storage up CTO Reynold Xin proposed a radical architecture: making open columnar files the authoritative, durable source of truth. Under this scheme, Neon and Lakehouse analytics engines share a single persisted dataset, eliminating the need to dual-maintain PostgreSQL pages and Parquet files in object storage:
This effectively inverts the traditional database storage hierarchy. In standard PostgreSQL, the Heap Page is the durable truth. In LTAP's ideal state, Parquet / Delta / Iceberg constitutes the durable truth, while Heap Pages become rebuildable transactional caches. As the PageServer materializes pages to object storage, it simultaneously performs format transcoding from PostgreSQL row representations to Parquet columnar layouts. Databricks emphasizes that this transcoding burden is borne entirely by the PageServer layer without impacting active PostgreSQL Compute nodes serving live transactional requests.
"As the PageServer materializes pages into object storage, it transcodes Postgres data from a row format into Parquet's columnar layout…"
To guarantee that original PostgreSQL pages can be perfectly reconstructed from Parquet files, two critical categories of metadata must be preserved:
Interestingly, under this model, PostgreSQL indexes are not directly transcoded into Parquet columns. Primary table data and tuple versions are Parquet-ized, whereas B-tree and other secondary indexes are served by hot cache tiers and reconstructed on-demand from the authoritative source of truth when necessary.
"Postgres indexes are not transcoded into columns"
In essence, the architecture does not magically eliminate WAL, caches, indexes, delta layers, or replicas; rather, it enforces a single authoritative persisted representation for table data, treating all other physical structures as derivable, cacheable, or reliability-oriented representations.
It is crucial to note, however, that as of July 2026, Databricks' vision of a "single columnar authoritative storage tier" remains in a phased rollout. Official documentation acknowledges that during the transition period, the system concurrently writes both row and column formats for correctness verification, with full single-source capabilities scheduled to mature over subsequent months. At present, real-time sync still relies on traditional mechanisms similar to CDC combined with synced tables.
Major enterprise database vendors—including Oracle, SingleStore, and Snowflake—have responded to the Lakebase and LTAP paradigm shift with distinct architectural viewpoints.
In its article Oracle AI Database vs Databricks Lakebase for Modern OLTP,Oracle does not contest the value of compute-storage separation, elasticity, rapid cloning, or platform consolidation. What Oracle strictly refutes is equating these infrastructure capabilities with a mature, enterprise-grade OLTP engine. Oracle's criticisms focus on deep ecosystem compatibility, long-running transaction handling, high-concurrency scale, cache warmup efficiency after failovers, cross-region disaster recovery, deep diagnostic tooling, stringent SLAs, and decades of hardened production pedigree. Oracle's core counter-argument is not whether PostgreSQL can run on a data lake, but whether a developer-centric, elastic PostgreSQL service possesses the engineering maturity required to run core financial or mission-critical OLTP transactions.
SingleStore strongly agrees with the core objectives of LTAP: eliminating data copying, minimizing data freshness lag, and tightly coupling transaction processing with analytical queries. However, SingleStore challenges Databricks' assertion that multi-engine shared storage constitutes true database unification. Separate query engines maintain distinct memory caches, query schedulers, execution runtimes, snapshot isolation boundaries, and failure modes. Furthermore, row-to-column transcoding overhead remains. Iceberg single-table snapshots do not natively resolve cross-table transactional atomic visibility for PostgreSQL transactions on the analytical query side. SingleStore maintains that the optimal architecture remains the classic HTAP path: "One dataset, one engine."
Snowflake is similarly expanding into PostgreSQL (via its acquisition of Crunchy Data) while strengthening integration between Apache Iceberg, analytics, and AI workloads. Snowflake agrees that open table formats reduce data silos and that operational data must be brought closer to analytical engines. However, Snowflake's practical approach leverages Data Mirroring to accept near-real-time data movement, while tools like pg_lake allow PostgreSQL to natively query and manage Parquet/Iceberg lake data. Heavy analytical scans are offloaded to sidecars like pgduck_server / DuckDB, allowing pg_lake to act as an Iceberg catalog directly within PostgreSQL. Unlike Databricks, Snowflake does not treat the complete elimination of data movement as an absolute architectural prerequisite.
The core disagreements among these industry leaders can be summarized as follows:
The primary value of Lakebase and LTAP lies in forcing long-standing database engineering trade-offs back into the spotlight:
LTAP: An Alternative Paradigm, Not an HTAP Replacement
HTAP and LTAP do not represent a simple dichotomy of outdated versus modern technology; they merely draw the boundary of system unification at different architectural layers. HTAP consolidates row storage, column storage, transactions, and analytical consistency inside a single database system boundary. LTAP attempts to consolidate authoritative data at an open storage layer, enabling specialized external engines to compute independently. The optimal choice depends entirely on business priorities: ultra-low latency consistency, open storage formats, ecosystem compatibility, independent scaling, or simplified operational fault boundaries.
Open Formats Are a Starting Point, Not Full Database Semantics
Parquet, Delta Lake, and Apache Iceberg solve data openness, columnar scanning, snapshot management, and multi-engine accessibility. However, PostgreSQL semantics extend far beyond basic table rows. Preserving Multi-Version Concurrency Control (MVCC), long-running transactions, cross-table atomicity, explicit locks, integrity constraints, collations, TOAST storage, custom types, specialized index access methods (AM), Vacuum routines, and Point-in-Time Recovery (PITR) inside open columnar storage is a complex challenge that simple row-to-column transcoding cannot fully resolve.
AI Amplifies Integration Needs But Does Not Validate a Single Architecture
AI Agents transformed databases from passive query targets into active context management systems. Agents must read operational history, evaluate analytical context, maintain execution state, trigger external tools, and commit results back atomically—blurring the traditional boundaries between OLTP and OLAP. However, while this accelerates the demand for TP, AP, and AI convergence, the rise of AI represents a shift in application requirements rather than conclusive proof for any single vendor's storage architecture.
For LTAP to achieve broad production adoption beyond compelling architectural diagrams, several critical engineering hurdles must be decisively addressed:
Transaction Latency: How will object storage, PageServers, and multi-tier caching guarantee predictable tail latencies for high-concurrency OLTP workloads?
Semantic Fidelity: How can PostgreSQL's MVCC, custom data types, TOAST structures, collations, and secondary index access methods (B-tree, GIN, GiST, HNSW) be perfectly reconstructed from columnar storage?
Data Freshness & Compaction: How efficiently can authoritative object storage data be merged with unmaterialized, uncommitted delta buffers, and what is the maximum acceptable delta window?
Mutation Costs: How are high-frequency updates and deletes mapped into Parquet without inducing small-file proliferation, severe write amplification, and endless compaction loops?
Indexes & Cold-Start Recovery: Can massive secondary indexes truly be rebuilt on-demand without severe performance penalties, and what are the costs associated with cold caches during failover events?
Transactional Consistency: How are multi-table PostgreSQL transactions mapped into consistent snapshots that external analytical engines can observe atomically?
Fault Domain Definition: How is a unified recovery boundary established across PostgreSQL Compute, Safekeepers, PageServers, object storage, metadata catalogs, and analytical engines?
True Openness: Can third-party analytical engines independently read and write the "authoritative" Delta/Iceberg tables with full fidelity outside of the proprietary Databricks ecosystem?
If these engineering questions remain unanswered, LTAP will remain an impressive theoretical blueprint. If they are successfully resolved, LTAP could fundamentally alter how database persistence formats are designed.
LTAP is far from a mere marketing gimmick. It poses a bold and fundamental question to database architects:
Must PostgreSQL pages permanently serve as the authoritative persistence format for transactional databases? Can open columnar data become the definitive source of truth, demoting row pages, indexes, and caches to rebuildable, transient service layers?
This vision is compelling, yet the simple promise of "one dataset, multiple engines" does not instantly resolve decades of database engineering trade-offs. Eliminating explicit data copying does not remove physical format transcoding; unifying persistence formats does not automatically unify transactional semantics; and sharing object storage buckets does not guarantee identical snapshot, failure, or operational boundaries.
At this stage, LTAP is best viewed as an innovative storage paradigm exploration that warrants close study and rigorous production validation. Where transactional, analytical, and AI workloads ultimately converge—whether within a single engine, a unified database system, a shared authoritative storage tier, or a common governance platform—remains an open question.
Databricks has put forward its architectural answer. Whether it proves to be the definitive one is far from settled.
References
AI Era Database Infrastructure: Exploring Vectorized Execution in PostgreSQL
How a Leading ERP Vendor Entered the AI Fast Lane — A YMatrix Field Story
MatrixShift for YMatrix: A Practical Guide to Migrating from Greenplum
How MARS3 Works: Hybrid Row/Column Storage for High-Frequency Writes and Fast Analytics
YMatrix: A Unified Database Foundation for the AI Era
From TCP/UDP to Tunnel: How YMatrix Breaks the Communication Bottleneck in Large-Scale MPP Clusters
