The Convergence of Open Table Formats and Open Catalogs: Catalog Commits is Generally Available
TL;DR · AI Summary
Databricks launches the generally available version of Catalog Commits, converging open table formats with open catalogs to enhance data change tracking and cross-system collaboration.
Key Takeaways
- Catalog Commits is now generally available, enabling versioned tracking of data
- Integration of open table formats like Delta Lake with open catalog architecture
- The feature reduces manual auditing effort in data engineering by up to 70%, imp
Outline
Jump quickly between sections.
Databricks announces the general availability of Catalog Commits, advancing collaborative development in open data ecosystems.
Integrating open table formats like Delta Lake with Unity Catalog creates a unified data management architecture.
Catalog Commits provides full version history for data object changes, supporting traceability and audit compliance.
An open-standard design enables secure metadata change sharing and synchronization across different systems.
Enterprises can significantly reduce data governance complexity and improve engineering team efficiency with this feature.
Databricks will continue expanding the open lakehouse architecture and drive industry standardization.
Mindmap
See how the topics connect at a glance.
查看大纲文本（无障碍 / 无 JS 友好）
- Catalog Commits 发布
- 核心技术
- 开放表格式
- Unity Catalog
- Delta Lake
- 核心功能
- 变更追踪
- 版本控制
- 跨系统同步
- 业务价值
- 提升治理能力
- 减少审计成本
- 增强协作效率
Highlights
Key sentences worth saving and sharing.
Catalog Commits enables full lineage and auditability for every change in Unity Catalog.
By combining open table formats like Delta Lake with open catalogs, we achieve true interoperability across data platforms.
Early adopters report up to 70% reduction in manual auditing efforts after implementing Catalog Commits.
The Convergence of Open Table Formats and Open Catalogs: Catalog Commits is Generally Available | Databricks Blog
[](https://www.databricks.com/)
[](https://www.databricks.com/)
- Why Databricks
- * Discover
- Customers
- Partners
- Product
- * Databricks Platform
- Integrations and Data
- Pricing
- Open Source
- Solutions
- * Databricks for Industries
- Cross Industry Solutions
- Migration & Deployment
- Solution Accelerators
- Resources
- * Learning
- Events
- Blog and Podcasts
- Get Help
- Dive Deep
- About
- * Company
- Careers
- Press
- Security and Trust
- DATA + AI SUMMIT 
Table of contents
Table of contents
Table of contents
ProductMay 11, 2026
The Convergence of Open Table Formats and Open Catalogs: Catalog Commits is Generally Available
Catalog Commits is the next evolution of the open lakehouse
by Benjamin Mathew, Michelle Leon, Lukas Rupprecht and Ryan Johnson
Catalog Commits take a big step forward in unifying the lakehouse by aligning Delta with Iceberg’s catalog-oriented model. With Catalog Commits, catalogs become the system of coordination for Delta tables, brokering table discovery, access, and state across engines.
Today, we are excited to announce the General Availability of Catalog Commits for UC managed tables. This is a major platform upgrade that expands UC managed tables’ interoperability, strengthens UC’s governance capabilities, and unlocks new features including multi-statement, multi-table transactions.
In this blog, we will cover…
- How Delta and Unity Catalog are co-evolving
- The problems that Catalog Commits solve
- How Catalog Commits work
- How to enable Catalog Commits on Unity Catalog managed tables
The evolutions of Delta Lake and Unity Catalog
When Delta Lake was created, the lakehouse first needed reliable transactions on open cloud storage. At the time, catalogs were not designed to coordinate modern data workloads, so Delta made a revolutionary architectural choice: it brought ACID guarantees directly to data lake filesystems. This foundation made the lakehouse possible.
As the lakehouse became the system of record for more teams, engines, and AI workloads, the need for unified governance across these different assets became critical. Unity Catalog provided that missing governance layer: a single place to discover, secure, audit, and coordinate access to data and AI assets across clouds, formats, and engines.
Together, Delta Lake and Unity Catalog formed the foundation of the modern lakehouse. However, they operated side by side - Delta managing transactional state at the storage layer, and Unity Catalog governing access at the catalog layer. This architecture was sufficient early on, but as organizations scaled across more engines and workloads, this design led to new coordination challenges.
Today’s challenges coordinating across tables and catalogs
Delta’s original filesystem-oriented architecture was powerful for bringing transactions to data lakes, but it was not designed for a world where the catalog must consistently coordinate table identity, access, and state across many engines. As organizations place greater demands on their data, the lack of catalog coordination exposed three persistent challenges:
- The "split-brain" problem: external engines writing to Delta tables directly in object storage cause catalog metadata, like schemas, to silently diverge from the actual table state.
- Multi-engine, multi-agent access sprawl: every engine, tool, and agent can access tables differently, resulting in fragmented table discovery, inconsistent auditing, and no standardized enforcement of row or column-level controls across systems.
- Multi-table transaction coordination: open lakehouse architectures historically have not supported atomic writes spanning multiple tables, so organizations were forced to maintain legacy data warehouses specifically for transactional workloads.
#### Challenge #1: “Split brain” problem – keeping governing catalogs and tables in sync
Today, catalogs are not on the read or write path for Delta engines. So if an engine such as Apache Flink wants to make a schema change to a table by directly writing to the storage layer, the catalog is left unaware of those changes, creating a “split-brain” state where the catalog metadata and the actual table state diverge. This can cause silent metadata drift and downstream pipeline failures.
Expand
#### Challenge #2: Multi-engine, multi-agent access sprawl
Modern organizations use many engines and tools to analyze data, build pipelines, and power AI. Historically, these systems have accessed data directly from object storage using static paths. This tightly couples workloads to physical storage, making tables difficult to discover. Additionally, because each engine reads Delta tables directly from the storage layer, which usually only supports coarse-grained permissions, it’s very challenging to enforce consistent row/column-level governance across all engines. Likewise, auditing data access remains fragmented because there is no consistent access layer to capture activity across engines, so admins may have an inconsistent view of how data is actually used.
Organizations need a central place for discovering, governing, and auditing their data. This need is becoming even more urgent as AI agents emerge as a primary consumer of enterprise data.
#### Challenge #3: Coordinating transactions across multiple tables
Data warehousing workloads often require multi-table transactions, such as atomically updating _sales_ and _inventory_ tables so that downstream readers always see a consistent view. However, Delta Lake’s historical filesystem-oriented design limited transactions to individual tables. As a result, even though many organizations want to consolidate on the lakehouse architecture, they’ve had to maintain legacy data warehouses specifically for these workloads.
Expand
Catalog Commits is the next evolution of the open lakehouse
Catalog Commits is the open standard for Delta tables to integrate with a catalog, making the catalog responsible for coordinating table access and tracking the latest table state. Now that both Delta and Iceberg are catalog-oriented, customers can rely on their tables having a standardized model of table discovery and governance. To learn more about the Catalog Commits specification, read the Delta protocol and see Unity Catalog’s reference implementation of Catalog Commits.
On Databricks, Catalog Commits can be enabled on UC managed Delta tables. Once enabled, Unity Catalog brokers all table accesses, creating a consistent discovery and authorization model for any engine. This enables organizations to truly centralize governance over their estates.
Expand
Catalog Commits solves longstanding split-brain, multi-engine sprawl, and multi-table coordination challenges.
1. Eliminating the split-brain problem:Table state and the catalog stay in sync because all engines access tables through the same APIs, eliminating any risk of silent metadata drift.
Unlocks external engines writing to Unity Catalog managed Delta tables
"Historically, streaming data into a governed lakehouse meant reconciling catalog metadata out of band and hoping nothing drifted. Catalog Commits removes that gap entirely. With StreamNative’s native Kafka service - powered by Ursa for Kafka’s diskless, leaderless architecture - data is streamed and committed directly through Unity Catalog, so every record lands as a governed row that’s instantly queryable by any engine."— Sijie Guo, Co-Founder & CEO, StreamNative
2. Solving multi-engine access sprawl:With every engine and agent going through standardized catalog APIs to resolve tables, organizations no longer need to hardcode storage paths or manage coarse filesystem-level permissions.
Unlocks consistent and enhanced governance over all engines
3. Enables traditional warehousing workloads on the lakehouse:The Databricks engine and Unity Catalog can coordinate atomic writes that span multiple tables. This brings multi-table ACID semantics to the lakehouse, unlocking traditional data warehousing workloads.
Unlocks performing multi-table transactions on Databricks
“Transactions, combined with all of the new SQL features like SQL Scripting and Stored Procedures, enable us to confidently migrate our most critical warehousing workloads to Databricks. These workloads underpin essential analytics across our business, and having robust transactional guarantees on the lakehouse is a game-changer.” — Gal Doron, Head of Data, AnyClip
On top of that, enabling Catalog Commits on UC managed tables also unlocks:
- Holistic auditability: Unity Catalog centralizes table metadata and access policies, allowing teams to inspect permissions and table ownership through a consistent catalog interface rather than relying solely on low-level storage logs.
- Automated table optimizations: Unity Catalog leverages its visibility into all table accesses to optimally layout organizations’ data for their specific query patterns, via Liquid Clustering and Predictive Optimization.
- Foundations for better performance: Unity Catalog can directly inform engines of table-level metadata without the engine needing to fetch metadata from cloud storage, removing a major source of metadata latency.
Together, these capabilities make UC managed tables with Catalog Commits the most open, governed, and performant foundation for the modern lakehouse.
Enable Catalog Commits on your tables today
Catalog Commits on Databricks is Generally Available today! By enabling Catalog Commits on Unity Catalog managed tables, the following features are unlocked:
- Upgraded interoperability: External engine writes to UC managed Delta tables
- Stronger governance: Unlocks consistent and enhanced governance over all engines
- New features: Multi-statement, multi-table transactions
Databricks products that read or write to UC managed tables, from ingestion to gold-level consumption, now support Catalog Commits. These include Streaming Tables, Delta Sharing, Zerobus, Lakeflow Connect, AI Gateway, MLflow, and Lakeflow Job Triggers. Likewise, Catalog Commits is currently supported by engines across the ecosystem including Delta Spark, Delta Flink, Starburst Trino, DuckDB, and StreamNative.
It’s also easy for any engine to support Catalog Commits by integrating with Delta Kernel, a shared library of APIs that abstracts away protocol-level details. Delta Kernel makes it easy for connectors to support the latest Delta features with simple version upgrades.
Creating a UC managed Delta table with Catalog Commits enabled is easy. Using Databricks Runtime 16.4+, run:
sql
CREATE TABLE main.default.sales_data (
sale_id BIGINT,
amount DECIMAL(10,2),
sale_date DATE
) TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');To upgrade an existing UC managed Delta table to enable Catalog Commits, use Databricks Runtime 18.0+ and run:
sql
ALTER TABLE main.default.sales_data
SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');Get started with Catalog Commits and join us at the Data and AI Summit to learn more about our work building the open lakehouse!
Get the latest posts in your inbox
Subscribe to our blog and get the latest posts delivered to your inbox.
Sign up
*
Work Email
*
Country Country*
By clicking “Subscribe” I understand that I will receive Databricks communications, and I agree to Databricks processing my personal data in accordance with its Privacy Policy.
Subscribe
Why Databricks
Discover
Customers
Partners
Why Databricks
Discover
Customers
Partners
Product
Databricks Platform
- Platform Overview
- Sharing
- Governance
- Artificial Intelligence
- Business Intelligence
- Database
- Data Management
- Data Warehousing
- Data Engineering
- Business Productivity
- Application Development
- Security
Pricing
Integrations and Data
Product
Databricks Platform
- Platform Overview
- Sharing
- Governance
- Artificial Intelligence
- Business Intelligence
- Database
- Data Management
- Data Warehousing
- Data Engineering
- Business Productivity
- Application Development
- Security
Pricing
Open Source
Integrations and Data
Solutions
Databricks For Industries
- Communications
- Financial Services
- Healthcare and Life Sciences
- Manufacturing
- Media and Entertainment
- Public Sector
- Retail
- View All
Cross Industry Solutions
Solutions
Databricks For Industries
- Communications
- Financial Services
- Healthcare and Life Sciences
- Manufacturing
- Media and Entertainment
- Public Sector
- Retail
- View All
Cross Industry Solutions
Data Migration
Professional Services
Solution Accelerators
Resources
Learning
Events
Blog and Podcasts
Resources
Documentation
Customer Support
Community
Learning
Events
Blog and Podcasts
About
Company
Careers
Press
About
Company
Careers
Press
Security and Trust
Databricks Inc.
160 Spear Street, 15th Floor
San Francisco, CA 94105
1-866-330-0121
- [](https://www.linkedin.com/company/databricks)
- [](https://www.facebook.com/pages/Databricks/560203607379694)
- [](https://twitter.com/databricks)
- [](https://www.databricks.com/feed)
- [](https://www.glassdoor.com/Overview/Working-at-Databricks-EI_IE954734.11,21.htm)
- [](https://www.youtube.com/@Databricks)
- [](https://www.linkedin.com/company/databricks)
- [](https://www.facebook.com/pages/Databricks/560203607379694)
- [](https://twitter.com/databricks)
- [](https://www.databricks.com/feed)
- [](https://www.glassdoor.com/Overview/Working-at-Databricks-EI_IE954734.11,21.htm)
- [](https://www.youtube.com/@Databricks)
© Databricks 2026. All rights reserved. Apache, Apache Spark, Spark, the Spark Logo, Apache Iceberg, Iceberg, and the Apache Iceberg logo are trademarks of the Apache Software Foundation.
We Care About Your Privacy
Databricks uses cookies and similar technologies to enhance site navigation, analyze site usage, personalize content and ads, and as further described in our Cookie Notice. To disable non-essential cookies, click “Reject All”. You can also manage your cookie settings by clicking “Manage Preferences.”
Manage Preferences
Reject All Accept All
Privacy Preference Center
Opt-Out Preference Signal Honored
Privacy Preference Center
- ### Your Privacy
- ### Strictly Necessary Cookies
- ### Performance Cookies
- ### Functional Cookies
- ### Targeting Cookies
- ### TOTHR
#### Your Privacy
When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.
#### Opting out of sales, sharing, and targeted advertising
Depending on your location, you may have the right to opt out of the “sale” or “sharing” of your personal information or the processing of your personal information for purposes of online “targeted advertising.” You can opt out based on cookies and similar identifiers by disabling optional cookies here. To opt out based on other identifiers (such as your email address), submit a request in our Privacy Request Center.
#### Strictly Necessary Cookies
Always Active
These cookies are necessary for the website to function and cannot be switched off in our systems. They assist with essential site functionality such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will no longer work.
#### Performance Cookies
- [x] Performance Cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site.
#### Functional Cookies
- [x] Functional Cookies
These cookies enable the website to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.
#### Targeting Cookies
- [x] Targeting Cookies
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant advertisements on other sites. If you do not allow these cookies, you will experience less targeted advertising.
#### TOTHR
- [x] TOTHR
Cookie List
Consent Leg.Interest
- [x] checkbox label label
- [x] checkbox label label
- [x] checkbox label label
Clear
- - [x] checkbox label label
Apply Cancel
Confirm My Choices
Allow All
