DATA + AI SUMMIT ![Image 3: Data+ai summit promo JUNE 15–18|SAN FRANCISCO Last chance to save 50% — ends April 30. Register](http://www.databricks.com/dataaisummit?itm_source=www&itm_category=home&itm_page=home&itm_location=navigation&itm_component=navigation&itm_offer=dataaisummit)

1. All blogs 2. / Platform

Table of contents

ProductApril 28, 2026

Agentic Data Engineering with Genie Code and Lakeflow

Genie Code streamlines data pipeline development, orchestration, and deployment

by Gal Oshri, Camiel Steenstra, Lennart Kats and Joanna Zouhour

Summary

Genie Code is an autonomous AI partner built specifically for data
Data engineers can use Genie Code directly within Lakeflow, from building pipelines in the Pipeline Editor to orchestrating workflows in Lakeflow Jobs
Genie Code supports the full data engineering lifecycle - from development and orchestration to monitoring and debugging - within a single agent experience

With Genie Code, data engineers can use natural language to generate production-ready data pipelines, orchestrate them with jobs, and debug failures. Tasks that used to take weeks - finding data, building transformations, stitching together jobs, and fixing failures - can now be done in hours, while staying aligned with governance and operational standards.

Below, we'll walk through how this works in practice: discovering data, building pipelines, orchestrating jobs, and debugging failures, all from a single conversation.

Build and orchestrate complete, production-ready pipelines and jobs using natural language

Genie Code can now take you from exploration to scheduled pipelines and jobs in one thread, helping you author and operate them end-to-end.

It accelerates the development of Lakeflow Spark Declarative Pipelines and simplifies how pipelines and notebooks are orchestrated and run through Lakeflow Jobs. Genie Code understands your pipeline and job context, accessing the code, configuration, and run results.

Genie Code helps across key stages of the data engineering lifecycle:

**Search over data assets, not just code**: Genie Code uses popularity, lineage, code samples, and Unity Catalog metadata to identify the most relevant datasets for your task. For example, you can ask Genie Code to explain how tables relate or trace how data flows through a pipeline. At SiriusXM, teams use Genie Code to understand table relationships more quickly.
**Build and modify pipelines**: Start by describing the pipeline you want in plain language, such as a fraud detection pipeline built on a medallion architecture. Genie Code generates a Spark Declarative Pipeline with Bronze, Silver, and Gold layers, including sources, transformations, data quality expectations, and outputs. From there, you can ask for changes, review the proposed diffs, and run and test the pipeline.

!Image 4: Lakeflow Spark Declarative Pipelines Expand

**Define and orchestrate jobs:**No need to manually define and maintain orchestration logic. You describe the job you want, including tasks, dependencies, and schedule. Genie Code configures it for you, then helps modify, debug, and fix orchestration issues in natural language.

!Image 5: Orchestrate Jobs Expand

**Extend and evolve existing workflows**: As requirements change, Genie Code helps you update pipelines and jobs with new datasets and transformations. It understands the current structure and results of your pipelines, and can extend them by writing AutoCDC flows for change data capture, configuring Auto Loader, applying data quality expectations, and following the medallion architecture.
**Embrace best practices with**Declarative Automation Bundles (DABs): Genie Code can work directly within your existing DABs projects: adding resources, updating configurations, validating bundles, and deploying to your targets. So you can adopt software engineering best practices like source control, testing, and CI/CD for your data projects without hand-writing YAML.
**Work faster without lowering standards:**These capabilities reduce manual effort while keeping workflows aligned with enterprise requirements. Pipelines remain governed through Unity Catalog and follow established patterns for performance and data quality, while jobs inherit consistent configuration for scheduling, retries, and dependencies. Data engineers stay in control, but spend less time on repetitive work.

Monitor, diagnose, and debug pipelines and jobs

**Understanding and improving pipeline behavior**: Genie Code can inspect datasets and pipeline outputs to help you understand a pipeline end-to-end. For example, it can summarize transformations, trace how data flows into downstream tables, and highlight unexpected changes in row counts or schemas.
**Debug and diagnose job and pipeline failures**: When a pipeline or job fails, Genie Code helps you work through the issue. It analyzes errors, proposes updates across the relevant files, and shows you the diffs before applying any changes. You can review each update and decide what moves forward. This turns long, manual debug cycles into faster guided iterations.

!Image 6: Debug Pipelines and Jobs Expand

**Extend and customize Genie Code:** Genie Code is not limited to built-in capabilities. Teams can extend it with custom instructions, agent skills and integrate external systems through MCP servers, allowing Genie Code to operate on domain-specific logic, internal tools, and custom workflows. This ensures Genie Code adapts to your environment and domain knowledge.

What’s next

More capabilities are coming to extend Genie Code across pipelines, jobs, and the broader platform. One exciting feature on the horizon is **AI-optimized workloads**. In the future, you can allow Genie Code to also run in the background to keep your platform working efficiently, so you can hand off those repetitive and time-consuming tasks. This includes responding to job failures and managing routine upgrades, but also automatically right-sizing cluster use.

Curious to learn more about these updates and best practices? Make sure to register for Data+AI Summit where we have hundreds of sessions covering Genie Code, Lakeflow and much more!

Try Genie Code’s data engineering capabilities

Open Genie Code in agent mode and ask it to help you build or update your pipelines and jobs. Check out the demo for more details .

Review the documentation to learn more.

Get the latest posts in your inbox

Subscribe to our blog and get the latest posts delivered to your inbox.

Sign up

Work Email

Country Country*

By clicking “Subscribe” I understand that I will receive Databricks communications, and I agree to Databricks processing my personal data in accordance with its Privacy Policy.

View all blogs

![Image 7: databricks logo](https://www.databricks.com/)

Why Databricks

Discover

Customers

Customer Stories

Partners

Why Databricks

Discover

Customers

Customer Stories

Partners

Product

Databricks Platform

Pricing

Open Source

Integrations and Data

Product

Databricks Platform

Pricing

Open Source

Integrations and Data

Solutions

Databricks For Industries

Cross Industry Solutions

Data Migration

Professional Services

Solution Accelerators

Solutions

Databricks For Industries

Cross Industry Solutions

Data Migration

Professional Services

Solution Accelerators

Resources

Documentation

Customer Support

Community

Learning

Events

Blog and Podcasts

Resources

Documentation

Customer Support

Community

Learning

Events

Blog and Podcasts

About

Company

Careers

Press

Security and Trust

About

Company

Careers

Press

Security and Trust

![Image 9: databricks logo](https://www.databricks.com/)

Databricks Inc.

160 Spear Street, 15th Floor

San Francisco, CA 94105

1-866-330-0121

[](https://www.linkedin.com/company/databricks)
[](https://www.facebook.com/pages/Databricks/560203607379694)
[](https://twitter.com/databricks)
[](https://www.databricks.com/feed)
[](https://www.glassdoor.com/Overview/Working-at-Databricks-EI_IE954734.11,21.htm)
[](https://www.youtube.com/@Databricks)

See Careers

at Databricks

[](https://www.linkedin.com/company/databricks)
[](https://www.facebook.com/pages/Databricks/560203607379694)
[](https://twitter.com/databricks)
[](https://www.databricks.com/feed)
[](https://www.glassdoor.com/Overview/Working-at-Databricks-EI_IE954734.11,21.htm)
[](https://www.youtube.com/@Databricks)

We Care About Your Privacy

Databricks uses cookies and similar technologies to enhance site navigation, analyze site usage, personalize content and ads, and as further described in our Cookie Notice. To disable non-essential cookies, click “Reject All”. You can also manage your cookie settings by clicking “Manage Preferences.”

Manage Preferences

Reject All Accept All

Privacy Preference Center

Opt-Out Preference Signal Honored

Privacy Preference Center

### Your Privacy
### Strictly Necessary Cookies
### Performance Cookies
### Functional Cookies
### Targeting Cookies
### TOTHR

#### Your Privacy

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.

#### Opting out of sales, sharing, and targeted advertising

Depending on your location, you may have the right to opt out of the “sale” or “sharing” of your personal information or the processing of your personal information for purposes of online “targeted advertising.” You can opt out based on cookies and similar identifiers by disabling optional cookies here. To opt out based on other identifiers (such as your email address), submit a request in our Privacy Request Center.

More information

#### Strictly Necessary Cookies

Always Active

These cookies are necessary for the website to function and cannot be switched off in our systems. They assist with essential site functionality such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will no longer work.

#### Performance Cookies

[x] Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site.

#### Functional Cookies

[x] Functional Cookies

These cookies enable the website to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

#### Targeting Cookies

[x] Targeting Cookies

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant advertisements on other sites. If you do not allow these cookies, you will experience less targeted advertising.

#### TOTHR