We’re getting ready to pilot the Databricks Detective Series ahead of its launch at Data + AI Summit 2026, and we’re looking for Community members to help test it. Databricks Detective Series is a free, gamified, hands-on experience where you become...
Calling all builders!Since GA, thousands of teams have moved production workloads onto Lakebase. We want to hear how you’re using it.A few examples of stories we’d love to capture: Building stateful AI agents that need persistent memory alongside lak...
Already registered for Data + AI Summit 2026? Join us on Monday, June 15 for the Databricks Apps & Agents for Good Hackathon 2026, hosted in partnership with OpenAI. Secure your spot and apply by Sunday May, 31 Note: Marking RSVP on this Community ...
Data + AI Summit 2026 is almost here! We’ve added a community poll on the homepage and would love to hear how you’re planning to attend this year. Cast your vote from the poll section on the right side of the Community homepage.
Limited seats are available to join our revamped training program at Data + AI Summit 2026 in San Francisco, with early-bird pricing that gives you 50% off training if you register by April 30. Key highlights Big savings on training and certification...
Community Space for Lakebase Insights Are you building with Lakebase? If so, do you have a single source to stay ahead of every technical shift and architectural breakthrough? More importantly, do you know what practitioners are actually saying o...
Keeping up with financial crime is complex. Banks and financial institutions need to deal with changing regulations, sophisticated money laundering techniques and large volumes of data that must be analyzed to detect risk and protect client assets an...
The Weekly Digest • May 04 – May 10 Community Pulse This week's top threads, breakthroughs, and knowledge drops from the community. Top Contributors This Week Contributors leading with curiosity and kindness: @nayan_wylde @Sumit_7 @amirabed...
We need to import large amount of Jira data into Databricks, and should import only the delta changes. What's the best approach to do so? Using the Fivetran Jira connector or develop our own Python scripts/pipeline code? Thanks.
Hi @greengil good question, I went through this something similar recently, so sharing what I found.My instinct was also to build it in Python, but once I dug in, the "just write a script" path hides a lot of pain:Deletions are invisible. Jira's RES...
I am fairly new to Databricks and am learning it because a company I am working on is going to use it. One of the things they are going to use is Lakebase postgres so I thought I'd set it up on my personal account. First I don't see app switcher, sec...
If it is available, you would see at Compute->Lakebase and tabs for Provisioned and Autoscaling. This option Lakebase is next to Apps. There is another option dotted grid on the top right corner, the option just before your profile name, if you expan...
Databases have powered software for decades, but their underlying architecture hasn’t kept pace with modern application needs—especially in the era of AI and rapid development. Lakebase introduces a new approach by separating compute from storage an...
What is your point of view on Provisioned instances Vs Autoscaling instances.
We had ADF pipelines extracting POS data to Snowflake, but needed real-time operational tracking—job statuses, data quality alerts, user audit logs. Traditional RDS/SQL databases created ETL sync nightmares between ops and analytics layers.Lakebase s...
This is great! I was wondering if there is a specific rationale behind Lakebase Provisioned instance as opposed to Autoscaling instance. More insights would be helpful.
We’re getting ready to pilot the Databricks Detective Series ahead of its launch at Data + AI Summit 2026, and we’re looking for Community members to help test it. Databricks Detective Series is a free, gamified, hands-on experience where you become...
Okay, I'm totally sucked in now . This looks really intriguing. I'll give this a go tonight or tomorrow and post my experience with it. Thanks for providing your write-up @Brahmareddy! @Tushar_Parekar Absolutely love the concept. Kudos to you guys.Al...
I am preparing for Databricks Certified Data Engineer Associate/Professional Certifications but not sure how to get access to lab tutorials / notebook demonstrated in lecture, any help would be greatly appreciated. Thanks in advance.Arijit
@Arijit14, I can see you're a Databricks Partner, I assume that means you'll have access to this area: https://partner-academy.databricks.com/learn/catalog/view/89and here: https://partner-academy.databricks.com/pages/47/partner-labs You can find the...
Hi Databricks Team,I've been trying to transfer my certifications from my previous employer email Id to my current employer email Id. I've opened a ticket with the partner databricks certification team and haven't heard back from them for a while. He...
What i did . Added my company email id to accredible site and made that as my primary email id . if that doesn't resolve the issue send email to certificates@accredible.com and support@accredible.com . it should solve the problem
Calling all builders!Since GA, thousands of teams have moved production workloads onto Lakebase. We want to hear how you’re using it.A few examples of stories we’d love to capture: Building stateful AI agents that need persistent memory alongside lak...
When we switched our production workloads to Lakebase, it honestly felt like a weight lifted off our shoulders. Before, we were juggling multiple systems just to keep data flowing between analytics and operations. Now, everything lives in one place, ...
I came across a blog post comparing Databricks and Google BigQuery for AI-ready data teams. The workload angle stood out.That feels like a useful way to frame the discussion here in the Databricks Community. A lot of platform questions come back to t...
We connect Tableau and Power BI to our Databricks SQL warehouse via OAuth tokens. Does Unity Catalog row filters apply at the SQL layer regardless of the BI tool, or do we need additional enforcement at the warehouse level?
Unity Catalog row filters apply at the SQL/query layer, so if Tableau or Power BI is querying a Databricks SQL warehouse, the filters are enforced there — you do not need a separate warehouse-level row-filter feature. Row filters and column masks are...
We’re currently designing our Medallion Architecture pipelines using Lakeflow Jobs, and I wanted to get some opinions on orchestration best practices.Right now, our approach is essentially 1 job per target table (for example, each Bronze/Silver/Gold ...
We typically organize our workloads with one job per catalog, and then use one or more pipelines to load tables into the appropriate schemas. As our data engineers ingest raw data, this structure is primarily applied in the Silver and Gold layers of ...
Hi Everyone, I am facing issue in loading the Notebook, also on top of it. Some of the other application are also not loading. I am attaching the image for it. Please see if anyone can help on it.
I have seen this error occasionally as well . Try refreshing the page or just go to home and then to the path . this is most likely due to temporary backend/service failure.
If your CI/CD pipelines suddenly started failing out of nowhere with this error:"error downloading Terraform: unable to verify checksums signature: openpgp: key expired"and you’re using Databricks CLI - you’re probably hitting the same issue I did.Th...
Hi Databricks Community,I am preparing for the Databricks Certified Data Engineer Associate exam and had a concern I couldn't find a clear answer to.What happens if my internet/WiFi connection drops in the middle of the exam? Specifically:- Does the...
Manufacturing moves too fast for generic AI. In this episode, see how specialized AI agents built on Agent Bricks help manufacturers improve planning, forecasting, logistics, and quality at scale. The impact: Planning & inventory: Manufacturers cut p...
| User | Count |
|---|---|
| 2264 | |
| 1315 | |
| 1199 | |
| 1193 | |
| 958 |
| User | Count |
|---|---|
| 426 | |
| 286 | |
| 220 | |
| 63 | |
| 51 |
You've spent the afternoon building a StatefulProcessor for your TransformWithState streaming job. It tracks per-user sessions, accumulates running totals, or deduplicates events. Now you want to know...
While data volumes are growing, the time windows to process them are getting stricter. Enterprise customers are generating more data than ever, and the speed is only accelerating with more agentic wor...
Summary External engines — Apache Spark™ (batch and Structured Streaming), DuckDB, Apache Flink, Starburst, and StreamNative Kafka Service — can now create, read, and write Unity Catalog-managed Delta...
Introduction LLM-powered apps have two costs that compound fast: every request costs money, and users ask the same question in many ways. "How do I reset my password?", "I need to reset my password", ...
