Datapao Full-Day Workshop

Date: November 11th (Full-Day Workshop)
Speaker: Kiss Benedek & Gabor Tompa

In this intensive, hands-on workshop, participants will build a complete personal finance tracking application using Databricks as the core platform. The workshop demonstrates how modern data engineering practices can be supercharged with AI-assisted development to accelerate the creation of robust data applications.

Workshop Overview

This full-day session guides participants through the entire development lifecycle of a personal finance tracking application, from initial data modeling to creating interactive dashboards. We'll leverage AI-driven code generation to streamline development while exploring Databricks' comprehensive feature set.

Key Topics

• Setting Up the Databricks Environment: Workspace configuration, cluster creation, and security setup
• Data Ingestion and Modeling: Creating schemas in Unity Catalog for financial transactions, budgets, and goals
• ETL Pipeline Development: Using Lakeflow to process financial data with AI-generated transformations
• Machine Learning Integration: Building expense categorisation models and spending pattern detection
• Dashboard Creation: Developing interactive visualisations with AI/BI Dashboards and discussing your spending with Genie
• Data Security and Governance: Implementing privacy controls for sensitive financial information using Unity Catalog
• Development Workflow: Setting up a CI/CD ready developer workflow closely integrated with Databricks Asset Bundles.
• CI/CD Pipeline Setup: Automating testing and deployment with AI-optimised workflows

Workshop Format

The workshop follows a practical, step-by-step approach where participants actively build components of the application. Throughout the day, we'll demonstrate how to leverage AI coding assistants to accelerate development, focusing on: - Using AI to generate metadata and transformation code - Implementing AI-assisted debugging and optimisation - Creating natural language interfaces for data exploration - Automating documentation with AI tools

Target Audience

This workshop is ideal for data engineers, data scientists, and developers who want to learn how to build end-to-end data applications on Databricks while leveraging built-in AI to enhance their development workflow. Participants should have basic familiarity with Python and SQL, but extensive Databricks experience is not required.

Takeaways

By the end of this workshop, participants will have: - A functioning personal finance tracking application built on Databricks - Practical experience with AI-driven development techniques - Deep understanding of Databricks' core features and capabilities - Templates and patterns that they can apply to their own projects - Strategies for incorporating AI into their development workflows