Setting Up My First dbt Project: uv + Databricks + Medallion Architecture

I’m currently learning dbt as part of my push toward a junior data/analytics engineering role, following along with Ansh Lamba’s tutorial series and testing everything against a free Databricks workspace. This post is my own write-up of the setup process, partly for my own reference, partly for anyone else starting from zero.

Prerequisites

• Python installed
• A free Databricks workspace (Community Edition or trial works fine for learning)
• uv for Python environment and package management
• VS Code with the dbt Power User extension (optional, but genuinely helpful once installed)

Step 1: Set up the Python environment with uv

Instead of the classic python -m venv + pip install combo, uv handles environment creation and dependency management together, and it’s noticeably faster.

uv venv
uv init
uv add dbt-databricks
uv sync

• uv venv creates the virtual environment
• uv init scaffolds a pyproject.toml for the project
• uv add dbt-databricks adds the Databricks adapter as a dependency (this pulls in dbt Core automatically)
• uv sync installs everything from the lockfile into the venv

Step 2: Initialize the dbt project

dbt init

This scaffolds the actual dbt project: dbt_project.yml, and the standard folders (models/, seeds/, macros/, tests/, snapshots/). It also prompts you for your Databricks connection details, which get written to a separate file, not inside the project folder.

Step 3: Understand profiles.yml (your connection file)

On Windows, this lives at:

C:\Users\<you>\.dbt\profiles.yml

This file is kept outside the project folder on purpose: it holds your warehouse credentials, so it never gets accidentally committed to Git. It looks roughly like:

dbt_course_tutorial:
  target: dev
  outputs:
    dev:
      type: databricks
      catalog: your_catalog
      schema: your_schema
      host: your-workspace.cloud.databricks.com
      http_path: /sql/1.0/warehouses/xxxxx
      token: "{{ env_var('DBT_DATABRICKS_TOKEN') }}"

Your dbt_project.yml links to this by name via the profile: field: the two names have to match exactly, or dbt debug will fail to find your credentials.

Step 4: Verify the connection

cd dbt_course_tutorial
dbt debug

This checks that dbt can find your project config, read the matching profile, and successfully connect to Databricks, all before you write a single model.

Step 5: Understand dbt_project.yml

This file is what tells dbt “this folder is a dbt project,” and controls how models get built by default.

name: 'dbt_course_tutorial'
version: '1.0.0'
profile: 'dbt_course_tutorial'

model-paths: ["models"]
seed-paths: ["seeds"]

models:
  dbt_course_tutorial:
    bronze:
      +materialized: view
    silver:
      +materialized: view
    gold:
      +materialized: table

The models: block sets defaults per folder, so every model in bronze/ and silver/ builds as a view by default, while everything in gold/ builds as an actual table.

Step 6: Declare your raw sources

Upload your raw CSVs to Databricks first (I used 6 tables: fact_sales, fact_returns, dim_customer, dim_product, dim_store, dim_date). Then declare them in a sources.yml file. This doesn’t move or transform any data, it just tells dbt where the raw tables already live:

version: 2

sources:
  - name: raw
    database: your_catalog
    schema: source
    tables:
      - name: fact_sales
      - name: fact_returns
      - name: dim_customer
      - name: dim_product
      - name: dim_store
      - name: dim_date

Step 7: Build the medallion layers

This is the standard bronze → silver → gold pattern (dbt’s own docs call it staging → intermediate → marts, same idea):

Bronze: a near-raw copy, referencing the source directly:

-- models/bronze/brz_fact_sales.sql
select
    sale_id,
    customer_id,
    product_id,
    sale_date,
    amount
from {{ source('raw', 'fact_sales') }}

Silver: cleaning, type-casting, filtering, referencing the previous model (not the raw source):

-- models/silver/slv_fact_sales.sql
select
    sale_id,
    customer_id,
    product_id,
    cast(sale_date as date) as sale_date,
    amount
from {{ ref('brz_fact_sales') }}
where amount is not null

Gold: business-ready aggregation, built as an actual table since dashboards will query it directly and repeatedly:

-- models/gold/gld_daily_sales_summary.sql
{{ config(materialized='table') }}

select
    sale_date,
    count(*) as total_orders,
    sum(amount) as total_revenue
from {{ ref('slv_fact_sales') }}
group by sale_date

Notice: source() is only ever used once, in bronze. Everything after that chains together with ref(), and that chain is exactly what lets dbt figure out the correct build order automatically, without ever being told explicitly.

Why view for bronze/silver but table for gold?

• Views cost nothing to store: they’re just saved SQL that re-runs live every time someone queries them. Fine for bronze/silver since nothing queries them directly; they only feed the next model in the chain.
• Tables are physically stored: built once during dbt run, then read instantly afterward. Worth the storage cost for gold, since that’s the layer a BI tool like Power BI hits directly, often, and shouldn’t have to recompute the entire chain on every click.

What’s next

Still working through: schema tests, generic vs singular tests, and eventually snapshots for slowly-changing dimensions. Will follow up with a part two once I’m further into the course.