Every professional in our network passes rigorous vetting assessments and only the top 0.5% make the cut. From full-stack developers to growth marketers and accountants, you’ll only meet the best of the best on South.
.svg){kind=logo}
dbt is a data transformation framework used primarily inside modern cloud data platforms.
It allows teams to transform data that has already been loaded into a warehouse or data platform.
A simplified modern data stack might look like:
Operational systems
↓
Fivetran, Airbyte, or custom ingestion
↓
Snowflake, BigQuery, Databricks, Redshift, or another data platform
↓
dbt
↓
BI, analytics, operational data products, and AI
dbt primarily owns the transformation layer in the middle.
Teams write SQL models describing how raw data should become cleaner, reusable datasets.
dbt then helps manage:
This brings many software-engineering practices into analytics development.
dbt originally stood for data build tool.
The product and ecosystem are now generally referred to simply as dbt.
The key idea is straightforward:
Transform data where it already lives.
Instead of extracting warehouse data into a separate transformation engine for every modeling step, dbt generally executes transformation logic inside the connected data platform.
This approach is commonly associated with ELT:
Extract → Load → Transform
rather than the older pattern:
Extract → Transform → Load.
dbt can support several parts of an analytics and data workflow.
The most fundamental use case is transforming raw source data into clean, understandable datasets.
For example:
Raw Stripe transactions
↓
Clean payments
↓
Customer revenue
↓
Monthly recurring revenue
The transformations become explicit models rather than disconnected SQL queries.
dbt is closely associated with analytics engineering.
Analytics engineering applies software-development practices to analytical data.
That can include:
This helps data teams treat transformation logic as a maintained codebase.
dbt can organize transformation logic inside warehouses such as:
and other supported data platforms.
A warehouse may contain enormous amounts of raw information.
dbt helps turn it into datasets designed around how the organization actually analyzes the business.
Teams use dbt to create reusable models representing concepts such as:
Those models can then feed multiple dashboards and analyses.
BI tools often sit downstream from dbt.
Instead of every dashboard recreating business logic independently, dbt can calculate and organize trusted datasets upstream.
That can improve consistency across:
Organizations frequently develop conflicting definitions of business metrics.
For example:
Revenue in Dashboard A
may not equal:
Revenue in Dashboard B.
dbt can help centralize the transformation logic and semantic definitions behind business metrics.
dbt tests can automatically check assumptions about data.
For example:
These tests can run as part of development and production pipelines.
Models, columns, metrics, and sources can be documented close to the code that creates them.
This makes documentation easier to maintain than a separate data dictionary disconnected from implementation.
dbt understands dependencies between project resources.
That allows teams to see how upstream information contributes to downstream models.
For example:
Stripe payments
↓
stg_stripe__payments
↓
int_customer_payments
↓
fct_revenue
↓
Executive revenue dashboard
Lineage becomes increasingly useful as the number of models grows.
AI applications increasingly depend on structured business data.
If definitions such as:
aren't consistent, AI systems can inherit the same confusion.
A governed dbt transformation and semantic layer can provide cleaner, more consistent context for downstream AI applications.
Models are the foundation of a dbt project.
A model is usually defined through a SQL SELECT statement.
For example, a model might:
When dbt runs, the SQL is executed in the connected data platform.
Depending on configuration, the result may become:
dbt creates a directed acyclic graph, or DAG, from relationships between models and other project resources.
A simplified DAG might look like:
raw_customers
↓
stg_customers
↓
dim_customers
↓
customer_metrics
Dependencies allow dbt to understand:
This graph powers much of dbt's:
ref()The ref() function connects one dbt model with another.
Instead of hard-coding a database object name, a model can reference another dbt model.
Conceptually:
ref('stg_orders')
tells dbt:
This model depends on stg_orders.
dbt can then:
Using ref() consistently is central to building a maintainable dbt DAG.
Sources describe upstream data that dbt doesn't create itself.
Examples include raw tables loaded from:
A source definition gives dbt structured information about those upstream datasets.
Teams can then use source references instead of repeatedly hard-coding raw table names.
Some pipelines need to know whether source information has arrived on schedule.
Source freshness checks can identify when data is older than expected.
For example:
Orders expected every hour
but newest order is four hours old
may indicate an ingestion problem upstream from dbt.
Freshness expectations should reflect the actual business requirement.
A weekly financial dataset doesn't need the same freshness threshold as a near-real-time operational dashboard.
A common approach organizes transformation logic into layers.
There isn't one mandatory folder structure, although several patterns have become widely used.
Staging models clean and standardize raw source information.
A staging model might:
Examples:
stg_salesforce__accounts
stg_stripe__payments
stg_app__users
Staging models should usually remain relatively close to the source.
Intermediate models represent reusable transformation steps between staging data and final business-facing models.
They may:
These models can prevent final marts from becoming enormous SQL files.
Marts provide datasets designed around business consumption.
Examples include:
Models may include:
dim_customers
fct_orders
fct_revenue
These datasets frequently power:
dbt often works with dimensional modeling patterns.
Fact tables generally represent measurable business events.
Examples include:
Dimensions provide descriptive context.
Examples include:
A reporting model might combine:
fct_orders
with:
dim_customers
and:
dim_products.
Every model should have a clearly defined grain.
Grain answers:
What does one row represent?
Examples:
One row per customer
One row per order
One row per account per month
Undefined grain creates confusion around joins, duplication, and metrics.
Materializations determine how dbt models are represented in the data platform.
A view stores the query definition rather than the full query result.
Views can work well for:
Complex chains of views can eventually become slow.
A table materialization persists the full model output.
This can improve downstream query performance.
The tradeoff is that the table needs to be rebuilt when the model runs.
Incremental models update only part of an existing dataset instead of rebuilding everything every time.
They're useful when:
Ephemeral models aren't created as separate database relations.
dbt incorporates their logic into downstream queries.
They can be useful for lightweight reusable transformations.
Heavy use can also make compiled queries more complicated.
Advanced teams can create custom materializations for specialized workflows.
This should solve a genuine platform need rather than replacing standard materializations unnecessarily.
Incremental models become important as datasets grow.
Imagine an events table containing billions of rows.
Reprocessing every historical event every hour may waste substantial warehouse compute.
Instead, an incremental model can focus on:
while retaining previously processed information.
is_incremental()dbt provides logic that allows SQL to behave differently during incremental runs.
A model might process all historical data during its first build and only recent data during later runs.
Some incremental strategies use a unique key to identify records that may need to be updated.
Examples might include:
Teams may periodically need to rebuild an incremental model completely.
This can happen when:
Exact incremental capabilities vary according to the underlying data platform.
Developers should understand how their warehouse handles:
rather than assuming one strategy fits every platform.
Testing is one of dbt's most important capabilities.
A transformation can execute successfully while producing incorrect business data.
Tests validate assumptions about the result.
Data tests query data and return failures when an assumption is violated.
Examples include:
Customer IDs should be unique.
Order status should belong to an approved set.
Every payment should relate to a known customer.
Common reusable tests include:
not_nulluniqueaccepted_valuesrelationshipsThese cover many basic data-quality expectations.
Teams can create reusable tests for organization-specific rules.
For example:
valid_email
or:
positive_amount
could be applied across several models.
Singular tests are standalone SQL queries that identify invalid data.
They can support more complicated business conditions.
For example:
Completed subscriptions should have a start date.
dbt also supports unit-testing transformation logic with controlled input and expected output.
Unit tests can be particularly useful for models containing:
Testing every column mechanically can create noise.
A stronger strategy asks:
Testing should protect important contracts in the data.
Model contracts allow teams to define expectations around a model's structure.
Depending on configuration and platform support, a contract can specify information such as:
This can help protect downstream users from unexpected schema changes.
Contracts become especially useful when models operate more like formal data products.
dbt allows documentation to live alongside models and configuration.
Teams can document:
Useful documentation explains meaning.
Instead of:
customer_status: customer status
write something closer to:
customer_status: Lifecycle category based on subscription state at the end of the reporting day.
The goal is to help another person understand how to use the data.
dbt's project graph creates model-level lineage.
Teams can use lineage to answer questions such as:
Modern dbt tooling is also moving toward richer column-level lineage and project metadata.
dbt uses Jinja templating to make SQL more dynamic and reusable.
Jinja can support:
For example, repeated SQL across many models can sometimes be generated through a macro.
Jinja should make SQL easier to maintain.
If templating becomes so abstract that nobody can understand the compiled query, it has gone too far.
Macros create reusable functions written with Jinja.
They can help standardize patterns such as:
A project with dozens of copies of the same transformation may benefit from a macro.
A transformation used once may be clearer as ordinary SQL.
dbt packages allow teams to reuse project functionality.
Packages can provide:
Community packages can eliminate unnecessary custom work.
Teams still need to understand the dependencies they introduce.
Seeds are small CSV files stored inside a dbt project and loaded into the data platform.
They can be useful for relatively static reference information such as:
Seeds aren't a replacement for a full data-ingestion system.
Snapshots allow dbt to track how mutable source records change over time.
Imagine a customer table where:
plan = Pro
changes tomorrow to:
plan = Enterprise.
If the source keeps only the current state, the previous plan may disappear.
A snapshot can preserve that history.
This can support:
Teams should define how changes are detected.
Exact configuration depends on the source and dbt version.
The important architectural question is:
Which source changes need historical preservation?
Snapshotting every mutable table can create unnecessary storage and processing.
The dbt Semantic Layer allows teams to define governed metrics on top of existing models.
Instead of implementing:
Monthly recurring revenue
separately in several dashboards, organizations can create reusable metric definitions.
Those definitions can then be consumed by downstream tools.
Metrics represent business calculations such as:
Dimensions allow those metrics to be analyzed by attributes such as:
A semantic layer can help organizations align around:
one definition, many consumers.
This becomes increasingly valuable when metrics feed:
A small analytics team can often maintain one dbt project.
Large data organizations may have many teams working across hundreds or thousands of models.
dbt Mesh provides patterns for organizing data development across those boundaries.
Important concepts include:
The goal is to give teams autonomy without turning the warehouse into disconnected projects.
dbt can distinguish how models should be consumed.
Some models may exist only as internal implementation details.
Others can provide stable interfaces to external teams.
This creates clearer boundaries between:
Groups associate project resources with owners.
Ownership helps answer:
Technical lineage becomes much more useful when paired with organizational ownership.
Exposures describe downstream uses of dbt data that exist outside the dbt model graph.
Examples include:
Documenting exposures can help teams understand the full impact of upstream changes.
dbt Core is the open-source command-line implementation of dbt.
Teams can run dbt Core through their own:
Typical commands include:
dbt rundbt testdbt builddbt compiledbt Core gives teams greater control over how their dbt workflows are hosted and orchestrated.
The dbt platform provides managed development and production capabilities around dbt.
Depending on the product and plan, that can include:
Teams should choose between self-managed and managed dbt workflows according to infrastructure, governance, team size, and operational requirements.
dbt is undergoing a major engine transition.
The newer Fusion architecture is built around a Rust-based engine with deeper SQL understanding.
Its capabilities can provide:
As dbt's engine architecture evolves, strong dbt practitioners should understand both the established dbt project model and the newer development capabilities rather than tying their expertise to one particular runtime version.
Traditional dbt builds often execute resources because they belong to a scheduled job.
Newer state-aware capabilities can make execution more selective.
The system can consider whether:
before determining whether work needs to run.
This can reduce:
The underlying idea is important even as the exact product capabilities continue evolving:
Build what needs rebuilding.
dbt transformations need to run at appropriate times.
Teams may orchestrate dbt through:
For example:
Fivetran load completes
↓
dbt staging models
↓
Intermediate models
↓
Business marts
↓
BI refresh
The orchestration layer needs to understand dependencies between ingestion, transformation, and downstream consumers.
Apache Airflow provides general workflow orchestration.
dbt specializes in transformations and their data graph.
Teams may use:
Airflow → orchestrates the complete pipeline
dbt → manages warehouse transformations
The technologies can complement each other.
SQL provides the primary language used to write dbt models.
dbt adds the development framework surrounding that SQL.
SQL provides:
dbt provides:
Strong dbt expertise therefore starts with strong SQL.
Some dbt environments also support Python models.
Python may be useful for selected transformations that don't fit naturally into SQL.
Use cases might involve:
SQL remains the foundation of most dbt projects.
Teams should avoid converting straightforward SQL transformations into Python merely because Python models are available.
Snowflake is a common platform for dbt projects.
dbt executes transformation logic inside Snowflake while managing the project structure around that SQL.
Performance considerations may include:
A strong dbt practitioner understands both dbt and the characteristics of the warehouse underneath it.
BigQuery is another common dbt platform.
Developers may need to understand:
The dbt model can be identical in purpose while requiring different performance decisions according to the warehouse.
Databricks can support dbt transformations across lakehouse environments.
Developers may work with:
Again, dbt expertise doesn't replace data-platform expertise.
Amazon Redshift can also provide the execution environment for dbt transformations.
Relevant concerns may include:
Git is central to the software-engineering workflow around dbt.
A typical process might be:
Developer creates branch
↓
Changes models
↓
Runs tests
↓
Opens pull request
↓
CI validates changes
↓
Code reviewed
↓
Merge
↓
Production deployment
This makes analytics development more reviewable and auditable.
Continuous integration can evaluate dbt changes before they reach production.
Checks may include:
For large projects, teams may avoid rebuilding the entire data warehouse for every pull request and instead use state-aware or changed-model workflows.
dbt provides tests, freshness, and project metadata.
Organizations may combine these capabilities with broader data-observability platforms.
Observability questions include:
dbt testing is an important part of data reliability while broader observability may extend across ingestion and downstream systems.
dbt model performance depends heavily on the underlying data platform.
Potential improvements include:
The right optimization begins by understanding:
Where is the time and warehouse compute actually being spent?
A dbt project can become expensive when:
Cost optimization may involve:
Data correctness remains the first priority.
The goal is trusted data without unnecessary compute.
A modern data workflow might look like:
dbt becomes the engineering framework connecting raw warehouse data with trusted business information.
A dbt Developer specializes in building and maintaining dbt transformation projects.
An Analytics Engineer frequently uses dbt as one of their primary tools while owning the broader transformation and analytics modeling layer.
A Data Engineer may use dbt for transformations while also owning:
A Data Warehouse Engineer may use dbt while focusing more deeply on:
A SQL Developer may contribute to dbt transformations while specializing more broadly in SQL.
A BI Developer may work downstream from dbt, using trusted models to create reports, semantic layers, and dashboards.
SQL is a language.
dbt is the framework surrounding analytical SQL transformations.
SQL can exist without dbt.
dbt projects generally depend heavily on SQL.
Use the SQL skill guide for deeper coverage of:
Use this dbt page for:
dbt specializes in data transformation.
Airflow provides general workflow orchestration.
dbt understands relationships between transformation models.
Airflow can coordinate broader tasks such as:
Companies frequently use both.
Traditional ETL platforms may transform information before loading it into the warehouse.
dbt generally works after data is loaded.
That approach allows teams to use warehouse compute for transformations.
dbt also emphasizes:
The best architecture depends on where transformations need to occur.
Stored procedures can perform substantial transformation logic inside databases.
dbt provides a different development model built around modular transformation models, dependencies, tests, documentation, and Git.
Existing stored procedures may remain appropriate for selected transactional or database-specific workflows.
Analytics transformations that have grown into a large network of procedures may benefit from the visibility and engineering workflow dbt provides.
Data engineering is a broad discipline.
dbt is one technology used within it.
A Data Engineer may manage:
dbt primarily focuses on the transformation and modeling portion of that stack.
dbt is a data transformation framework that allows teams to build modular, tested, documented, version-controlled transformations inside modern data platforms.
No.
dbt primarily uses SQL for transformation logic and Jinja for templating and reusable project behavior.
Models are transformation definitions, usually written as SQL SELECT statements, that dbt builds into views, tables, incremental datasets, or other supported materializations.
An incremental model processes selected new or changed data instead of rebuilding the entire dataset on every run.
dbt tests validate assumptions about data.
They can check properties such as uniqueness, nullability, relationships, accepted values, and custom business rules.
Macros are reusable Jinja-based functions that can generate or standardize SQL and other project logic.
Snapshots preserve historical versions of mutable source information so teams can analyze how records changed over time.
The dbt Semantic Layer allows teams to create governed metric definitions that can be reused by downstream analytics and other applications.
dbt Mesh provides architectural patterns for managing larger dbt environments across several teams and projects, including ownership, access, contracts, and cross-project dependencies.
No.
dbt executes transformations against a connected data platform such as Snowflake, BigQuery, Databricks, or Redshift.
dbt Developers, Analytics Engineers, Data Engineers, Data Warehouse Engineers, SQL Developers, and BI Developers may use dbt at different levels.
Understanding dbt helps you identify whether your data platform needs stronger transformation modeling, SQL, testing, lineage, documentation, semantic metrics, performance, or deployment practices.
If you need someone dedicated to building and maintaining your analytics transformation layer, South can help you hire dbt Developers in Latin America.
Schedule a free call and find remote data and analytics talent in Latin America with South.
