Skip to content

This dbt package contains a set of pre-built, pre-integrated Load and Transform dbt models for common SaaS applications.

Notifications You must be signed in to change notification settings

mouldyjon/ra_data_warehouse

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

This repository contains a set of pre-built dbt Load and Transform models for common SaaS applications to create an integrated ("conformed") data warehouse dimensional model. In its current incarnation it supports Google BigQuery as the target data warehouse and Stitch (and Segment warehouse destinations) as the data pipeline technology.

Current Dimensional Model

Dimensional Model

Design Goals

  1. To standardise how we source and model SaaS data sources in-general
  2. To make it simpler to run data quality tests than to not, by defining these tests in-advance
  3. To enable merging of customer, product, contact and other shared entity data with no single authoratitive source
  4. To pre-create derived analytics measures for individual and combinations of sources
  5. In-time, to create a means of selecting sources or subject areas ("modules") and have just those sources/modules loaded (and deployed for a customer)
  6. To do all of this in a way that embraces, rather than avoids, community additions to these sources and derived analytics models

What are the Layers in the Warehouse Data Model?

Model Layers

See Design and Coding Approach for Implementation Details

What SaaS Sources are Currently Supported?

  • Hubspot CRM
  • Harvest Timesheets
  • Xero Accounting
  • Stripe Payments
  • Asana Projects
  • Jira Projects
  • Mailchimp Email Marketing
  • Segment Events
  • GCP Billing Exports
  • Google Ads
  • Facebook Ads

data sources on roadmap for Q2 2020 include:

  • Mixpanel Events
  • Salesforce CRM
  • Bamboo HRMS
  • Chargebee Payments
  • Intercom Messaging
  • Google Analytics (Free & 360)

What Warehouse modules are Currently Modelled?

  • Finance (Invoices, Chart of Accounts, Currencies)
  • CRM (Deals, Contacts, Companies)
  • Projects (Timesheet Projects, Timesheet Tasks, Delivery Projects, Delivery Tasks, Timesheets, Users)
  • Marketing (Email lists, Email sends, Email campaigns, Ad Campaigns, Ad Performance, Web Page Views, Web Sessions)

How Do We Setup a New Environment for Testing, Client Deployment?

See Setting up a new Warehouse Environment for instructions on how to set-up your own dev environment, or a new client environment

About

This dbt package contains a set of pre-built, pre-integrated Load and Transform dbt models for common SaaS applications.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages