Hero Image
GCP Batch Ingestion 1 - Local Development

An introduction to locally developing code for a batch ingestion process using python and the GCP API. Introducing the motivation behind this series of articles and the code concepts.

May 13, 2023 Read
Hero Image
Productionising the Process

So your code is working fine locally but you don’t want to be stuck running manual scripts every day. In this article, I introduce automating the process with Flask, Docker and Cloud Run.

May 8, 2023 Read
Hero Image
Finally, a better way to deploy DBT on Google Cloud!

For the last year or so I’ve been looking for a good way to productionise DBT pipelines on Google Cloud Platform but I’ve been frustrated by the solutions I’ve found on the web.

April 16, 2022 Read
Hero Image
Local Environment Setup For DBT With GCP Using Conda

This article is also on medium.com Setting up a local environment for DBT (BigQuery) using conda Recently dbt has been increasing in popularity as a structured and robust framework for writing SQL logic. Although the dbt documentation is a great resource, it is not too specific on local configuration. So here is a quick article on setting up a local dbt-bigquery environment with anaconda that works on both Windows and a Mac …

April 16, 2022 Read