• Posts
Data Science
• Batch Ingestion
  • 1. Local Development
  • 2. Productionising
DBT
• Local Setup GCP
• Production Setup GCP
Web Development
• Src Image GDrive
Markdown Sample
Shortcodes Sample

GCP Batch Ingestion 1 - Local Development

An introduction to locally developing code for a batch ingestion process using python and the GCP API. Introducing the motivation behind this series of articles and the code concepts.

May 13, 2023 Read

Productionising the Process

So your code is working fine locally but you don’t want to be stuck running manual scripts every day. In this article, I introduce automating the process with Flask, Docker and Cloud Run.

May 8, 2023 Read

Google Drive Source Image

This is a quick tutorial which shows you how to use images from your google drive on a website. First create a new folder on your Google Drive if you don’t have one already. Usually, when I create a website I like to have a folder insite my website directory called “Static” with all of the images for the site. I will make this folder public to avoid repitition. Next, make sure that the folder is shared publicly.

March 27, 2023 Read

Finally, a better way to deploy DBT on Google Cloud!

For the last year or so I’ve been looking for a good way to productionise DBT pipelines on Google Cloud Platform but I’ve been frustrated by the solutions I’ve found on the web.

April 16, 2022 Read

Local Environment Setup For DBT With GCP Using Conda

This article is also on medium.com Setting up a local environment for DBT (BigQuery) using conda Recently dbt has been increasing in popularity as a structured and robust framework for writing SQL logic. Although the dbt documentation is a great resource, it is not too specific on local configuration. So here is a quick article on setting up a local dbt-bigquery environment with anaconda that works on both Windows and a Mac …

April 16, 2022 Read

Markdown Samples

This is a sample post intended to test the followings: A different post author. Table of contents. Markdown content rendering. Math rendering. Emoji rendering. Markdown Syntax Rendering Headings The following HTML <h1>—<h6> elements represent six levels of section headings. <h1> is the highest section level while <h6> is the lowest. H1 H2 H3 H4 H5 H6 Paragraph Xerum, quo qui aut unt expliquam qui dolut labo. Aque venitatiusda cum, voluptionse latur sitiae dolessi aut parist aut dollo enim qui voluptate ma dolestendit peritin re plis aut quas inctum laceat est volestemque commosa as cus endigna tectur, offic to cor sequas etum rerum idem sintibus eiur?

June 8, 2020 Read

Shortcodes Samples

This is a sample post intended to test the followings: Default hero image. Different shortcodes. Alert The following alerts are available in this theme. This is sample alert with type="success". This is sample alert with type="danger". This is sample alert with type="warning". This is sample alert with type="info". This is sample alert with type="dark". This is sample alert with type="primary". This is sample alert with type="secondary". Image A sample image without any attribute.

June 8, 2020 Read