Niels ClaeysindatamindedbeThe building blocks of successful Data TeamsBased on my experience I will elaborate on key criteria for building successful data teams7 min read·May 3, 2024--4--4
Niels ClaeysindatamindedbeYou can use a supercomputer to send an email but should you?Discover the next evolution in data processing with DuckDB and Polars6 min read·Mar 12, 2024--1--1
Niels ClaeysindatamindedbeMy key takeaways for building a data engineering platformHaving been a member of a product team for two years, I aim to share three valuable insights that I have gained.6 min read·Feb 15, 2024--1--1
Niels ClaeysindatamindedbeQuacking Queries in the Azure Cloud with DuckDBThis post describes 2 Duckdb extensions that enable you to read data from Azure blob storage. It also shows code for both Python and dbt.7 min read·Jan 10, 2024--1--1
Niels ClaeysindatamindedbeHow we reduced our docker build times by 40%This post describes two ways to speed up building your Docker images: caching build info remotely, using the link option when copying files5 min read·Oct 4, 2023--13--13
Niels ClaeysindatamindedbeHead-to-head comparison of dbt SQL enginesCompare usage and performance of dbt against 3 popular open-source SQL engines, namely: Spark, Trino and Duckdb8 min read·Sep 8, 2023--5--5
Niels ClaeysindatamindedbeUse dbt and Duckdb instead of Spark in data pipelinesDbt has become very popular for transformation on top of your data warehouse. We see potential to use dbt with Duckdb on top of a data…7 min read·Apr 12, 2023--16--16
Niels ClaeysindatamindedbeWhy data engineers should be more like software engineersData engineers are better when using a product mindset as well as software best practices: cicd pipelines, test code, develop iteratively.7 min read·Jan 24, 2023----
Niels ClaeysindatamindedbeThe rise of remote development environmentsGitpod and Codespaces are the first remote development environments that we would use ourselves and may also be useful for you.5 min read·Nov 16, 2022----
Niels ClaeysindatamindedbeMake Spark resilient against spot interruptions on kubernetesBased on our experience of running spark in production at our customers, we discuss 3 ways to improve the resilience of spark on kubernetes7 min read·Jul 25, 2022----