Consistent hashing In this blog post we will dive in to consistent hashing and implement it in go. Lets start with a problem that consistent hashing would help solve. Imagine we have a distributed systems with 3 database. Business is booming and we realize we need to scale out to more shards. Lets assume we selected which node to send the data to using a hash function f(x) and get the node:…

The blog post will just be a long list of short summaries of python tricks, that I find good and like to remember(and find fast). Lets go! Table of Contents Example Example2 Third Example Fourth Example List List comprehensions List comprehensions are a neat one liner python trick. Not only is it handy it also sometimes faster. a = [1, 2, 3, 4, 5] b = [val**2 for val in a] Lazy initialize list To initialize a list with x values that have the same number:…

In this blog post we will go over some learnings from writing recursive queries. For example data we will use the Recursive queries blog post and the set up provided in the post as well, and thus not go over how to bring up the env in this post. Avoiding cycles Writing Recursive queries it is easy to have cycles that. For a process at work I tried to do a graph traversal but ended up in a infinite loop, to avoid spilling sensitive information we will rely on a mock example based upon this example.…

TLDR: https://github.com/Njorda/cloudsql/tree/main The goal with this blog post is to build a small tool to query Google Cloud buckets. We will do this using ChatGPT, I feel like I need to be even better at prompting for coding help. So to start by setting some constraints to limit the scope and make it a reasonable task to finish in a hour we will use go and we will limit our self to SELECT with WHERE we will support predicate push downs but thats it.…

I coupe of weeks ago I came across that Postgres supports recursive queries and thus we will take a deep dive in this blog post on the concept of recursion in SQL. The docs for postgres and recursive queries can be found here if you are not familiar with CTE I recommend you to start with that. In this blog post we will start with exploring this concept in Postgres before we jump to DuckDB.…

All the code can be found here In this blog post we will dive in to how to set up neon on your k8s cluster. We will use Minikube but feel free to use the setup of your choice of k8s. The first step is to define the k8s resources. In this case we will take a short cut and start of with generating them from the docker compose files used for testing.…

Way finetune Finetuning gpt turbo seams to lead to a couple of improvements Improved steerability Reliable output formatting Custom tone Decreased prompt size The code for this tutorial can be found here Step by step guide First you need to create an openAI api_key. The eaiest way to do this is through the openAI web page under API keys. . Copy the key and add it to an env file with the name open_ai.…

Run docker container in GCP cloud shell In this blog post, we will briefly go over how to build and run a Docker container as quickly as possible on Google Cloud Platform (GCP). First, you need to create a Docker image. Something very simple should suffice. Here is a sample Dockerfile: FROM python:3 RUN apt-get update && apt-get install -y vim RUN python -m pip install pandas After creating the Dockerfile, the next step is to build and push the image to Artifact Registry.…

Gorutine high level Go:s native concurrency features is usually one of the first things developers bring up when describing the advantages of using go. However due to its eas of use it is also wildly miss used. “You can have a second computer once you’ve shown you know how to use the first one.” –Paul Barham We will not deep dive in to when to use and not to use Gorutines specifically even though we touch upon some hints.…

The current development in CPU design is going towards large amount of cores rather than faster cores and thus writing parallel code becomes more important in order to utilize the full potential (Concurrency is not Parallelism). In this blog post we will dive into scoped threads, what it is and what is the difference between threads in rust in general. First of all only use threads if you need the speed up, introducing threads to a program adds complexity which both makes the program harder to maintain but if not done correct also slower to run(due to communications between threads and scheduling).…