An MVCC-like columnar table on S3 with constant-time deletes

- Parquet is excellent for analytical workloads. Columnar layout, aggressive compression, predicate pushdown, but deletes require rewriting entire files. Systems like Apache Iceberg and Delta Lake solve this by adding metadata layers that track delete files separately from data...

Exploring PostgreSQL to Parquet archival for JSON data with S3 range reads

- PostgreSQL handles large JSON payloads reasonably well until you start updating or deleting them frequently. Once payloads cross the 8 KB TOAST threshold and churn becomes high, autovacuum can dominate your I/O budget and cause other issues. I have been exploring the idea of...

How to Use an AWS S3 Bucket as a Pulumi State Backend

- Create an S3 Bucket, create an IAM User, use IAM credentials with the Pulumi CLI....

Seven Years of Firecracker

- Seven Years of Firecracker Time flies like an arrow. Fruit flies like a banana. Back at re:Invent 2018, we shared Firecracker with the world. Firecracker is open source software that makes it easy to create and manage small virtual machines. At the time, we talked about...

Computer architecture

Meanwhile in DevOps #43: Ellis manages productivity

- Ellis has called a team meeting. Arlo, Murray, and Ellis are sitting in the boardroom drinking coffee. Arlo looks a little worried, and Murray looks like he’s thinking of being on holiday. “We need to boost productivity,” Ellis begins. “What do you need?” “Our machines are...

Using Claude Code from Anywhere

- I've been using multiple instances of Claude Code and Codex CLI almost every day. But I've gotten frustrated enough to build something that solidifies my workf...

Meanwhile in DevOps #37: Doug innovates

- Murray and Doug are passing in the hallway. Murray is looking confused and a little vacant. “I’ve lost my earbuds again!” Murray laments. Doug has a think and then hits a lightbulb moment. He starts sketching a pair of wireless earbuds. He then adds a line between the two buds...

Cross-Account and Cross-Region Backups with AWS Backup (and friends)

- In today’s edition of “don’t trust LLMs”, we learn that despite what AI tells you, AWS Backup doesn’t support Cross-Account and Cross-Region backups. It supports Cross-Account copying and Cross-Region copying, but apparently not together. As part of Masset’s Data Protection and...

Picking uncontested private IP subnets with usage data

- Picking uncontested private IP subnets with usage data If the device you are reading this on has an IPv4 address, it is very likely not a publicly routeable one. This is because the wide scale dep...

Save yourself time: Set the JVM TTL for DNS name lookups in AWS

- I like to keep track of times I’m one of the lucky 10,000. Had such an instance today with the JDK and DNS behavior. Apparently most versions of the JDK cache DNS lookups indefinitely. If that’s not surprising to you, I’m guessing you’ve had a weekend debugging session similar...

Decomposing Aurora DSQL

- Decomposing Aurora DSQL Riffing, I guess. Earlier today, Alex Miller wrote an excellent blog post titled Decomposing Transaction Systems. It’s one of the best things I’ve read about transactions this year, maybe the best. You should read it now. In the post, Alex breaks...

Please stop externalizing your costs directly into my face

- This blog post is expressing personal experiences and opinions and doesn’t reflect any official policies of SourceHut. Over the past few months, instead of working on our priorities at SourceHut, I have spent anywhere from 20-100% of my time in any given week mitigating...

On How We Moved to Kubernetes

- As our deployment and CI costs grew, we started looking into migrating off from AWS ECS to AWS EKS. Here’s a brief history of this change.

Set up a GitHub Action to perform nightly Postgres backups

- In this guide, you'll learn how to configure nightly Postgres backups using a scheduled GitHub Action and pg_dump. This is part two of a two-part guide. Go bac......

Create an S3 bucket to store Postgres backups

- This guide will walk you through setting up an AWS S3 bucket to store your Postgres backups. This is part one of a two-part guide. Continue to part two here. ......

How to Send email using Gmail and GitHub Actions

- If you've found yourself here, you're probably looking for a way to send an email using GitHub Actions. Great—I needed to do this too! I'll briefly explain why,......

Blocking Spam Email Domains with Self-Hosted Ghost

- I noticed an uptick in spam signups on my self-hosted Ghost instance. Specifically those coming from email to SMS gateways[1]. I suspect that they use the email address to send a text message to verify the phone number. I'm not sure how many people actually use these...

Fix "Efficient Cache Policy" Warning on PageSpeed Insights When Using Amazon CloudFront

- Get rid of the efficient cache policy warning on PageSpeed Insights by adding a Cache-Control header to CloudFront....

How to Add a Custom Response Header to an Amazon Cloudfront Distribution

- Adding a custom header to AWS CloudFront Distribution responses....

100K Views, 1K GitHub Stars, and 100+ Messages Later

- Recently I shared my startup journey on LinkedIn. In...

DSQL Vignette: Wait! Isn't That Impossible?

- DSQL Vignette: Wait! Isn’t That Impossible? Laws of physics are real. In today’s post, I’m going to look at how Aurora DSQL is designed for availability, and how we work within the constraints of the laws of physics. If you’d like to learn more about the product first, check out...

DSQL Vignette: Transactions and Durability

- DSQL Vignette: Transactions and Durability The hard half of a database system? In today’s post, I’m going to look at the other half of what’s under the covers of Aurora DSQL, our new scalable, active-active, SQL database. If you’d like to learn more about the product first,...

DSQL Vignette: Reads and Compute

- DSQL Vignette: Reads and Compute The easy half of a database system? In today’s post, I’m going to look at half of what’s under the covers of Aurora DSQL, our new scalable, active-active, SQL database. If you’d like to learn more about the product first, check out the official...