This week
- Ellis is set up at a big AI conference, ready to draw in the punters to his new company, Pik-Tec. He’s standing in front of a large banner that reads, “AICon: Not just AI, but also a con.” “You should totally buy our AI image generator,” Ellis calls out to an attendee. “No...
About a month ago
- How to set up GitHub Actions for Pulumi when the state is stored in an AWS S3 Bucket....
about 2 months ago
- The whole team is in a panic. Doug is frantically reviewing paperwork, and Ellis is running around with his hands in the air. “Our cloud bill is out of control,” cries Doug. “Yikes,” Ellis shouts. “An extra fifty thousand dollars every month.” “That’s odd,” one of the developers...
- A database permissions change ended up knocking Cloudflare’s proxy offline. Pinpointing the root cause was tricky – but Cloudflare shared a detailed postmortem. Also: announcing The Pragmatic Summit...
2 months ago
- I’ve been curious about how far you can push object storage as a foundation for database-like systems. In previous posts, I explored moving JSON data from PostgreSQL to Parquet on S3 and building MVCC-style tables with constant-time deletes using S3’s conditional writes. These...
- DSQL: Simplifying Architectures Complexity is a choice. While we were designing and building Aurora DSQL, we spent a lot of time thinking about our experience building and running database-backed systems. We saw that building great, fast, cost-effective, highly-available,...
3 months ago
- On Monday, a major AWS outage hit thousands of sites & apps, and even a Premier League soccer game. An overview of what caused this high-profile, global outage...
- Ellis and Blake are having a catch-up over a coffee. Ellis is looking a little stressed. “We hired a new product manage,” Ellis explains. “So we can be more user-centric.” “Great!” Blake responds. “How often will they speak to the users?” There’s a long silence, which gives...
- In the previous post, I explored a Parquet on S3 design with tombstones for constant time deletes and a CAS updated manifest for snapshot isolation. This post extends that design. The focus is in file delete operations where we replace a Parquet row group and publish a new...
- Parquet is excellent for analytical workloads. Columnar layout, aggressive compression, predicate pushdown, but deletes require rewriting entire files. Systems like Apache Iceberg and Delta Lake solve this by adding metadata layers that track delete files separately from data...
- PostgreSQL handles large JSON payloads reasonably well until you start updating or deleting them frequently. Once payloads cross the 8 KB TOAST threshold and churn becomes high, autovacuum can dominate your I/O budget and cause other issues. I have been exploring the idea of...
4 months ago
- Why GitHub Actions runners are slow and how bare metal servers can make your CI/CD 2-10x faster while costing 10x less...
- Self-host Pulumi state with an S3 Bucket, an IAM User, and the Pulumi CLI....
- Seven Years of Firecracker Time flies like an arrow. Fruit flies like a banana. Back at re:Invent 2018, we shared Firecracker with the world. Firecracker is open source software that makes it easy to create and manage small virtual machines. At the time, we talked about...
- Ellis has called a team meeting. Arlo, Murray, and Ellis are sitting in the boardroom drinking coffee. Arlo looks a little worried, and Murray looks like he’s thinking of being on holiday. “We need to boost productivity,” Ellis begins. “What do you need?” “Our machines are...
- I've been using multiple instances of Claude Code and Codex CLI almost every day. But I've gotten frustrated enough to build something that solidifies my workf...
5 months ago
- Dynamo, DynamoDB, and Aurora DSQL Names are hard, ok? People often ask me about the architectural relationship between Amazon Dynamo (as described in the classic 2007 SOSP paper), Amazon DynamoDB (the serverless distributed NoSQL database from AWS), and Aurora DSQL (the...
6 months ago
- I Spent $450 in 3 Weeks Building 100k Lines of Code (And Didn't Want to Burn It Down)...
- Murray and Doug are passing in the hallway. Murray is looking confused and a little vacant. “I’ve lost my earbuds again!” Murray laments. Doug has a think and then hits a lightbulb moment. He starts sketching a pair of wireless earbuds. He then adds a line between the two buds...
7 months ago
- In today’s edition of “don’t trust LLMs”, we learn that despite what AI tells you, AWS Backup doesn’t support Cross-Account and Cross-Region backups. It supports Cross-Account copying and Cross-Region copying, but apparently not together. As part of Masset’s Data Protection and...
- Picking uncontested private IP subnets with usage data If the device you are reading this on has an IPv4 address, it is very likely not a publicly routeable one. This is because the wide scale dep...
- I like to keep track of times I’m one of the lucky 10,000. Had such an instance today with the JDK and DNS behavior. Apparently most versions of the JDK cache DNS lookups indefinitely. If that’s not surprising to you, I’m guessing you’ve had a weekend debugging session similar...
9 months ago
- Decomposing Aurora DSQL Riffing, I guess. Earlier today, Alex Miller wrote an excellent blog post titled Decomposing Transaction Systems. It’s one of the best things I’ve read about transactions this year, maybe the best. You should read it now. In the post, Alex breaks...
Rows per page
