Security Fix Ahead! Upgrade or Wait?

Hi ,

AI can be a effing nightmare ... sometimes. It turns criminals into criminals on steroids. Defense and prevention are difficult but sorely needed. But how can you protect code against unknown future attacks? Applying the Principle of Least Privilege to your code is a start (see featured article #1). And when a dependency your code uses turns out to have a vulnerability, immediate action is required—but at what risk? Any security fix might come piggybacked with malware snugged in through a supply chain attack. So better wait? A true dilemma (more in the Spotlight article).

On the lighter side of things, it must be oddly satisfying™ when you finally manage to squeeze more performance out of your algorithms, even if the overall performance gain for most real-world apps melts down to less than 1% (details in the second featured article). Much more satisfying than just popping bubble wrap! (And with more positive effects in the long run.)

I can feel where the motivation to fight for improvements, no matter how small, comes from: It's not perfectionism (which is driven by the fear of not being good enough) but rather plain "hey, let's make this better!" enthusiasm.

And ultimately, enthusiasm is (or, at least, should be) the launchpad for every great software project.

Have an enthusiastic week!

–Christoph

Featured articles

Secure Go Code using the Principle of Least Privilege | golangbot.com

Pass a *sql.DB pointer to an HTTP handler and you give it access to the whole database. This and other "accidental" security holes are avoidable, and the secret sauce isn't that much of a secret: It's the Principle of Least Privilege. Stick to it and eliminate a whole class of security vulnerabilities.

Size-Specialized Memory Allocation - The Go Programming Language

The garbage collector got a little bit faster in Go 1.27 by treating small allocations different than the rest. TL;DR: For small allocations (that is, up to 80 bytes), the GC doesn't reserve memory of the exact size; instead, it uses a range of size classes and reserves a memory chunk of the size class the allocation request falls into. For example, an allocation request for 29 bytes would fall into the 25-32 bytes class and hence reserve a 32-byte chunk of memory. Memory management of many 32-byte memory allocations can be made more efficient than managing many individually sized allocations between 25 and 32 bytes. Naturally, the performance gain is biggest if code only allocates such small chunks of memory; 20-30% speedup is possible. Real-world applications, though, are expected to be 1% faster with Go 1.27 but only if they're allocation-heavy.

Podcast corner

A totally racist episode so cache(sh) me outside how bout dah?!

No worries, Jonathan and Shay are just a bit "-race-ist" this time.

About the second part of the title, the clanker explained it to me as a 'play on the 2016 viral Dr. Phil meme "Cash me outside, howbow dah?"'

Am I too young or too old for 2016 memes?

Spotlight: The New Dependency Dilemma

Granted, the dilemma isn't really new. But AI lift it to new levels.

AI helps attackers find and exploit security holes much faster than ever before. To stay on the safe side, you'd have to update affected dependencies as quick as possible; otherwise, your app could be vulnerable to attacks. Yet, you shouldn't update any dependency in a haste, because nowadays, the rate of supply chain attacks increases, too. The dependency you need to update could contain malicious code that installs a backdoor or extracts data from your app. The reasonable response to this would be to let some time pass by before updating. The community or some security researchers might discover a supply chain attack during this time.

However, this cooldown phase contradicts the need to close security holes as soon as possible. It's a Catch 22: Decide for a quick update and you might install malicous code; decide for a cooldown phase and your app might get hijacked from the net. Either way seems wrong.

So what to do?

The perils of quick dependency updates

When one of your dependencies announces a new vulnerability, assume that a zero-day exploit is already created and actively scanning servers for vulnerabilities. It's only a matter of time until they find and attack your app. The sooner the issue is fixed in your app, the smaller the attack window.

But what if the fix is contaminated with malware? That's not a purely theoretical thought. Attacks like this happened already, and they often work like this: An attacker establishes themselves as a contributor to a popular open source project. When someone detects a vulnerability in the project, quick action is key, and the attacker is quick at providing a PR that fixes the vulnerability but also sneaks in malicious code. The urgency of the fix lets the rest of the team run the tests that prove the vulerability to be fixed but keeps them from investigating the PR deeper, possibly overlooking the malicious code.

Releasing the fix along with a prominent security advice causes dozens, hundreds, or thousands of users to upgrade their projects to include the fix. In no time, the malicious code has infected a large number of servers.

So let's wait a bit? For a day or a week, maybe?

This seems a valid mitigation strategy: Just wait a bit and see if someone discovers anything unusual. If no warning signals appear, go ahead and update the dependency. On a closer look, this strategy has two flaws:

1. It depends on others to eventually find something. How can you tell if anyone is actively scanning the code for a supply-chain attack? Does the lack of security alerts mean that the code is safe? Or did the malicous code simply evade detection?
2. It requires to decide how long to wait. The shorter the time span, the less likely is it that malicious code gets detected. The longer the time span, the longer your app is vulnerable to outside attacks.

So here we are: Quick is wrong, and slow is wrong.

Is there something we can do that's a bit less wrong?

Stay calm and weigh the risk carefully

Remember this is about critical fixes only. For any non-critical update to a dependency, a cooldown phase is the way to go. Go's dependency management system follows the minimum version selection (MVS) approach; so no dependency updates happen without you taking action (by bumping a dependency version in go.mod and calling go mod tidy, or by calling go get). You have the time to scan a new release for anything suspicious (or chase security scanners or LLMs across the diffs).

How the Go ecosystem can help

Go has two quite effective measures to secure a project's supply chain.

The first measure consists of the Go module proxy and the checksum database.

In a nutshell:

• Whenever a new module release (technically, a tagged commit) is pulled for the first time, the Go proxy calculates a cryptographic hash of the module and stores it in the Sum database
• Subsequent pulls are pinned to this exact hash, provided your toolchain uses a Go proxy for go getting dependencies (which is the default behavior of a newly installed Go toolchain but can be configured)

Manipulating the repository therefore has no effect on proxied downloads; even unpublishing the repo cannot break client code.

This mechanism ensures that tampering with an existing release is detected; however, neither the proxy nor the sum DB can protect the very first download of a release, when the proxy still has to calculate the hash. A malicious author or an attacker could provide a new release with malicious code in it, which is why it's so important to not download new releases right away.

While the Go proxy and the sum DB cannot prevent malicious releases, they protect you from post-publish tampering of whatever library you pull in. With Go's minimal version selection approach, you can stay on a known good version until the new release has been verified through close analysis.

The mechanisms behind Go's module managment that protect against supply chain attacks are much more detailed than I can describe in the context of this spotlight; for an overview, check out this Go Blog article.

How to treat a critical security fix

For critical releases, a quick reaction narrows the window where the app is vulnerable to attacks. Rather than setting a cooldown phase and hoping that others may scan the release and probably detect malicious code if there is any, it's safer to apply the usual mitigations (as one would do, BTW, in any ecosystem): diff reviews, provenance checks, and employing security scanners.

You could even automate the latter part by setting up your own custom Go proxy that runs security scans and vulnerability checks for every initial download that has no cryptographic hash yet. Have your favorite LLM screen even the innocent-looking parts of new code. Malware might mimic test code processing some compressed test data that actually contains the main part of the exploit (like in the famous xz incidenet).

Supply chain attacks become more sophisticated than ever while security vulnerabilities are exploited faster than ever. I'm not saying protection is easy, but every bit counts.

Remember seat belts that many considered pointless or even dangerous when introduced. Today, seat belts have saved countless of lives.

Fasten your security seatbelts.

Quote of the Week: Languages with fast compilers and tests

When generating tokens is not the bottleneck, it will suddenly matter a lot whether it can read a file in 100ms vs 10ms, or whether it can run your tests in 500ms vs two seconds. Fast tool calls are going to be the difference between a near-instant response and having to wait several minutes. There is thus going to be enormous pressure to do agentic coding in languages with fast compilers and tests, like Golang

– Sean Goedecke

🥳 Just what I keep saying: Go is the ideal language for agentic AI, on both sides: for driving the agents and being a handy tool for agents.

More articles, videos, talks

How Data Is Replicated In Distributed Systems | Raft From Scratch - Part 1 | Sushant Dhiman

Designing distributed apps isn't for the faint of heart: nodes can become unavailable in unpredictable ways, the network can break into disconnected subnetworks so that nodes only see a fraction of the other nodes, and network hiccups can delay delivery beyond timeouts. However, a distributed app needs to reach some sort of consensus between nodes about the overall state of the app. Consensus protocols like Raft help keeping an app running in the face of node or network failures, through constant evaluation of the network and leader election if the current leader node disappears. (Part one of a two-part article series.)

TIL: go fix Reaches Into Third-Party Code Too

The revamped go fix got much more powerful and even contains hidden gems: If you maintain a library and want to deprecate a function, you can add a go fix directive. Any library users who runs go fix on their code get their deprecated calls fixed without any action required from their side (besides running go fix, of course). Cool!

Projects

Libraries

DouglasMai4/kori

A small toolkit to complement Chi and add some convenience.

Tools and applications

eliau2005/statixagent: Self-hosted, featherweight VPS/laptop monitoring with a private Telegram bot per server

StatixAgent was designed to be minimal in features and infrastructural efforts: A single binary, no extra moving parts (such as a DB server), and just monitoring a single machine.

markel1974/webautoma: a powerful Go-based WebDriver W3C automation tool

Record web UI test steps with Selenium and run the resulting .side file with webautoma.

The author has begun to open source projects from their 4-decades long IT career; hence the projects are new (to the public), but I'd expect them to be quite mature.

incubator: goqu

A great idea: a tool that tests for code quality (in terms of Small files, documented symbols, tested exports and low complexity) rather than more "countable" but less meaningful metrics such as test coverage.

A terrible idea: naming the tool the same as an ORM that already went quite popular. (But hey, who scans all of GitHub, GitLab, Codeberg etc. for similarly named projects? Name clashes happen.)

ankushT369/gossh: gossh a lightweight SSH-over-HTTPS proxy for secure and firewall‑friendly remote access.

The author ported gossh from C ("the only language where [memory-based security issues] regularly happen" – Xe Iaso) to Go after they realized that they can develop software faster in Go.

chinmay-sawant/gowkhtmltopdf: Pure-Go HTML template engine: HTML→PDF and HTML→image for invoices, certificates, storybooks, posters, statements, and tables. No browser, no cgo, wkhtmltopdf work-alike.

What if you could generate PDF without a headless Chrome browser running in the background? gowkhtmltopdf aims at becoming a wkhtmltopdf alternative with a dramatically smaller footprint.

gojargo/jargo: Conversational-AI framework for Go.

Pipecat without Python: jargo is a "streaming transcription → reasoning → speech pipeline" that lets you build audio chatbots.

Completely unrelated to Go

Behind the Scenes: How the OpenTelemetry Plugin Maps Your Microservices in Real-Time

Architecture is widely thought of as something static, and when you look into a software architecture document and discover (often through hard work on the codebase) that it's helplessly outdated, you realize the deeper meaning of "static" here. JetBrains found an unusual way of reconstructing (or should I say: reverse-engineering?) the architecture of a software project: by examining traces. Like contrast medium injected into blood vessels, traces collected by telemetry infrastructure tell a detailed story where data flows from and to. With some sophisticated analysis steps, these traces turn into real-life maps of the system.

Jev means structured output is interesting again

A new kind of AI model has been released: Jev doesn't chat with you. Instead, it takes unstructured state with structured requests and replies with structured output: booleans, choices, and probabilities; each tagged with a level of confidence . And it is fast. The makers of Jev position it as an AI that talks to software rather than humans. Whenever a (Go) project needs a "smart" evaluation, a content classification, a fuzzy evaluation, or anything where hard-coded logic fails, it can ask Jev and get results back in milliseconds.

Sean Goedecke took a closer look.

Why do we listen to people with no skin in the game?

If you wanted to stop smoking or drinking, would you take advice from someone who never succeeded in getting away from smoking or drinking?

Software engineers, however, get advice from non-software engineers all the time. Most of the time, the correct reaction is to not listen to them. Or, listen to them and try to understand their perspective but remain confident in your own expertise.

Thorsten Ball - What I believe about the future of software development

"Yes, there are still Italian shoe makers around. But look at your feet."

You can run git on object storage if you re-make packfiles | Tigris Object Storage

"What is a Git? A miserable little pile of objects!" Xe Iaso builds a Git server backed by S3-like object storage, because... I dunno?