Performance, Security, and a Diamond (Problem) • The Applied Go Weekly Newsletter 2024-04-07

Your weekly source of Go news, tips, and projects

Recently, I have been thinking about the role of IT in consumption of natural resources and carbon dioxide production. Two areas truly stand out: Blockchains and Generative AI.

Bitcoin's energy consumption per year equals that of Egypt. Or Malaysia, or Sweden, depending on who you ask. GenAI consumes as much energy as (pick one:) Ireland, 40 million US households, or…

There are ways of reducing energy usage: Ethereum switched from proof-of-work to proof-of-stake. AI research works hard to find energy-efficient ways of training a model and increase the efficiency of AI hardware.

Granted, these are extreme examples. But the rest of the IT world suffers from inefficiencies, too. From over-engineered software architectures to badly chosen algorithms, many small choices can add up to an impressive level of wasted energy. Or, as a Scottish proverb goes, many a mickle make a muckle.

So, creating energy-efficient software starts in the small. This includes making decisions that appear to be negligible at first, such as deciding between returning a pointer to a data structure, or rather a copy of it.

Welcome to the world of counterintuitive results and to this issue of The Applied Go Weekly Newsletter.

Go: Pointers for Performance?

Passing pointers around is obviously faster than creating copies of data, right? Not so fast. Many pointers reference data allocated on the heap. Function calls, on the other hand, can copy the data on the call stack, which is a quite cheap operation. Software Chats examines a case where returning a large struct by value is much cheaper than returning only a pointer to the struct.

Go: Pointers for Performance? - YouTube

Go: Pointers for performance redux

Based on the many reactions to the previous video, Software Chats created a follow-up video that delves deeper into the nuances of pointers-vs-values performance measurements.

Go: Pointers for performance redux - YouTube

Prevent sensitive data from leaking

How can you prevent passwords, tokens, or other sensitive data from leaking into log files? With the help of only three interfaces from the standard library! Willem Schots explains how.

Prevent sensitive data from leaking in Go/Golang

New minor updates for Go: 1.22.2 and 1.21.9

Go 1.22.2 and Go 1.21.9 have been released. They include a security fix to the net/http package, as well as bug fixes to the compiler, the go command, the linker, and the encoding/gob, go/types, net/http, and runtime/trace packages.

And no, this security update is not related to the xz attack. This week's Cup O' Go episode discusses the security issue ("HTTP/2 CONTINUATION Flood") that these updates fix, including links to a 2-part mini blog series on that topic. (See the Podcast Corner below.)

Go 1.22.2 release notes

Podcast corner

Cup o' Go

Step 1: Update your Go installation to 1.22.2 or 1.21.9, respectively.

Step 2: Listen to this episode to learn about the HTTP CONTINUATION Flood attack.

Cup o' Go | 🌊 Avoid HTTP/2 floods, 🤐 don't log your secrets, and 🗣️ upcoming conferences

Go Time

The previous episode was about debugging, and this one traces Go's tracing abilities. Felix Geisendörfer, Michael Knyszek, and Natalie Pistunovich discuss tracing best practices, use cases, with an obligatory detour to AI.

The magic of a trace with Felix Geisendörfer & Michael Knyszek (Go Time #310)

go podcast()

Dominic St-Pierre works on a multi-tenant SaaS application. To deploy an app for a new tenant, he wrote a deployment orchestrator in Go.

go podcast() | 033: Deployment orchestrator in Go, part of my upcoming SaaS

Did you know...? Go has a Diamond Problem!

In object-oriented languages, the Diamond Problem is a well-known pitfall of the multi-inheritance model.

In a nutshell, inheritance enables classes (that is, data structures with methods attached) to take over code from other classes and expand or override parts of that code. For instance, imagine a "parent" class A that defines a method Foo(). "Child" classes derived from A can use, and even override, this method. Say, two classes, B and C, inherit from A, and each one changes method Foo() to behave slightly differently (but still in accordance with the original definition).

Now imagine a class D, which, thanks to multiple inheritance, inherits from both classes B and C. It does not override method Foo().

    +-+
    |A|
    +-+
   /   \
+--     --+
|B|     |C|
+-+     +-+
   \   /
    ---
    |D|
    +-+

What happens if code calls method Foo() on class D? Which of the three definitions of Foo() shall class D use?

This is the Diamond Problem, named after the diamond shape of the inheritance diagram.

What does all this has to do with Go? Without inheritance, let alone multiple inheritance, what could go wrong?

Turns out that it is entirely possible to construct a diamond-shaped dependency relationship in Go, through struct embedding. While struct embedding is not the same as inheritance, it also is a form of "promoting" methods from an embedded struct to the embedding struct. If an embedded struct B has a method Foo(), the embedding struct D can call Foo() directly instead of calling B.Foo(). You can see where this is going: IF struct D embeds B and C, which both define a method Foo() (that can optionally implement an interface A), then we have a Diamond Problem. (An interface A is not strictly needed, the problem can be constructed without it, effectively creating a "V-shaped" dependency. — Playground Link.)

Luckily, Go prevents you from inadvertently calling the wrong Foo(). Go's solution even made it into Wikipedia:

Go prevents the diamond problem at compile time. If a structure D embeds two structures B and C which both have a method F(), thus satisfying an interface A, the compiler will complain about an "ambiguous selector" if D.F() is called, or if an instance of D is assigned to a variable of type A. B and C's methods can be called explicitly with D.B.F() or D.C.F().

Another good example of why it is so important and helpful that a language catches as many problems as possible at compile time.

Quote of the week: Computing power

The world would probably be a better place if the absolute amount of computing power would no longer grow. Then development would have to focus on making things more efficient rather than relying on brute force everywhere.

– Felix Stalder

I second this. In the face of the projected rise in energy consumption of data centers over the next years that does not exactly help to keep global warming at bay, it is mandatory to think about ways of doing more computing with less energy, rather than the other way round. As Gophers, we have a wonderful tool right at our fingertips. Go is much more efficient than many languages in common use; moreover, its superfast compiler shortens development time (thus saving more energy). In the spirit of Go, the simple and boring language, we can use our brain capacity (which, BTW, is highly efficient, with an energy budget of only ∼20 Watts) to write simpler, faster, easier-to-maintain, boring code.

More articles, videos, talks

HTTP/2 CONTINUATION Flood - nowotarski.info

The two-part blog series about the HTTP CONTINUATION Flood attack featured in the Cup O' Go podcast starts here.

Building an interactive shell in Golang

The Dolt team uses abiosoft/ishell for their Dolt SQL shell. In this article, they show first steps for building your own interactive shell.

The Myth of Down Migrations; Introducing Atlas Migrate Down | Atlas | Manage your database schema as code

Database down migrations are a tricky thing. Defined ahead of an up migration, they need to anticipate a future state of the application, or else the down migration will fail. But what if the up migration fails and leaves the database in an undefined state? The down migration would then surely fail, too. The Atlas team goes a new way. Instead of writing down migrations in advance, their new down migration task calculates the necessary steps on the fly.

Deep Dive into Go SQL Internals. Exploring Connection Management and… | by Hossein Zolfi | Apr, 2024 | Medium

Deep knowledge about the interaction of Go and SQL databases can turn out to be worth a lot when troubleshooting or optimizing database-facing code. A great way of gaining such knowledge is to examine the database packages from the standard library.

Sounds tedious? Well, no worries. Hossein Zolfi already did it for you.

xgo: monkey patching in go using ‐toolexec | Blogs of xhd2015

In the Libs and Apps section, you'll find a new monkey patching tool, testaroli. Its announcement on /r/golang spawned a discussion between the author of testaroli and the author of xgo. The xgo author took this discussion as an inspriation to write an article about how xgo works.

Projects

Libraries

GitHub - g10z3r/ason: Go AST to JSON, with support for converting back to Go code

Serialize Go code to JSON and turn the JSON back into Go code. Looks similar to asty-org/asty.

GitHub - fasibio/autogql: CRUD SQL generator plugin for 99designs/gqlgen

A gengql plugin for GORM, similar to entgql but with a schema-first approach.

GitHub - gen2brain/jpegli: Go encoder/decoder for JPEG based on jpegli

A CGo-free wrapper around the original jpegli library. The original lib is compiled to a WASM and consumed via wazero.

GitHub - vacaramin/Go-Serve: GoServe is a collection of simple server projects, including proxy, reverse proxy, auth, api, chat, database, dhcp, file, game, grpc, iot, mail server, stream and url-shortner server

This project aims to collect various kinds of Go servers, to help beginners get a server project up and running without too much hassle. The owner asks for contributions.

GitHub - areknoster/hypert: Go package for rapid testing of real HTTP APIs integrations.

Record and replay API interactions for testing HTTP clients.

Tools and applications

GitHub - sentriz/untree: like gron, but generalised on indentation

Structured data in textual form is hard to search with simple, line-based tools like grep. Parsing the data is too costly most of the time. untree takes an "easy" way and determines structures by indentation, allowing you to get searchable (primarily greppable) data from structured text (like functions, structs, json data, etc).

GitHub - amalshaji/portr: Open source ngrok alternative designed for teams

Portr uses SSH remote port forwarding to expose local HTTP and TCP connections to the public internet. It is meant for use by small teams who want to expose development servers on a public URL.

GitHub - dhth/prs: Stay updated on PRs without leaving the terminal

A nice little, single-purpose TUI app. Display and navigate through your open PRs. Built with bubbletea.

figuerom16 / gosqliteadmin · GitLab

One of the goals of this project is to provide an example for creating a SQLite Query/Exec interface using only the Go standard library. But it also makes a nice, simple database explorer for your SQLite-based app. Supports both mattn/sqlite3 and modernc.org/sqlite.

GitHub - fastschema/fastschema: A simple and schema-based headless CMS

My first gut feeling would position FastSchema somewhere between Pocketbase and strapi.io. It's an interesting project in any case.

GitHub - qrdl/testaroli: Monkey patching unit testing utility for Go

Monkey patching is a testing technique that overrides functions with mocks or stubs directly in the binary. The use of this tool requires disabling all compiler optimizations and inlining. Also be aware of the pitfalls listed in the Wikipedia article liked from the README. Apart from that, this can be another tool to add to your testing tool belt if other unit testing approaches don't work out well for a specific purpose, or if you just dislike using dependency injection via interfaces. (Side note: see also xgo and gomonkey.)

Completely unrelated to Go

On March 29, Andres Freund discovered a backdoor in the xz compression library. Russ Cox did not waste any time and compiled a detailed log of the activities that helped the attacker gain trust as a project contributor and inject the malicious code.

research!rsc: Timeline of the xz open source attack

In a second post, Russ examines the shell script that was injected into a Makefile to insert an object file into the build pipeline.

research!rsc: The xz attack shell script

You can imagine what the attacker might have been thinking as this carefully planned and executed attack was crumbling to pieces!

Happy coding! ʕ◔ϖ◔ʔ

Questions or feedback? Drop me a line. I'd love to hear from you.

Best from Munich, Christoph

Not a subscriber yet?

If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding. 

Find the subscription form at the end of this page.

How I can help

If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all):

On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff.

Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff. 

My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go.

Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like.

Check it out.

Christoph Berger IT Products and Services
Dachauer Straße 29
Bergkirchen
Germany