It's (not) magic • The Applied Go Weekly Newsletter 2024-08-25
Your weekly source of Go news, tips, and projects
Magic can be wonderful. Shake a wand like a conductor of an orchestra and turn that impenetrable door into dust. Quaff a magic potion and gain enough superpowers to beat up a cohort of Roman soldiers in 50 BC. Use The Force to levitate an x-wing fighter.
Even programming languages can expose some forms of magic. But when can a language feature be considered magic? Difficult to say!
In the new Spotlight section below (which replaces the Go Tip section), I share some thoughts on language magic.
But first, here are the featured articles of the week: The magic of the map type's inner workings, the magic of designing new range loops with function iterators, and the magic of writing an LLM from scratch.
Featured Articles
Go Maps Explained: How Key-Value Pairs Are Actually Stored
The first part is a quick refresher of a map's behavior (and some quirks), but in section "Map Anatomy", things get interesting. Here, the article explores the hashmap structure of Go's map type. – By Phuong Le.
Range Over Function Types - The Go Programming Language
Ian Lance Taylor does a roundup on iterators: Why they were added to the language, how they work, and how to use them.
Llama Nuts and Bolts: A holistic way of understanding how Llama and its components run in practice, with code and detailed documentation.
Reinvent the wheel for fun and insights! Follow the journey of Adil Alper Dalkiran into the world of transformers and LLMs, and "develop a console application from scratch, without using any existing ML or mathematics libraries, which generates meaningful text outputs using pretrained Llama 3.1 8B-Instruct model weights."
Podcast corner
Cup o' Go: Where is the "Advanced Mode" switch?
Yeah, where is it? And where's Shay? And who's that new co-host?
Go Time: Lightning Talk life
The art of lightning talks, or: how to present a topic in a short timeframe.
Spotlight: Magic, or the lack thereof
"Any sufficiently advanced technology is indistinguishable from magic."
– Artur C. Clarke
Go is often called a non-magical language.
But what does "non-magical" exactly mean?
For years, I have used to point people to this rant in the middle of Joel Spolsky's article "Make Wrong Code Look Wrong," to give them an idea of magic vs. non-magic code. (If the link does to show the highlight right away, search for "language features that hide things" in that article.)
Now, thanks to Jeremy Bowers, aka Jerf, I have a more general and more nuanced take on the magical aspect of programming languages to refer to:
A Definition of Magic in Programming Languages
(Fun fact: Jerf's article goes back to a 2017 post on HackerNews that is probably the first time Go was labelled non-magical. Even Peter Burgon's classic article A theory of modern Go that probably made the notion of Go as a non-magical language popular (don't hold me accountable on this postulate) links back to the HN post.)
In his article, Jerf unfolds several ways in which language features can be considered magical, and he concludes that magic in code is not inherently good or bad, but the benefit that a magic feature adds must be carefully weighed against the cost it adds. (Where cost mostly equals to the added mental burden of deciphering the effects of code using, or being based on, a magic feature.)
I don't have to look far to find an example of magic that was largely appreciated by its users. Years ago, my favorite scripting language was Perl. This language was full of magic. For example, consider the following tiny piece of Perl code:
while (<>) {
print if /perl/;
}
If you don't know Perl, it's almost impossible to deduce what this code does. The first magic involved here is the <> operator (also called diamond operator). It's nothing more than a less-than and a greater-than symbol concatenated, but it does the bulk work of this code:
For every file name listed on the command line that invoked this script, <> opens that file, assigns each line of this file to the default variable $_ (another piece of magic) and yields this variable to the loop body. If no file names are passed as arguments to the script, <>reads from stdin. Theprintline in the loop's body tests the content of$_ against a regular expression and prints the content of $\_ if the regexp matches.
Easy, huh? ;-)
There is surely enough magic in those three lines to make everyone not familiar with Perl have a "WTF?!" moment. Yet, this pattern is so ubiquitous in Perl scripts that virtually every Perl programmer would recognize it at the blink of an eye. This is a rare example of magic without cost.
I think this Perl example shows the most prevalent benefit of magic language features: the avoidance of boilerplate code. Ater all, boilerplate code quickly becomes boring.
On the other hand, the biggest drawback of magic code features is that they tend to tuck away masses of logic behind innocent-looking symbols or keywords. It's the readers task to track down the origins of a magic piece of code, often having to dig through levels over levels of indirections.
I agree with Jerf that code magic is inherently neither good nor bad—see the Perl example above for an example of good magic. However, the more magical features a language provides, the more are the users of that language tempted to act "clever" (or so they think) and use all these features to their extreme, just to save a few keystrokes, or maybe even just to earn some bragging rights.
Nah.
I prefer verbosity to cleverness. Reading and comprehending code should be a straightforward and mostly linear task. I don't want to jump through hoops to understand what that plus sign in front of me actually does. I'd rather read through some extra amount of code if this enables me to quickly and unambiguously comprehend the code I'm reading.
Clear is better than clever.
– Go proverb
Quote of the Week: Durable software
Most software out there is incurably infected by incentives to serve lots of people in the short term. Focus as far as possible on software without lots of logos on the website, stuff that is easy to build, has few dependencies, definitely doesn't auto-update. Once you filter by these restrictions, the amount of durable software humanity has created so far is tiny.
More articles, videos, talks
Iterators: storing data in control flow (iterator functions) is objectively better than storing data in structs (iterator interface+impl)
The new function-based iterators could have well be designed as iterator interfaces instead. Redditor u/zldu demonstrates why function-based is better.
Pavel Gabriel: Mastering ISO 8583 messages with Golang
ISO... what? ISO 8583 is an "international standard for card-originated financial transaction messages that defines both message format and communication flow."
In this article, Pavel Gabriel describes how he implemented the Go package moov-io/iso8583. Even if you are only marginally interested in financial transaction thingies, this article might serve as a blueprint for implementing similar ISO standards.
GopherCon UK 2024 · Jamie Tanna | Software Engineer
Jamie Tanna visited GopherCon 2024, and here he shares his conference notes, including how he got nerd-sniped.
Wails usage survey
Attention Wails users: The Wails team invites you to shape the future of Wails by taking a user survey.
GitHub - uber-go/mock: GoMock is a mocking framework for the Go programming language.
Mock or stub? See also this classic article from Martin Fowler.
Performance optimizations of WebSocket compression in Go application | Centrifugo
Compressing WebSocket traffic? What would that buy me?
Well, up to $12,000 of savings per month. At least it did for one of Centrifugal Lab's customers. – By Alexander Emelin
Introducing FauxRPC
Feed this server with delicious protobufs, and it'll start speaking gRPC/gRPC-Web/Connect and REST to your apps.
How to implement Server-Sent Events in Go | by Alex Pliutau | Aug, 2024 | ITNEXT
You don't need a package for using Server-Sent Events (SSE) with Go. Alex Pliutau shows you how to write SSEs from scratch.
A downside or two of function keyword arguments (and default values)
Do you wish Go had keyword arguments (like, f(length=5)) or optional arguments? Chris Siebenmann isn't excited about either of these, and in this article, he explains why. In this follow-up article, he explores the difficulties of actually implementing keyword arguments: It's not simple to add function keyword arguments to Go
SentencePiece BPE Tokenizer in Go
Want a challenge? Implement a SentencePiece tokenizer that Google's AI models use to chop prompts into tokens. Or maybe take the easier road and use Eli Bendersky's go-sentencepiece package.
Projects
Libraries
GitHub - Oudwins/zog: Go with simple schema validation
Zog is Zod for Go.
GitHub - rschio/grafo: Graph Theory package.
This is the first library using function iterators that I see in the wild. Almost all graph functions are based on an interface with a depth function and an iterator.
(Tip: Find an introduction to graph theory (with Go) at yourbasic.org.
GitHub - absolutelightning/go-immutable-adaptive-radix
A drop-in replacement for hashicorp/go-immutable-radix.
Tools and applications
GitHub - Oudwins/zog: Go with simple schema validation
Zog is Zod for Go.
GitHub - rschio/grafo: Graph Theory package.
This is the first library using function iterators that I see in the wild. Almost all graph functions are based on an interface with a depth function and an iterator.
(Tip: Find an introduction to graph theory (with Go) at yourbasic.org.
GitHub - absolutelightning/go-immutable-adaptive-radix
A drop-in replacement for hashicorp/go-immutable-radix.
Completely unrelated to Go
Let small fires burn | Swizec Teller
Bubble-sort your bugs!
If you try to fix too many tiny bugs to please your customers, you risk letting the serious ones slip through and wreak havoc.
Swizec Teller recommends comparing your bugs pairwise to identify the more important one of each pair and repeating this step until all bugs are sorted by importance.
How we deleted 4195 code files in 9 hours - by Anton Zaides
Managers who love to measure developer productivity by lines of code will hate this weird trick!
Or: Why cleanathons could be the next hot thing after hackathons.
Happy coding! ʕ◔ϖ◔ʔ
Questions or feedback? Drop me a line. I'd love to hear from you.
Best from Munich, Christoph
Not a subscriber yet?
If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding.
Find the subscription form at the end of this page.
How I can help
If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all):
On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff.
Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff.
My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go.
Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like.
Christoph Berger IT Products and Services
Dachauer Straße 29
Bergkirchen
Germany