A Gopher, an Enum, and an Undefined • The Applied Go Weekly Newsletter 2024-02-25
Your weekly source of Go news, tips, and projects
A Gopher, an Enum and an Undefined walk into a bar...
I don't know how to continue that joke, but after reading the article on enums and the Go Tip on undefined JSON data, perhaps you can come up with a punch line.
But more realistically, you probably will be tackling the one billion rows challenge, listen to the latest podcast episodes, try out GoTH, work on calling C from Go, or try to wrap your brain around hashsplitting.
That's totally ok, as long as you're done with all that until the next issue.
Go Enums Suck
I tend to disagree: Enums in Go cannot suck because they don't even exist. Haha. Joking aside, the lack of enums forces developers to construct some DIY enums, usually using constants and iota. This article shows the limitations of this approach and works through each of them in an attempt to create a feasible enum construct. Finally, the author decided to write an enum generator package that takes all the tedious steps away from the developer.
One Billion Rows Challenge in Golang
Matt Boyle tackled the one billion row challenge. It took quite a few iterations to optimize the code, but the result is an impressive speedup from 6 minutes down to 14 seconds.
One Billion Rows Challenge in Golang
Robust generic functions on slices
How to use the slices package more efficiently (by learning about slice internals).
Robust generic functions on slices
Podcast corner
Cup o' Go
This week's Cup O' Go episode is all about proposals: one declined, four (likely) accepted ones, and a blog post about the rangefunc proposal.
Cup o' Go | 🔁 Iterating through the week's news
Go Time
Is your Go app too slow? Look at these things first.
Foundations of Go performance with Miriah Peterson & Bryan Boreham (Go Time #304)
Go tip of the week: nil, null, and nothing at all
Programming languages tend to differ about the notion of absence of values. This is particularly apparent for Go developers when writing code for unmarshaling JSON data. The problem here is that Go's nil does not match well with JSON's ideas about the absence of values.
JSON data has two variants of absent values:
- A field exists, and its value is
null. Example:{"Name": "Platypus", "Order": null} - A field does not exist at all. Example:
{"Name": "Quoll"}. Here, theOrderfield is completely absent.
(The examples are borrowed from the json package's Unmarshal example.)
Javascript, the language JSON originates from, has no problem with case #2. If Javascript code reads the above JSON data, Quoll's Order field would be set to undefined.
Go has no notion of undefined variables. If a variable is created without assigning anything, Go assigns this variable a zero value.
(Examples of the zero value are 0 for integers, "" for strings, or nil for slices, maps, interfaces, channels, pointers, and functions.)
This concept simplifies variable handling a lot, but it can go in the way of unmarshalling JSON.
Assume you want to update data on a remote service. You will want to have a way to say, "update this field's value to null" and a way to say, "don't touch this field, leave it as it is."
To illustrate this, let's modify the Unmarshal example from the json package a bit.
The following code defines an Animal struct. The Order field shall be optional, and we model this by using a pointer.
package main
import (
"encoding/json"
"fmt"
)
type Animal struct {
Name string
Order *string
}
The code shall read a JSON array with two elements into an Animal variable.
- The first entry's Order value is
null - and the second entry has no Order value.
func main() {
var jsonBlob = []byte(`[
{"Name": "Platypus", "Order": null},
{"Name": "Quoll"}
]`)
So let's define an empty slice of Animals and unmarshal the JSON blob into that slice.
animals := []Animal{}
err := json.Unmarshal(jsonBlob, &animals)
if err != nil {
fmt.Println("error:", err)
}
If we now print each animal's name and order, what do you think gets printed for the two Order fields?
for _, a := range animals {
fmt.Printf("Name: %s, Order pointer: %v", a.Name, a.Order)
if a.Order != nil {
fmt.Printf(", Order: %s", *a.Order)
}
fmt.Println()
}
}
You're correct (or so I assume): Both are nil.
Name: Platypus, Order pointer: <nil>
Name: Quoll, Order pointer: <nil>
(Run this code in the Playground.)
How can we know whether an order field is set to null (meaning: "that animal has not been assigned any order yet"), or whether it was absent in the JSON data (meaning: "that animal may or may not have an order, we just don't know this right now")?
If a concept is not part of the language, we can model this concept instead.
So if we want to have a data type that can be undefined, let's construct a data type with an attribute that signals (un-)definedness.
Nothing easier than that: a simple struct type is sufficient.
type Order struct {
Value string
IsDefined bool
}
Now we can replace the Order pointer by our new Order type.
type Animal struct {
Name string
Order Order
}
What does this buy us?
How can we get the undefined status of Quoll's order into our Order type when unmarshalling the JSON blob?
Easy: Write a custom UnmashalJSON() method for the Order type. The json.Unmarshal() function looks automatically uses such custom unmarshal methods.
We use this feature to set the IsDefined field as soon as this method is called. (Because if this method is called, this means that json.Unmarshal() has found an Order field in the JSON blob.)
The function then only need to treat the special case of "null", in which case it leaves the Name field empty, and then unmarshal the field's value as usual. If the input data contains no valid string value, json.Unmarshal() returns an error; otherwise, it fills the Order field with the unmarshaled value.
(For the "null" case, you could also decide to write something like "null", "no order", "unknown", etc. into the Order field.)
func (o *Order) UnmarshalJSON(bytes []byte) error {
o.IsDefined = true
if string(bytes) == "null" {
return nil
}
if err := json.Unmarshal(bytes, &o.Value); err != nil {
return err
}
return nil
}
Now we only need to change the print loop to inspect the IsDefined field.
func main() {
// ...
for _, a := range animals {
fmt.Printf("Name: %s, Order: '%s', Order is defined: %t\n", a.Name, a.Order.Value, a.Order.IsDefined)
}
}
The output tells us if a field is undefined or defined but null.
Name: Platypus, Order: '', Order is defined: true
Name: Quoll, Order: '', Order is defined: false
(Run this code in the Playground.)
(IRL, both animals belong to an order. A platypus is a monotremata, and a quoll is a dasyuromorphia. (I trust the json.Unmarshal example on this.))
Takeaway: If a concept does not exist in Go, this does not mean you can't add it.
This is sometimes easier (as with the concept of undefined), and sometimes harder (see the Enum article in the featured articles section above), but this should not keep you from trying.
Overheard on Mastodon
Honestly, #GoLang is nice. It doesn't have nice features like algebraic data types and pattern matching, but that hardly matters. What matters is that the code is easy to read, and control flow is obvious, and the language does provide that.
– mohaneds
More articles, videos, talks
Test helpers might be an undervalued feature of the
testingpackage. Elton Minetto's demo may convince the reader to try out test helpers.
Strategy pattern in Go | Redowan's Reflections
He said, "pattern"! Run! (Only joking.)
No, calling C code from Go is not straightforward. Go is garbage-collected while C relies on manual memory management, and Go and C have different calling conventions, just to name two significant differences. For better mediation between these two worlds, the Go team has created
cgoas a bridge between the two languages. Eric Chiang shows you howcgolets you share Go data types like slices, strings, or pointer field with C code.
GoTH — I like the name! — GoTH is Go,
a-h/templ, and HTMX. Author: "(...) with this stack I found myself super productive, much more productive than I was in the JavaScript world." 'Nuff said.
Articles about Go interfaces are plenty. This one goes one step further and disucsses the two common pitfalls:
nilness of interfaces and the interference of interfaces with pointer and value receivers.
Neologism: Context Awareness | matttproud.com (blog)
Be aware of the context when you talk about context awareness! Especially in the context of
context.Context.
A list of OpenAPI tools for Go, wether you need to generate code from a spec, specs from code, or something entierly different.
How Kubernetes "time-slices" available CPU capacity to individual processes. If your Go binary gets only 25% of the CPU, you now know why.
Transactions in Go Hexagonal Architecture | by Khaled Karam | The Qonto Way | Medium
Hexagonal architecture is not too difficult to grasp, but the devil is in the design details.
Real programmers don't use
go run! Or do they? Chris Thompson, at least, is someone who lovesgo run. Here is why.
Gostmortem Table - Visual Studio Marketplace
Before I wrote this comment, I headed over to my VSCodium window and tried to install this extension, but unfortunately, it is not available on the Open VSX marketplace (yet).
Julio Merino argues that if you know C, then you already know three major programming topics well.
How to use Google's new open-weight model "Gemma" in Go. - By Eli Bendersky.
Projects
Libraries
"Polymorphic JSON" is another way of saying, "this data structure has a crappy design". No, scratch that. There may be valid reasons to create data structures that have no fixed structure but rather represent a grab bag of different types. And they exist anyway, so the best way of dealing with these is to gracefully map them to Go structs with as little friction as possible. Meet
polygo.
"Hashsplitting is a way of dividing a byte stream into pieces based on the stream’s content." I am afraid I cannot explain this better in a single sentence, but maybe it's enough to know that hashsplitting is used by various well-known applications like Rsync, Syncthing, Perkeep, or Kopia.
GitHub - wcharczuk/go-incr: A go implementation of Jane Street's "incremental".
If you have to implement some specific kind of computation where changes in the input set require only partial re-calculations of the result, have a look at this library. "Think Excel spreadsheets and formulas, but in code."
GitHub - orsinium-labs/wasm4go: Framework for making WASM-4 games with Go (and TinyGo).
Write games in Go that run in the web browser, as well as on low-powered microcontrollers or obsolete hardware.
Mapping a sphere (such as, the surface of this planet) to a flat surface is inherently problematic and never perfect, which is why so many projection algorithms exist. This package helps geographers to map the world.
GitHub - yc90s/xrpc: xrpc is a concise and lightweight RPC framework based on message queues
Remote procedure calls on top of NATS.
Tools and applications
A ToDo app that requires a wagonload of backend and infrastructure tooling? At first, my thoughts meandered between "huh, this is quite a bit over-engineered" and "is this the next Fizz Buzz Enterprise Edition?" But if you take this app as a showcase for backend architecture, it starts to make sense.
GitHub - wizsk/goshare: share files in local network with added perks.
Ad-hoc filesharing in the local network with password protection and a web UI.
GitHub - ahmedakef/goshell: a REPL shell for golang
A Go REPL (Read-Execute-Print Loop), similar to rango or Yaegi.
Never miss a NYC subway train anymore!
GitHub - scosman/zipstreamer: Zip File Streaming Microservice - stream zip files on the fly
Zip and download many files with a single request.
Completely unrelated to Go
If you want to quickly determine if a given object is definitely not part of a set but are ok if you get a few false positives, then you need a bloom filter. With Sam Rose's interactive tutorial, you can learn how bloom filters work.
This is the app you need on your iPhone! Never get lost again. Whenever you find yourself in the middle of nowhere, pull out your smartphone, start the app, and the pointer will tell you where the center of the galaxy is.
Now I am waiting impatiently for a version 2.0 that points to the center of the universe.
Fun fact: The author knew next to nothing about iPhone app development and asked ChatGPT for help. He describes the steps in the article.
New app! A compass that points to the centre of the galaxy (Interconnected)
Happy coding! ʕ◔ϖ◔ʔ
Questions or feedback? Drop me a line. I'd love to hear from you.
Best from Munich, Christoph
Not a subscriber yet?
If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding.
Find the subscription form at the end of this page.
How I can help
If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all):
On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff.
Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff.
My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go.
Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like.
Christoph Berger IT Products and Services
Dachauer Straße 29
Bergkirchen
Germany