Running in Circles • The Applied Go Weekly Newsletter 2024-09-15
Your weekly source of Go news, tips, and projects
When I wrote this week's spotlight, these lines from Tears For Fears' song "Mad World" came to my mind:
I find it hard to tell you
'Cause I find it hard to take
When people run in circles
It's a very, very
Mad world
The spotlight's topic is much more trivial than this classic weltschmerz song: Go dependencies should not run in circles, or bad things happen. No worries, I'll show you how to fix them.
But first, ...
...featured articles!
Building LLM-powered applications in Go
Retrieval Augmented Generation (RAG) lets you feed an LLM with knowledge from custom resources, from local text files to code repositories. The LLM then can use this new knowledge to answer questions.
In this Go blog post, Eli Bendersky presents a simple RAG server concept and discusses three variants of implementing such a server in Go.
Ten Thousand Pixels
/r/place for your pocket.
A Go + web sockets experiment. Everyone can change pixels, and everyone can see changed pixels in real time.
Share your feedback about developing with Go - The Go Programming Language
The Go Developer Survey 2024-h2 is open for input! Share your experience with Go, it costs only a few minutes to fill out the survey.
Go 1.23.1 is released
An update to Go 1.23 has been released, with security fixes for encoding/gob, go/build/constraint, and go/parser, and a few bug fixes.
Podcast corner
go podcast(): Adrian Hesketh and Joe Davidson on Templ
Adrian Hesketh and Joe Davidson, the creator and a maintainer, respectively, of the Templ module, join Dominic St-Pierre in his latest episode.
Cup o' Go: And now for something completely different
The weekly mix of news about proposals, events, packages, and other Go topics with Jonathan and Shay.
No link to any episode of Monty Python's Flying Circus, though.
Go Time: Home automation with Go
If you don't use the usual home automation solutions like Home Assistant, openHAB, or NodeRed, you'll likely have to dive into microcontroller programming. Ricardo Gerardi and Mike Riley decided to do this with Go.
Spotlight: How to break up circular dependencies
Import cycles can be convenient but their cost can be catastrophic.
– Rob Pike
What are circular package dependencies?
Go disallows circular import dependencies by design. So if package A imports package B, then package B cannot import package A. And if package A imports package B and package B imports package C, then package C cannot import package A or B.
In short, package dependencies must form a directed acyclic graph (DAG).
< announcement >
Yes, I know, A, B, C, blah blah. Here is a tangible example. Consider an e-commerce application with product, inventory, and order packages. A circular dependency can easily happen:
package product
import "example.com/project/inventory"
func (p Product) InStock() bool {
return inventory.HasProduct(p.ID)
}
package inventory
import "example.com/project/order"
type InventoryItem struct {
ID int
Quantity int
}
func (in Inventory) IsAvailable(productID int) bool {
return in.count(productID) - order.CheckPendingOrders(productID) > 0
}
package order
import "example.com/project/product"
func (o Order) Total() (total int) {
for _, p := range o.Products {
total += p.Price()
}
return total
}
Why would Go disallow such import circles?
Because there are considerable advantages of non-circular dependency graphs:
- Fast compilation: The compiler can walk the dependency graph faster, and skip unmodified packages easier. Seriously, who isn't delighted by Go's compilation speed?
- A sane application architecture: Circular dependencies often indicate poor design or overly coupled components. By disallowing them, Go encourages developers to create cleaner, more modular architectures with clear separation of concerns. This leads to more maintainable and understandable codebases.
- Simplified dependency management: Without cycles, it's easier to reason about package relationships, update dependencies, and manage versioning. This simplifies both development and long-term maintenance of projects.
How to remove import cycles
Once you unintentionally have created a circular dependency, you must get rid of it. The compiler will not engage in any discussion about this.
But... how? By checking for these aspects:
1. Missing separation of concerns
Ask yourself, are all concerns correctly separated?
Every entity in an application architecture should care about a very specific set of concerns and ignore others.
Look at the product code: Should a product know if it exists in some inventory? If a product could speak, and you ask it about the inventories that list it, the product would answer, "that's none of my business."
Move the InStock() function over to the inventory package and see the dependency cirle dissolve into little, puffy clouds:
package inventory
func (in Inventory) InStock(productID int) bool {
for _, item := range in.items {
if item.productID == productID {
return true
}
}
return false
}
2. (Too) close relationships
If two entities in different packages are very closely related to each other so that separation of concerns does not seem to make sense, consider lumping them together into a single package.
The e-commerce example is perhaps not the best example, but you might decide packing all inventory and product functionality into a single package, say, goods. Then you have only one dependency left, from order to goods.
3. Dependencies across application layers
Applications are often designed as layers, such as domain entities, business logic, interfaces, and external entities (storage, network, etc.). Layers should depend upon other layers in one direction only: from the bottom (the most concrete) layer to the top (the most abstract) layer.
Consider this mutual dependency between inventory (abstract) and inventory_storage:
package inventory
import "example.com/project/inventory_storage"
type Inventory struct {...}
type Item struct {...}
func (in Inventory) StoreItem(id int) {
inventory_storage.Store(in.item[id])
}
package inventory_storage
import "example.com/project/inventory"
func Store(item inventory.Item) {
// store the item
}
Here package inventory calls inventory_storage.Store(), passing down an inventory.Item. Now the two packages are tightly coupled in a mutual dependency.
The easy, and preferred, remedy for this mutual dependency is dependency injection.
Preferred, because it not only removes the mutual dependency but also removes knowledge from inventory (living in an abstract layer) about inventory_storage (living in a concrete layer).
Easy, because dependency injection is ridiculously easy in Go:
Just let an interface represent the inventory storage:
package inventory
type storage interface {
Store(Item)
}
type Inventory struct {
...
s storage // interface, will hold a real storage
}
func (in Inventory) StoreItem(id int) {
in.s.Store(item)
}
Field s of type Inventory is an interface with a Store() function. Now you only need to inject a real storage object in package main, using a suitable Inventory constructor:
package inventory
type Inventory...
func New(st storage) *Inventory {
return &Inventory {
s: st // now we have a real storage in s
}
}
package main
import "example.com/project/inventory"
import "example.com/project/inventory_storage"
func main() {
...
// Do the wiring:
is := inventory_storage.New(...)
in := inventory.New(is) // pass a storage object
...
in.Store(42)
...
}
Thanks to the interface abstraction, inventory does not depend on inventory_storage. Even the inventory constructor New() does not know about inventory_storage. It only expects an interface.
In package main, we can finally wire up a new inventory object with a concrete storage object. Done!
(Read more about easy dependency injection here.)
Keep your dependency graph clean
Now you know how to fix circular dependencies, all that's left is to prevent new circular dependencies from sneaking in.
A few hints in this regard:
- Separate concerns. Keep packages small and focused on a single concern. For example, don't let products care about inventories.
- Use interfaces to abstract dependencies away. For example, instead of reading from a file, read from an
io.Readerand wire up the input file in packagemain. - Use layered architectures that support only one direction of dependencies across layers.
In coding and in real life, don't go in circles.
Quote of the Week: Antiseptic to complexity
I thought — particularly capital-p — patterns made a programmer a good programmer. I embraced rote extraneous complexity, only to rapidly shed it after being doused in the antiseptic-to-complexity ecosystem that Go is in.
More articles, videos, talks
Go sync.WaitGroup and The Alignment Problem
Or: The forgotten art of counting goroutines.
Go Error Handling Techniques: Exploring Sentinel Errors, Custom Types, and Client-Facing Errors | Arash Taher Blog
Different error types and how to use them. An overview.
Frontend Development with Go Templates and htmx
Templ.guide may get all the hype these days, but why not start a web frontend project with Go's native html/template package?
Zero Trust SSH Client Explained
Build a zero-trust SSH client with OpenZiti.
Statically and Dynamically Linked Go Binaries | by Alex Pliutau | Sep, 2024 | ITNEXT
Alex Pliutau answers the questions you might have around static vs. dynamic linking.
How Go Tests "go test" | Atlas | Manage your database schema as code
go test tests Go code, but how is `go test´ being tested? It all started with a Bash script...
How to Pixelate Images in Go | Golang Project Structure
Combine this with face recognition for anonymizing people in photographs.
Pick A Park
Another great solo project: Don't know what to do over the weekend? Pick a park in your vicinity.
Author: "A couple of months ago, this community was very helpful to me when I ran into a memory limit on fly.io! I've recently come back to this project to complete it and deploy it: pick-a-park.com"
See also Pick A Park.
What has been the most surprising or unexpected behavior you've encountered while using the Go programming language?
"Error on retrieving null values from a db is quite like the Spanish Inquisition, nobody expects it first time it happens."
The GoLand 2024.3 Early Access Program Has Started!
VSCode or GoLand? The latter is a commercial IDE, but every EAP release can be tested for free for 30 days from build date. A great chance to compare the two most popular Go IDEs.
Projects
Libraries
GitHub - Sina-Ghaderi/vpngui: A VPN Client GUI in native Go and Win32 API's for Windows (without network stack)
This is a VPN client GUI for Windows.
For which VPN? For no VPN and any VPN!
It's just a GUI. Plug your own network stack underneath.
GitHub - tech-engine/goscrapy: GoScrapy: Harnessing Go's power for blazingly fast web scraping, inspired by Python's Scrapy framework.
Right now, this project is a proof of concept. It shall provide users a basis for writing spiders in Go.
GitHub - nanwp/nansql: nansql is a Golang library designed for managing connections to a database using the sqlx package. It provides a simple and efficient way to handle database connections, execute queries, and manage transactions.
The GitHub tagline says it all.
GitHub - catgoose/templ-goto-definition: Fixes golang templ goto definition to open .templ file instead of generated .go file
This plugin makes "Go to definition" on Neovim open .templ files instead of the generated templ.go files.
Tools and applications
GitHub - leg100/pug: Drive terraform at terminal velocity.
Make Terraform/OpenTofu/Terragrunt operations more convenient with this TUI made with Bubbletea and Lipgloss.
GitHub - aodr3w/keiji: task scheduler
A multi-process task scheduling system.
GitHub - bayraktugrul/modview: Effortlessly visualize mod graph with all external dependencies for your Go projects
Too many dependencies, and no way to cut them down? Modview keeps a bird's-eye view on your project.
GitHub - ksckaan1/hexago: A CLI tool for initializing and managing hexagonal Go projects.
Hexagonal Architecture is a flavor of layered software architecture concepts. hexago sets up a Go project based on hexagonal layout considerations.
Completely unrelated to Go
Going Buildless
Is buildless the new serverless?
Shell redirection syntax soup
A refresher on shell redirection.
I fixed the strawberry problem because OpenAI couldn't
LLMs frequently fail at answering this seemingly simple question: "How many r's are in the word strawberry?" But why? Can we possibly teach them how to count letters?
Xe Iaso explains why LLMs are unable to count letters and presents a surprising (and surprisingly simple) solution.
Happy coding! ʕ◔ϖ◔ʔ
Questions or feedback? Drop me a line. I'd love to hear from you.
Best from Munich, Christoph
Not a subscriber yet?
If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding.
Find the subscription form at the end of this page.
How I can help
If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all):
On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff.
Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff.
My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go.
Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like.
Christoph Berger IT Products and Services
Dachauer Straße 29
Bergkirchen
Germany