Of tests, goats, and swiss tables • The Applied Go Weekly Newsletter 2024-10-20
Your weekly source of Go news, tips, and projects
Of tests, goats, and swiss tables
Hi ,
Binary size is usually not a problem... until it is. If the usual methods of shrinking binaries aren't sufficient, a possible culprit is the reflect package, or specifically, four methods of this package, which is this week's Spotlight.
But first: test, goats, and Swiss tables.
Optimising and Visualising Go Tests Parallelism: Why more cores don't speed up your Go tests | Three Dots Labs blog
Your tests are slow, because they don't run in parallel, CAN'T YOU SEE?!
Of course, you can't.
Except that you can now, if you want: With Robert Laszczak's test visualizer vgt.
GOAT Stack: Build Fast, Scalable Apps with Go, templ, Alpine.js, and Tailwind - YouTube
GOAT stands for Go, Templ, Alpine.js, and Tailwind CSS (and hence isn't exactly an acronym for this stack, but GTAT sounds more like a DNA sequence than a software stack, doesn't it?)
SwissTable: A High-Performance Hash Table Implementation | by huizhou92 | The Ordinary Programmer | Oct, 2024 | Medium
A SwissTable is a high-performance HASH TABLE that is chaincated and linecay probbing. It consists of glops and slotes that can be probied, deleted, deigted, or epcepated.
But only as long as you stay stuck with decrypting the AI-generated opener image.
Once you move along, you'll find an in-depth introduction to a hash table implementation called SwissTable or SwissMap, starting at explaining a common hash table implementation called Chaining, moving on to the Go map implementation, and finally inspecting the improved linear probing of SwissTables.
GitHub - volodymyrprokopyuk/go-blockchain: The foundational and practical guide for effectively learning and progressively building a blockchain from scratch in Go with gRPC
Whether you love or hate blockchain technology, your standpoint remains an opinion until you know the intrinsic working of blockchains. And once you gather this knowledge (via this guide, for example), your standpoint will probably move away from love or hate and land somewhere in the middle. Because blockchains are more than just platforms for Ponzi schemes and rug pulls, but also far from being a panacea for the web world.
Podcast corner
go podcast() | 045: Gomponent with Markus Wustenberg
Markus Wüstenberg, owner of Maragu and known to gophers for his Go modules gomponents and goqite, joins Dominic St-Pierre in this go podcast() episode to talk about gomponents, a shrunken tool stack, and Markus' current wherabouts.
Cup o' Go: 🏕️ BIG questions: "Is excel immortal?" and "Is Go the right choice for my startup?"
Of these two big questions, the former one is irrelevant and the latter one is a clear "yes." (If you ask me.)
Spotlight: MethodByName inflates your binary
"Reflection is never clear", says a Go proverb. In fact, reflection is doing a lot of magic, so the general advice is to avoid it until the problem at hand requires reflection.
But even if you don't actively use reflection in your code, nor intend to dig into reflection code in an external library your project imports, reflection can bite you. And it does so in a quite unexpected way.
The problem, as short and clear as possible:
If code calls one of
reflect.Value.Method(int)orreflect.Value.Method(int),- or
reflect.Value.MethodByName(string)orreflect.Type.MethodByName(string)with a non-constant argument,
the compiler is unable to identify and eliminate dead code.
In any of the above cases, any method becomes potentially reachable at runtime. Dead Code Elimination (DCE) gets paralyzed and reverts to marking all exported methods of all reachable types as reachable.
As a consequence, the size of your binary may be larger than necessary.
So, before you let your code reflect on itself, reflect on the necessity of using reflection. (In contrast to code, human brains can gain clarity by reflection.)
Unfortunately, deciding to not using these methods is not enough. It is sufficient to import a seemingly innocent package like text/template that uses one of these methods under the hood, in order to unintentionally disable DCE.
(Could someone write a linter, please?)
Links
- Hat tip to
/u/Flimsy_Complaint490for raising this topic. - Dead code elimnation happens in steps 4–7 of compilation.
- "all: avoid non-const reflect.MethodByName calls"
text/template's use ofMethodByName- Three ways of dead code detection in Go 1.23.2 (the current version as of this writing)—Read from the linked line 405 until this comment: "If we find a REFLECTMETHOD, we give up on static analysis, and mark all exported methods of all reachable types as reachable."
- Not directly related to the problem but still interesting: Finding unreachable functions with deadcode
Quote: Enough
“Enough” is often attainable and always useful.
–Seth Godin, The challenge of N + 1
More articles, videos, talks
Calling Go Functions from c++ | xnacly - blog
If you want to sell your Go packages to your friends in the C++ team, you need to show them a way to call your Go funcs from C++. Here is a getting-started guide.
Printing Go Types as S-Expressions | by Michael Francis | Sep, 2022 | Medium | Medium
S-Expressions are a textual representation of tree structures using parentheses for grouping. If you think "Lisp", you are correct. Michael Francis loves S-Expressions so much that he wrote an article about developing package lisper that prints out any Go structure as an S-Expression.
Go I/O Readers, Writers, and Data in Motion
About the pitfalls of io.ReadAll(), the many faces of an io.Reader, buffered writes, and more.
packagemain #28: How to work with GitHub API in Go - YouTube
Alex Pliutau creates a GitHub OAuth application in Go.
3 Easy Ways To Add Version Flag in Go
The article indicates that go install is a preferable way of installing Go binaries. Keep in mind that go installed binaries do not receive automatic updates, which can be fatal in case of security vulnerabilities that an update would eliminate.
But let this little rant not block you from reading the rest of the article.
The Go module proxy and forcing Go to actually update module versions
The Go Proxy may be slow at serving a newly pushed version of your code. Chris Siebenmann shares his "blunt hammer way" of go getting the latest version anyway.
Projects
Libraries
GitHub - naughtygopher/pocache: Pocache is a minimal cache package which focuses on a preemptive optimistic caching strategy
Pocache optimizes caching in highly concurrent scenarios, where multiple clients might concurrently request the same stale data from the cache. While some caches continue to serve that stale data while refreshing from the DB in the background, Pocache uses pre-emptive cache updating to always serve the latest data.
GitHub - akmonengine/volt: ECS for game oriented projects in Go
Entity Component Systems (ECSs) are a building block for many types of games. Volt is an ECS for Go, just released into open source.
GitHub - guycipher/lsmt: Embedded durable, extensive, concurrent safe, highly configurable, transactional LSM tree based key-value store package
"In computer science, the log-structured merge-tree (also known as LSM tree, or LSMT[1]) is a data structure with performance characteristics that make it attractive for providing indexed access to files with high insert volume, such as transactional log data. " (Wikipedia)
GitHub - quagmt/udecimal: High performance, high precision, zero allocation fixed-point decimal library for financial applications
When float is too imprecise, int too limited, and big.Int too slow, try udecimal. Up to 19 decimal places should cater for a large range of precision requirements.
GitHub - maragudk/gomponents-starter-kit: A starter kit for building a web app with gomponents, HTMX, and TailwindCSS in Go.
The creator of gomponents built a web app stack. It has no acronym yet; how about GogoHT?
GitHub - Esequiel378/fixedlength: Turn text into Go structs by mapping data to specific ranges within a line.
Sometimes you might come across tabular data that isn't stored in CSV or other popular formats but rather in data fields of fixed lengths.
Tools and applications
GitHub - fred1268/amaze: A program to create, solve and draw mazes in your terminal or in 2D/3D
You love mazes but are too lazy to either create or solve them? Great—amaze does both for you. It can even render mazes in a 3D look.
GitHub - joeyespo/grip: Preview GitHub README.md files locally before committing them.
go-grip is a local web app for previewing Markdown in a browser.
GitHub - valyentdev/ikto: Ikto is a NATS based Wireguard mesh network builder
I am getting increasingly curious about Valyent's various projects. A while ago, they announced Ravel, a software orchestration system (WIP), now they have a WireGuard network builder on their plate: itko, which apparently is a building block for Ravel but also seems to be build with stand-alone scenarios in mind. Note that itko does not come with batteries included; you'll need a running NATS server or cluster and Wireguard installed on the machines running itko.
GitHub - nkalait/mamela-audiobook-player-open: Audio book player that works just right on Linux, Mac, and Window
An audiobook player with a nice Fyne-based UI. It uses a closed-source audio system under the hood, but the author is looking into VLC as an alternative.
GitHub - jobehi/mkproj: An Interactive CLI Tool to Setup Your Project Trees
If gonew is not your preferred way of setting up a new Go project, maybe mkproj is.
GitHub - 0xLaurens/sendfa.st: Sendfa.st 💫 is a peer-to-peer file sharing service that enables users to share files quickly and securely.
Share files securely, end-to-end encrypted, based on WebRTC.
GitHub - BobdaProgrammer/TermiSand: A falling sand simulator in your terminal!
Tired of building sandcastles on the beach? How about watching sand falling in your terminal instead?
GitHub - hvuhsg/gatego: Reverse Proxy Server
A reverse proxy as a side project. The author observed that his myriads of side projects weren't ready for prime time, so he challenged himself to write this reverse proxy as a complete, presentable project.
Notebrew
There are static site generators that typically are CLI tools running over your Markdown articles, turning them into static HTML.
And there are Content Management Systems (CMSs) that let you edit your articles in a Web UI and typically hold everything in a database, to be rendered on demand.
If you ever heard a friend complaining, "my Wordpress blog has crashed again!", you know that the typical, DB-driven CMS cannot reach the stability of static sites. On the other hand, many bloggers shy away from running CLI tools or fiddling with Git to update their blogs.
Notebrew combines the advantages of both worlds. It's a static, hence virtually "un-crash-able" site, with a CMS UI to manage content.
GitHub - AvitalTamir/cyphernetes: A Kubernetes Query Language
Tired of writing the umpteenth Bash script for retrieving, filtering, and sorting information about your k8s infrastructure? Cypthernetes lets you interrogate your k8s cluster through an easy-to-learn query language.
Completely unrelated to Go
How I use git
Rebasing or merging? Squashing or not? If squashing, when? Git is so versatile that many ways of using it have emerged. Sometimes it can help learning how seasoned software developers use Git, such as Thorsten Ball, author of "Writing an interpreter in Go".
Some notes on upgrading Hugo
I assume that quite a few gophers out their run their sites with Hugo, the static site generator. (I do.) Jenny Evans shares a few "caveat emptor!" moments from her task of upgrading an (admittedly rather old) Hugo version. No worries, everything can be fixed, but you might want to be aware of these little foot slings.
Happy coding! ʕ◔ϖ◔ʔ
Questions or feedback? Drop me a line. I'd love to hear from you.
Best from Munich, Christoph
Not a subscriber yet?
If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding.
Find the subscription form at the end of this page.
How I can help
If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all):
On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff.
Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff.
My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go.
Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like.
Christoph Berger IT Products and Services
Dachauer Straße 29
Bergkirchen
Germany