Can you make that faster? • The Applied Go Weekly Newsletter 2024-03-17
Your weekly source of Go news, tips, and projects
Recently, I used some AI chatbot again for quickly generating some code I was too lazy writing myself. Then, I came across an ambitioned company that works on building an AI software engineer. No, I don't want to spawn another will-AI-replace-our-jobs discussion. Indeed, I am skeptical about whether current AIs would be able to optimize network throughput at the same level as Andree Toonk did by carefully hand-optimizing the code. Or whether AIs would come up with separating date from time to fix certain time calculation issues as Danny Hermes describes.
Still, AI used well can help with software development in various ways. Do you use AI in your development workflow?
High-Speed Packet Transmission in Go: From net.Dial to AF_XDP
The average Web server doesn't need to handle crazy throughput. But if your app's networking turns out to be a performance bottleneck, try the optimizations in this article. The author, Andree Toonk, used them to achieve 186 GB/s, 18.8 Mpps with only eight cores.
High-Speed Packet Transmission in Go: From net.Dial to AF_XDP
The missing type in the Go standard library: Date! | Hardfin Engineering
Calculating dates and times is difficult and confusing. Especially when some information needs to be accurate to the date only (like, "clean up old records on the 28th") but the related calculations take the timestamp into consideration. Danny Hermes presents a package that helps to keep time and datetime values separated.
The missing type in the Go standard library: Date! | Hardfin Engineering
More powerful Go execution traces
For a long time Go's tracing feature was slow and kinda unwieldy. But it matured over the years. Michael Knyszek summarizes the progress made.
More powerful Go execution traces
What even is “Dependency Injection”? (a practical example using Go) | by LittleLeverages | Feb, 2024 | Medium
Dependency Injection in Go is trivial. Don't let the DI container camp tell you otherwise. LittleLeverages wrote a brief and clear reminder on injecting dependencies the Go way.
Podcast corner
Cup o' Go
This episode's interview: quobix, founder of Princess Beef Heavy Industries (pb33f) and Distinguished Engineer at Splunk, talks about pb33f's latest product, WireTap.
Cup o' Go | What makes a first-class Go port? Plus 👸 OpenAPI tools fit for a princess, with quobix
Go Time
Everybody loves greenfield development. Start from a blank slate and let your creativity flow.
Reality is no bowl of cherries, though. Chances are that you will be thrown onto some legacy project with structures that grew organically over the years. The Go Time panel (Ian, Johnny, Kris, and Jon) discuss how to survive in this situation.
Jumping into an existing codebase (Go Time #307)
go podcast()
Shim | ʃɪm | — a washer or thin strip of material used to align parts, make them fit, or reduce wear.
SHIM (Shove Hardware In the Middle) — Name for a piece of hardware that is placed between a Raspberry PI board and an extension board.
If a provider of an API sends a termination message on short notice, you're up for a possibly large rewrite on a tight timeframe. Other providers' APIs are typically totally different, and you'd wish that one of these providers would offer an API shim that maps the competitor's soon-out-of-order API calls to their own API.
go podcast() | 031: Using shim on API to prevent breaking changes
Go tip unpopular opinion: Go time format strings are not so bad after all
Go is certainly a biased language, and that's a good thing. Languages that try to be everybody's darling usually end up as feature behemoths with a bloated spec, slow compilers, and projects that are basically write-only.
One of Go's peculiarities has been (and continues to be) criticized as being completely unclear, which is quite the opposite of what Go generally aims to achieve (and, for most part, succeeded in achieving): clarity. Go is a powerful language, yet its syntax and semantics are clear and beginner-friendly. It is one of the few languages that enables juniors and seniors work side-by-side on a project without feeling like an impenetrable jungle to the former nor like a unusable toy languages to the latter.
Where was I?
Uh, yes: Go's time formatting. It's quite peculiar because the ages-old UNIX canon of time format strings goes like this:
YYYY-MM-DD hh:mm:ss
Here, the elements of a time string are represented by letters, and the combination of letters: YYYY stands for a four-digit year like 2024, YY for the two-digit version "24". "MM" is the two-digit month, "MMM" is the short month name like "Mar", "mm" is the minute in two digits, and so on. (The actual interpretation may vary between different datetime libraries, but you get the point.)
Go is different.
A time format string in Go may look like
"2006-01-02T15:04:05 -07:00:00"
or
"Mon Jan 2 15:04:05 MST 2006"
This does not look too bad for a format string, but the aspect that is being criticized so often is that the actual numbers, the values in these format strings are crucial for making the format string work.
From time package - time - Go Packages:
// We stress that one must show how the reference time is formatted,
// not a time of the user's choosing. Thus each layout string is a
// representation of the time stamp,
// Jan 2 15:04:05 2006 MST
// An easy way to remember this value is that it holds, when presented
// in this order, the values (lined up with the elements above):
// 1 2 3 4 5 6 -7
To clarify, to build a valid time string, you must use "Jan" or "01" to represent the month, "2006" or "06" to represent the year, "03" or "15" to represent hours in 12h or 24h format, respectively, and so on. "Wed Dec 7 05:32:17 MST 2024" is not a valid time format string.
To make this more confusing, the standard representation is the US-American format - "month, day, time, year, time zone". Other parts of the world would prefer "day, month, year, time, time zone". An unambiguous format that is understood worldwide would be "year, month, day, time, time zone".
But here is the unpopular opinion: All the criticism revolves around constructing a Go time format string. You have to keep remembering (or looking up) what value represents which part of a date/time value. On the other hand, reading a Go time format string is easy and quite straightforward:
"Mon Jan 02 15:04:05 -0700 2006"
"Monday, 02-Jan-06 15:04:05 MST"
"2006-01-02 15:04:05"
Just from looking at these strings, it is intuitively understandable how the resulting time string looks like, innit?
Overheard on Lenny's Podcast
It is a lot easier to deliver output than it is to deliver outcomes.
More articles, videos, talks
Go Test Doubles by Example | Amin Rashidbeigi
Mocks, Stubs, Dummies, Fakes,... — Amin Rashidbeigi explains all sorts of test doubles.
Writing a Postgres Logical Replication System in Golang
No, Postgres has no unlogical replication mode, even though the title might implicate that.
Presenting Scanme: Deep Dive into Network Scanning with Golang: Building a Port Scanner -
If you want to learn how
nmapworks, build your own port scanner tool. Alessandro Bresciani did this and wrote an overview about the aspects of port scanning and his tool Scanme.
How come I haven't heard of Risor before? I have to give it a try, the syntax looks nice, and it claims to be the fastest Go scripting language, at least in June 2023. Rizor compiles to a VM, rather than being interpreted, and this article shares some background info about the VM's opcodes.
Perform a "change signature" refactoring with gopls - YouTube
gopls, the Go language server, has a new feature called "change signature".
Exploring Go's Functional Iterators (Range-over Functions)
Did you know that the experimental range-over-function iterator feature can be used to manage cleaning up resources? Perfect Makanju explores this and other applications in this article.
Testing Golang Code with Toxiproxy
Toxiproxy is the chaos monkey for TCP connections. The Dolt team uses it in a few places, and Jason Fulghum shares some basics and tips around using this intentionally unreliable proxy server for resilience testing.
My Pi Day Journey with Go 64-bit alignment
Geoff Berl observed a failure on his Raspberry PI, exactly on pi day (3/14). Coincidence?!?
Projects
Libraries
GitHub - JJBordy/rules: Minimal rules engine, uses yaml as input
If you need something simpler than Grules, try
rules.
sugawarayuuta/refloat: Float parser that sacrifices nothing.
Storing decimal floating-point numbers in binary format causes quite a few conversion problems. The
refloatpackage implements a floating-point parser that "sacrifices nothing".
GitHub - neilotoole/streamcache: Golang in-memory caching stream reader
This in-memory stream caching library is "useful for scenarios where multiple readers may wish to sample the start of a stream, but only one reader will read the entire stream."
GitHub - knights-analytics/hugot: Huggingface transformer pipelines in Golang
Hugot helps to tightly couple Huggingface transformer models with Go applications. Use models in Go that were trained with Python. Run feature extraction or text or token classification on your own hardware. No need for running and maintaining a Python stack along with your Go app.
GitHub - reugn/equalizer: A set of performant rate limiters for Go
Rate limiting in three different ways (1. success or failure of previous requests, 2. time window, 3. token bucket).
GitHub - neilotoole/fifomu: Mutex with FIFO lock acquisition
fifomuis a mutex that returns a lock to callers in first-in-first-out mode. This package is used instreamcache, which is also listed in this issue. Neil O'Toole points out that if you needfifomu, you might have a code smell.
GitHub - jtarchie/sqlitezstd: Go sqlite VFS for using a zstd seekable compressed file.
TIL: SQLite files can be ZSTD-compressed. Also TIL: if the compression mode is "seekable ZSTD", you can read this SQLite file as a virtual file system with this Go package.
Tools and applications
GitHub - 0x3alex/fm: A very basic tree based cli file manager for linux
Like
treebut interactive. Built withtviewandtcell, two of the classic Go TUI packages.
GitHub - icholy/nsplinks.nvim: LSP textDocument/documentLink support for neovim
NeoVim users may find this tool useful: Place the cursor on a path (such as an import path) and type
gxto open that path (such as the imported package at pkg.go.dev).
Service discovery, load balancing, SSL termination, automatic scaling, distributed logging, messaging, monitoring, clustering, revision control in a single binary. Less configuration and management headache. (Alpha status.)
GitHub - Semior001/groxy: simple gRPC mocking server
Need to quickly fire up a gRPC mock server without much configuration headache? Consider
groxy.
Go + Gorilla Websockets + Vue.js = a stylish chat app.
What if algorithms were like Cricket games?
This project aims at performing inference on the LLaMa 2 7B-chat model completely outside of the Python ecosystem. Sounds like re-inventing the wheel, but the project's real goal is to learn about the various LLaMa layers.
Completely unrelated to Go
Should you be proud of having built that enormous, complex, sophisticated project (and survived)? How about being proud of being able to keep things simple?
Simplicity is not an end in itself. Simplifying software has a number of tangible benefits.
Simplicity, Please - A Manifesto for Software Development - InfoQ
Containerizing a complete development environment is a possible way of creating replicable and shareable environments. However, developing inside a container often feels like working on someone else's machine — it's just not the same as a local shell. Flox is a dev environment manager that aims at keeping the "local development feeling" while enabling the developer to replicate the environment on another machine or share it with colleagues. (In case you wonder, Flox is based on Nix.)
Flox | Your dev environment, everywhere
Yes, it's "awful", not "a full".
Happy belated π day.
Achieving awful compression with digits of pi | nicole@web
Happy coding! ʕ◔ϖ◔ʔ
Questions or feedback? Drop me a line. I'd love to hear from you.
Best from Munich, Christoph
Not a subscriber yet?
If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding.
Find the subscription form at the end of this page.
How I can help
If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all):
On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff.
Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff.
My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go.
Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like.
Christoph Berger IT Products and Services
Dachauer Straße 29
Bergkirchen
Germany