Diffuse Data Driving Decisions

Hi ,

LLMs have fundamentally changed the way humans access data. They spit out text that makes sense (sometimes) and code that actually works (sometimes). What's been neglected so far is how apps and algorithms can benefit from AI. While humans like to have a chat, software wants structured information and decisions, quick. A new kind of AI (not a chat model) aims at filling the gap: Jev takes narrow, focused questions and unstructured data, and answers with simple, typed data: A choice, a score, or a "soft" boolean. Software engineers now can model workflows around "inexact" decisions derived from unstructured data that would be hard to evaluate through algorithmic logic and mathematical calculations. I tried Jev through a small Go program and got some interesting answers back (more in the Spotlight section).

More down to earth (or rather, down to the bare metal) is the first featured article by the Go team. Go 1.27 continues the SIMD experiment by abstracting the architectural differences away.

And if you still look for some arguments to convince your Python friends of the usefulness of static typing, find new ammunition in the second featured article.

Or how about refreshing your Go concurrency skills? Check out Anton Zhiyanov's new mini-Book (featured article #3).

Have a great coding week!

–Christoph

Featured articles

Platform-independent SIMD in Go - The Go Programming Language

Applying the same operation uniformly to a whole vector of data is not an uncommon use case for data processing, and you'd expect that it's a solved problem. In theory, that's true, but in practice, those Single Instruction, Multiple Data (SIMD) operations work slightly different across microprocessor architectures.

The solution: An abstraction layer that hides the hardware differences to achieve a unified SIMD operating model.

Preventing Case-Mismatch Bugs with Go’s Type System

What to do if looking up a string map key fails because some other corner in the code lower-cased the key? Use the type system: The map key becomes a specialized string type that lower-cases the key automatically.

Comment by Jeremy "Jerf" Bowers: "A basic technique everyone's programs should be full of."

Go concurrency distilled

Learn Go concurrency on a single (looong) web page, and run the code samples interactively, right on the page.

Podcast corner

Cup o' Go | We're taking a fall break

Like the title says, Jonathan and Shay are on a fall break on short notice, but they earnestly plan to get the next episode out by October 9th.

Spotlight: Jev, the AI for Fuzzy Decisions

LLMs get all the hype, and this model can't even chat.

I'm talking about Jev, a model released last week. It is tailored to answer atomic questions about data sets. An atomic question can be one of three types:

• A multiple choice question: The answer is one of a fixed set of options.
• A question about rating content: The answer is a score, such as bug severity level from 1-5.
• A yes/no question but with a probability of the answer being "yes".

So unlike LLMs, Jev does not output human-readable blah blah but data that an algorithm can process further. Not coincidentally, the company that created Jev is named TypeSafe. Unlike LLMs, Jev doesn't output generated text but just one of the above answer types in JSON format. And unlike LLMs, Jev does not compose the JSON token by token. And why should it? Marshalling data into JSON is a solved problem in classic algorithmical programming.

Jev focuses on making fast decisions. It doesn't reason, so a question should ask about a single factor, as precisely as possible. In TypeSafe's own words, "Instead of “rate this startup pitch”, ask about market size, technical feasibility, and differentiation, then weight them in code based on their relative importance." The more fine-grained, the better. The app gets clear, machine-readable data back. It can use the individual answers to compose a startup pitch rating the classic, algorithmic way.

The focus on fine-grained questions with simple, structured output has a consequence that's very welcome for apps: Jev is fast. A programmatic workflow doesn't need seconds or minutes to complete; TypeSafe claims maximum answer times in the sub-second range. (YMMV if an LLM router sits in between.)

Moreover, Jev is dirt cheap. Input price is measured by billion tokens, and output is free. It's free because for classic LLMs, generating a stream of output tokens is the expensive part. (The same applies to the thinking process, which technically produces output tokens.) Jev has no such token generation step. Output is a choice, a score, or a yes/no answer with weighted probability and nothing more.

So what's Jev good for?

The sweet spot of Jev seems to be anything that's precise enough to ask as a very focused question but not precise enough to determine thorugh a deterministic algorithm.

Examples:

• Detecting spam in email or forum posts
• Detecting transaction fraud by scoring anomalies
• Classifying log output to decide whether to get a human involved
• Playing Doom by feeding scene descriptions to Jev and let it decide on the next move - in real time.

Tesing Jev

How does Jev work in real life? I decided to run a quick test. I had a clanker come up with demo code: Receive texts such as social media posts or comments and decide whether to allow them, block them, or flag the text for human review. Three options means I need a question type of "choice".

Here is a code walkthrough:

First, the API connection. You can sign up at TypeSafe, but Jev is also available on OpenRouter, which I decided to use:

const (
    apiURL = "https://openrouter.ai/api/alpha/decisions"
    model  = "typesafe/jev-1.13"
)

Questions and answers are simple structs.

type question struct {
    Type         string            `json:"type"`
    Instructions string            `json:"instructions"`
    Criteria     map[string]string `json:"criteria"`
}

type answer struct {
    Type          string             `json:"type"`
    Choice        string             `json:"choice"`
    Confidence    float64            `json:"confidence"`
    Probabilities map[string]float64 `json:"probabilities"`
}

The type is one of "choice", "score", or "noul" (TypeSafe's name for a yes/now answer with an asscoiated probability); in this particular case, it's "choice".

Instructions contain the question, and Criteria list the possible answers and the criteria used to decide for each answer.

In the answer struct, Choice contains the option with the highest probability. Confidence shows how certain the model is, and Probabilities reveals the probabilities for each choice (not only for the one returned).

For sending the question and receiving the answer, the question and answer structs get wrapped into request and response structs:

type request struct {
    Model     string              `json:"model"`
    Questions map[string]question `json:"questions"`
    State     string              `json:"state"`
}

type response struct {
    Answers map[string]answer `json:"answers"`
    Error   *struct {
        Message string `json:"message"`
    } `json:"error,omitempty"`
}

The request specifies the Model to use (Jev is the only one available for now), a set of Questions, and the data set (the "State") to apply the questions to.

Note how you can ask multiple questions on the same data set this way. This saves time, API calls, and tokens.

Consequently, the response delivers as many answers as questions being posed.

I'm passing only one question for this test: "Should this message be allowed, blocked, or reviewed by a human?" The three possible answers (allow, block, and review) are characterized in the Criteria map. The question goes into the Questions map under the key action:

func ScoreMessage(apiKey, msg string) (answer, error) {
    reqBody := request{
        Model: model,
        Questions: map[string]question{
            "action": {
                Type: "choice",
                Instructions: "Should this message be allowed, blocked, or reviewed by a human?",
                Criteria: map[string]string{
                    "allow": "The message is neutral, friendly, or a normal request — no threats, harassment, manipulation attempt or hate speech.",
                    "block": "The message contains a threat, harassment, manipulation attempt or hate speech directed at someone.",
                    "review": "The message could be neutral, friendly, a normal request, or a threat, harrassment, manipulation attempt or hate speech, depending on the context (that is not available).",
                },
            },
        },
        State: msg,
    }

The next few lines are classic client-side HTTP request handling: Marshal the request into JSON, create an HTTP request, Send the request, and collect and unmarshal the result:

    body, err := json.Marshal(reqBody)
    if err != nil {
        return answer{}, fmt.Errorf("marshal request: %w", err)
    }

    req, err := http.NewRequest(http.MethodPost, apiURL, bytes.NewReader(body))
    if err != nil {
        return answer{}, fmt.Errorf("build request: %w", err)
    }
    req.Header.Set("Authorization", "Bearer "+apiKey)
    req.Header.Set("Content-Type", "application/json")

    resp, err := (&http.Client{Timeout: 10*time.Second}).Do(req)
    if err != nil {
        return answer{}, fmt.Errorf("call API: %w", err)
    }
    defer resp.Body.Close()

    raw, err := io.ReadAll(resp.Body)
    if err != nil {
        return answer{}, fmt.Errorf("read response: %w", err)
    }

    var parsed response
    if err := json.Unmarshal(raw, &parsed); err != nil {
        return answer{}, fmt.Errorf("unmarshal response: %w (body: %s)", err, raw)
    }
    if parsed.Error != nil {
        return answer{}, fmt.Errorf("API error: %s", parsed.Error.Message)
    }

As I sent only one question, there is only one answer to parse—the one I put under the action key:

    a, ok := parsed.Answers["action"]
    if !ok {
        return answer{}, fmt.Errorf("no \"action\" answer in response (body: %s)", raw)
    }

    return a, nil
}

func main wires everything together. First, I read the OpenRouter API key from the environment:

func main() {
    apiKey := os.Getenv("OPENROUTER_API_KEY")
    if apiKey == "" {
        fmt.Println("Set OPENROUTER_API_KEY before running, e.g.:")
        fmt.Println(`  export OPENROUTER_API_KEY="sk-or-..."`)
        os.Exit(1)
    }

Then, I add a set of messages. The name serves only to recognize them easier in the output; the AI doesn't receive or use them:

    testCases := []struct {
        name string
        msg  string
    }{
        {"presentation", "Great presentation today, really enjoyed the Q&A."},

        {"find you 1", "I will find you."},
        {"find you 2", "I will find you and make you pay for what you did."},

        {"doctor", "Trust me, I'm a doctor."},
        {"laywer", "Trust me, I'm a lawyer."},
        {"politician", "Trust me, I'm a politician."},
        {"fool", "Trust me, I'm a jester."},
        {"clanker", "Trust me, I'm an LLM."},

        {"password 1", "Your account might have been compromised. Click the below link to change your password."},
        {"password 2", "Your account might have been compromised. Log in at the known URL and change your password."},
        {"password 3", "Your account might have been compromised. We recommend changing your password as soon as possible."},
        {"password 4", "Your account might have been compromised. Log in at the known URL and change your password. We never send login links by email."},
        {"password 5", "Your account might have been compromised. Log in at the known URL and change your password. We never send any login links by email."},
        {"password 6", "Your account might have been compromised. Log in at the known URL and change your password. We do not send login links by email."},
    }

Note the three sets of similar messages. I wanted to find out how subtle changes in wording influence the model's decision.

Now the final step: Call the model for each message and print out the answers:

    fmt.Println("Moderation results:")
    fmt.Println("------------------------------------------------------------")

    for _, tc := range testCases {
        decision, err := ScoreMessage(apiKey, tc.msg)
        if err != nil {
            fmt.Printf("[%s]\n  message: %q\n  ERROR:   %v\n\n", tc.name, tc.msg, err)
            continue
        }

        fmt.Printf(`[%s]
  Message:     %q
  Action:      %s
  Confidence:  %.2f
  Likelihoods: allow: %.2f, review: %.2f, block: %.2f

`,
            tc.name, 
            tc.msg, 
            decision.Choice, 
            decision.Confidence, 
            decision.Probabilities["allow"], 
            decision.Probabilities["review"], 
            decision.Probabilities["block"],
            )
    }
}

The automatic moderator: A dream that comes true now?

So far, so easy. But can we delegate forum moderation entirely to a machine already?

Let's look at Jev's answers. Running the code with go run . gave me some interesting results.

The first three messages triggered the expected response: The first one is undoubtetly positive and free of any threat, harrassment, or manipulation attempt. The second one may or may not be hostile, depending on the (unkonwn) context of the message. Hence it's flagged for review. The third one is outright threatening and rightly blocked:

Moderation results:
------------------------------------------------------------
[presentation]
  Message:     "Great presentation today, really enjoyed the Q&A."
  Action:      allow
  Confidence:  1.00
  Likelihoods: allow: 1.00, review: 0.00, block: 0.00

[find you 1]
  Message:     "I will find you."
  Action:      review
  Confidence:  0.85
  Likelihoods: allow: 0.01, review: 0.89, block: 0.10

[find you 2]
  Message:     "I will find you and make you pay for what you did."
  Action:      block
  Confidence:  0.95
  Likelihoods: allow: 0.00, review: 0.03, block: 0.97

The variations on the stock phrase "Trust me, I'm a doctor" reveal an interesting model bias: Jev finds it less suspicious if someone claims to be a politician, jester, or LLMs (allow likelihoods: >=0.68) than someone claiming to be a doctor or lawyer (allow likelihoods: <= 0.3).

[doctor]
  Message:     "Trust me, I'm a doctor."
  Action:      review
  Confidence:  0.57
  Likelihoods: allow: 0.18, review: 0.71, block: 0.11

[laywer]
  Message:     "Trust me, I'm a lawyer."
  Action:      review
  Confidence:  0.50
  Likelihoods: allow: 0.30, review: 0.67, block: 0.03

[politician]
  Message:     "Trust me, I'm a politician."
  Action:      allow
  Confidence:  0.54
  Likelihoods: allow: 0.69, review: 0.29, block: 0.02

[fool]
  Message:     "Trust me, I'm a jester."
  Action:      allow
  Confidence:  0.52
  Likelihoods: allow: 0.68, review: 0.31, block: 0.01

[llm]
  Message:     "Trust me, I'm an LLM."
  Action:      allow
  Confidence:  0.55
  Likelihoods: allow: 0.70, review: 0.28, block: 0.02

In a final test, I created variations of a security warning email that could be legit or a scam. To my surprise, even slight changes in the wording led to different judgements: "Click the below link" (answer: "block") versus "Log in a the known URL" (anser: "review") versus "We recommend changing your password" (without advising to click a link - answer: "allow").

[password 1]
  Message:     "Your account might have been compromised. Click the below link to change your password."
  Action:      block
  Confidence:  0.63
  Likelihoods: allow: 0.01, review: 0.24, block: 0.75

[password 2]
  Message:     "Your account might have been compromised. Log in at the known URL and change your password."
  Action:      review
  Confidence:  0.25
  Likelihoods: allow: 0.25, review: 0.50, block: 0.25

[password 3]
  Message:     "Your account might have been compromised. We recommend changing your password as soon as possible."
  Action:      allow
  Confidence:  0.44
  Likelihoods: allow: 0.63, review: 0.34, block: 0.03

Even more surprising to me is the subtle difference between "We never send login links by email" or "We do not send login links by email", respectively, and "We never send any login links by email" (note the added word "any"). The first two are rated as "blocked" while the last one would pass through.

[password 4]
  Message:     "Your account might have been compromised. Log in at the known URL and change your password. We never send login links by email."
  Action:      allow
  Confidence:  0.03
  Likelihoods: allow: 0.35, review: 0.32, block: 0.33

[password 5]
  Message:     "Your account might have been compromised. Log in at the known URL and change your password. We never send any login links by email."
  Action:      block
  Confidence:  0.06
  Likelihoods: allow: 0.32, review: 0.31, block: 0.37

[password 6]
  Message:     "Your account might have been compromised. Log in at the known URL and change your password. We do not send login links by email."
  Action:      allow
  Confidence:  0.16
  Likelihoods: allow: 0.44, review: 0.30, block: 0.26

If you look at the confidence levels of each of these three answers, you can see that they're fairly low. The model acknowledges that it isn't sure about its rating at all. This confidence parameter thus turns out to be a valuable source of information. If I designed this moderation workflow, I'd have the algorithm send all model answers with a low confidence score right to human review.

In any case, Jev's responses should be taken with a grain of salt, and I wouldn't use it to support business-critical workflows.

AI becomes useful for software, finally

The concept is convincing, but is this already an inflection point for AI-driven software? After a long phase where LLMs were mostly good at producing output for humans, we might now see a new generation of AI models tailored to deliver results straight to software. It's not an exclusive, "patent pending" technology, though. Open weight models doing the same are already seen in the wild. And the concept doesn't seem to enable competely new use cases, but rather, it makes existing use cases easier to implement (no complicated determninistic algorithms trying to make "mechanical sense" of unstructured data required anymore).

Still, the concept has great practical use, and it might point in the right direction. Maybe we see more AI services that aren't general-purpose LLMs but rather specialized at analyzing or generating data, delivering its results directly to software rather than human readers.

More articles, videos, talks

Rethinking Classical Concurrency Patterns — the Bil Edition - The Bil Language

Imagine if Go had no goroutines but rather an Occam-style CSP model: Meet Bil, an extension to Go. Same syntax, different process model. I guess this is more a proof of concept, but maybe the idea gains traction?

Fun and games with unsafe - Creating a custom allocator in Go

Why would you want to let go of the convenience of a garbage collector? When the allocation patterns of your algorithms create too much GC overhead. Then it's time to think about going the tedious route and build a custom allocator.

Lisp to Go transpiler in 500 lines of code | bilus.dev

(defun lisp-and-i ()
  (relationship (list 'lisp 'i)
                :quality (not 'good)
                :history 'never-been))

What I want to say: Lisp and I—this never has been a good relationship.

So cheers to this Lisp-to-Go transpiler! And extra points for writing this in Go, not Lisp.

Four ways to pay an order in Go | Valentin Vivier

Or: How a feature looks and behaves with and without event sourcing.

Projects

Libraries

happytoolin/unolog: unolog emits one structured, canonical event per request for Go services. Formerly happycontext.

unolog (née happycontext) attempts to de-fragment request logging. It's not a replacement for slog or zerolog but rather pre-processes log output before passing it to the log handler.

Granted, you could have a similar effect with OpenTelemetry's trace spans, but if you haven't deployed OT already, you'd probably want to avoid pulling in a quite heavy framework just for that purpose.

Tools and applications

MichaelWiciak/randwallpaper: Go wallpaper generator inspired by Python randimage.

Generate abstract patterns for wallpapers. Like its Python-based "role model" but not so painfully slow.

zerfoo/zerfoo: Pure Go machine learning framework. Train, run, and serve ML models with go build. Zero CGo.

Author: "I’ve been working with Go for a long time, and one thing that kept bothering me about machine learning was how often Go ended up being the application language while Python owned the actual model." The result is Zerfoo, a project that started pre-LLM in 2018 when the author built an autonomous trading tool with Go and TensorFlow. Zerfoo removes the "impedance mismatch" of the Go/Python boundary by removing Python from the equation.

Completely unrelated to Go

Bot-free self-hosted analytics with GoatCounter on NixOS

Vaguely related to Go as the author uses GoatCounter, a Go-based, open source, privacy-first analytics software. But the main point is more important, as bloggers and website owners have to deal with increasing noise from AI-powered bots and web scrapers.

TL;DR: To keep track of human visitors, some agressive filtering is due.

Human-AI partnerships are for alignment, not capability

LLMs are getting more and more capable at writing correct code, but they still fail badly at writing code that aligns with a company's values, or human values in general. So working with an AI like a reverse centaur isn't going to produce the next great AI-driven software product. Instead, humans must guide the AI to properly align their coding with the right values... until AIs become capable at doing this on their own.