Let's Ask An LLM • The Applied Go Weekly Newsletter 2025-07-27

Your weekly source of Go news, tips, and projects

                July 27, 2025

            Let's Ask An LLM • The Applied Go Weekly Newsletter 2025-07-27

Your weekly source of Go news, tips, and projects

Let's ask an LLM
Hi ,
This newsletter is in 
🌴 Summer Break Mode 🌴
until August 24th, which means reduced content, and I decided to take the opportunity to explore the various ways in which a Go app can interact with AI. 
In this issue, I start by having Go consult an LLM and get a response back suitable for further processing (as opposed to simply forwarding to the user). 
While social media spills over with hype and massive critic around AI and LLMs, truth isn't found in extreme opinions. My advise is to stay pragmatic and determine when and how AI can be turned into a useful tool. 
–Christoph
Spotlight: How To Call An LLM From Go
LLMs like Claude or ChatGPT are great for chatting, but why would one want to call a large language model (LLM) from code? Maybe to build a custom chatbot? Probably, but that would again lead to a human talking to an LLM. 
A more interesting question is: Can (and should) apps use LLMs to get results for their own purposes? 
&lt; announcement &gt; 
Absolutely; whenever an input isn't precise, or whenever an operation on input requires abilities like text comprehension, simple algorithms won't cut it. 
Imagine, for example, you want to write a user-friendly search interface. The user shall be able to query in natural language for data stored in a relational database. An LLM is the perfect way to deal with such imprecise input and turn it into proper SQL. 
But how difficult is it to call an LLM? In fact, it's embarrassingly easy! A simple REST API call with a few parameters in JSON format is all you need. 
Connect to any model
A wide array of LLMs is accessible through a REST API, and most of them use the OpenAI API specification. In the code I'm going to unroll below, I'll stick to the OpenAI API for this reason. With a few modifications, you can also call Anthropic's models or local models via Ollama. 
The OpenAI API has one endpoint, /chat/completions, that takes a model and one or more messages as input and delivers a message back. The input is a JSON structure, which I model in Go through the following types:
type ChatRequest struct {
    Model    string    `json:"model"`
    Messages []Message `json:"messages"`
}

type Message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type ChatResponse struct {
    Choices []struct {
        Message Message `json:"message"`
    } `json:"choices"`
}

For the purpose of this code, the Role will be set to "user" (that is, I use no "system" prompt), and I expect the response to never contain more than one choice. 
The API client
For the client, I build a custom HTTP client that knows the base URL, the API key, and the model to use:
type OpenAIClient struct {
    baseURL string
    apiKey  string
    model   string
    client  http.Client
}

func NewOpenAIClient(baseURL, key string) *OpenAIClient {
    return &OpenAIClient{
        baseURL: baseURL,
        apiKey:  key,
        client: http.Client{
            Timeout: 30 * time.Second,
        },
    }
}

The client shall call the /chat/completions endpoint with a model string and a prompt and extract the SQL query from the response.
For this, let's add a method "Chat" that first constructs a ChatReqest struct with all required input.
Then, it marshals the struct to JSON:
func (oc OpenAIClient) Chat(model, prompt string) (response string, err error) {
    request := ChatRequest{
        Model: model,
        Messages: []Message{
            {
                Role:    "user",
                Content: prompt,
            },
        },
    }
    jsonData, err := json.Marshal(request)
    if err != nil {
        return "", fmt.Errorf("Marshal: %v\n", err)
    }

Next, it builds the URL from base URL and end point. Keeping the baseURL a variable allows choosing the LLM provider later (provided the API is OpenAI-compatible).
    chatURL, err := url.JoinPath(oc.baseURL, "/chat/completions")
    if err != nil {
        return "", fmt.Errorf("JoinPath: %w", err)
    }

The following is basic http.Client stuff: Create a new POST request from the URL and the JSON data, set the request headers to specify the content type to be sent and the API key, and finally send the request:
    req, err := http.NewRequest("POST", chatURL, bytes.NewBuffer(jsonData))
    if err != nil {
        return "", fmt.Errorf("NewRequest: %w", err)
    }

    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", "Bearer "+oc.apiKey)

    resp, err := oc.client.Do(req)
    if err != nil {
        return "", fmt.Errorf("Do: %w", err)
    }
    defer resp.Body.Close()

Now, it can read the body, unmarshal the JSON blob, and extract the response content to return it to the caller:
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return "", fmt.Errorf("ReadAll: %w", err)
    }

    var chatResp ChatResponse
    if err := json.Unmarshal(body, &chatResp); err != nil {
        return "", fmt.Errorf("Unmarshal: %w", err)
    }

    return chatResp.Choices[0].Message.Content, nil

}

That's all! I said it would be easy. All that's left is setting up a client, creating a suitable prompt and call the endpoint.
To set up the client, we need to have the API key at hand, and the base URL of the provider. The code uses OpenAI but you can insert the base URL of your favorite OpenAI-compatible provider here:
func main() {
    apiKey := os.Getenv("OPENAI_API_KEY")
    if apiKey == "" {
        fmt.Println("Please set OPENAI_API_KEY environment variable")
        return
    }

    oc := NewOpenAIClient("https://api.openai.com/v1", apiKey)

Now, we need a prompt. For brevity, I "explain" the table and the relevant columns in plain words, but IRL, I would perhaps provide the complete table schema definition:
    prompt := `Turn the following request into a SQL query 
    against table 'users' with the columns 
    'name' (text), 
    'email' (text), 
    'status' (text), 
    'joined' (datetime), 
    and 'loggedin' (boolean): 

    'Find activated users who joined since June 2025. 
    I want to know their name and if they are logged in at the moment.' 

    Only return the bare SQL query, no markup or other text.`

This prompt might seem a bit sloppy but it works, as you can confirm after finishing the code with a call to Chat() and some code to "process" the result. (For brevity, I won't call a DB here but just print the SQL statement.)
For the model, I picked GPT-4o Mini as it is rather cheap. (Remember to have some $$ in your account's balance to run the model.)
    query, err := oc.Chat("gpt-4o-mini", prompt)
    if err != nil {
        log.Fatal(err)
    }

    // Process the generated SQL query
    fmt.Println(query)

}

To avoid stray API keys in my machine's environment, I usually set the API key environment variable only for the command that needs it, and I grab the value from gopass, a secrets manager that can sync through a Git remote. 
Running the code should yield a clear and precise SQL statement from the plain English search request (which is: 'Find activated users who joined since June 2025. I want to know their name and if they are logged in at the moment.")
OPENAI_API_KEY=$(gopass api/openai/cli) go run .
SELECT name, loggedin FROM users WHERE status = 'activated' AND joined < '2025-01-01';

Obviously, the prompt needs some refinement to advise the model to output only safe queries, to prevent prompt injection that could make the LLM generate a SQL injection. (A meta-injection, if you will.) Checking the generated SQL statement wouldn't be a bad idea either, but I omitted both for this demo code.
(Full code for copy&paste here.)
This code opens access to a broad range of models available through OpenAI API-compatible providers, and with a few tweaks, you can also call Anthropic's messages API or the Ollama API if you prefer to call local models.
Generating SQL statements is quite a simple task for an LLM; but the possibilities go beyond that.
How about –

Extracting data from unstructured documents
Applying sophisticated optical character recognition (OCR) to image-only PDFs
Filtering spam mails
Detecting log anomalies

or any task with either of these attributes:

A fuzzy input, 
An operation that can't be easily modeled by an algorithm (yes, LLMs are made of algorithms, I know, I know)
An operation that needs a vast amount of knowledge

Now pick an API key, a client, and a prompt and start building bridges between human ambiguity and algorithmic precision, just with the standard library and a few HTTP calls. 

Happy coding! ʕ◔ϖ◔ʔ
Questions or feedback? Drop me a line. I'd love to hear from you. 
Best from Munich, 
Christoph 
Not a subscriber yet?
If you read this newsletter issue online, or if someone forwarded the newsletter to you, subscribe for regular updates to get every new issue earlier than the online version, and more reliable than an occasional forwarding. 
Find the subscription form at the end of this page.
How I can help
If you're looking for more useful content around Go, here are some ways I can help you become a better Gopher (or a Gopher at all): 
On AppliedGo.net, I blog about Go projects, algorithms and data structures in Go, and other fun stuff. 
Or visit the AppliedGo.com blog and learn about language specifics, Go updates, and programming-related stuff.  
My AppliedGo YouTube channel hosts quick tip and crash course videos that help you get more productive and creative with Go. 
Enroll in my Go course for developers that stands out for its intense use of animated graphics for explaining abstract concepts in an intuitive way. Numerous short and concise lectures allow you to schedule your learning flow as you like. 
Check it out. 

Christoph Berger IT Products and Services

Dachauer Straße 29

Bergkirchen

Germany

Don't miss what's next. Subscribe to The Applied Go Weekly Newsletter: