Cover Page

Backend Page

`Go` with `Echo`

We assume that your chatterd code base has accumulated code from at least the llmPrompt, chatter backend tutorials. If you’ve, in addition, also accumulated code from latter tutorials, that’s fine.

`toolbox`

Let us start by creating a toolbox to hold our tools. Change to your chatterd folder and create a new Go file, name it toolbox.go:

server$ cd ~/reactive/chatterd
server$ vi toolbox.go

Put the following import() block at the top of the file:

import (
    "encoding/json"
    "fmt"
    "net/http"
)

The contents of this file can be categorized into three purposes: tool/function definition, the toolbox itself, and tool use (or function calling).

Tool/function definition

Ollama tool schema: at the top of Ollama’s JSON tool definition is a JSON Object respresenting a tool schema. The tool schema is defined using nested JSON Objects and JSON Arrays. Add the full nested definitions of Ollama’s tool schema to your file:

type OllamaToolSchema struct {
    Type     string             `json:"type"`
    Function OllamaToolFunction `json:"function"`
}

type OllamaToolFunction struct {
    Name        string               `json:"name"`
    Description string               `json:"description"`
    Parameters  OllamaFunctionParams `json:"parameters,omitempty"`
}

type OllamaFunctionParams struct {
    Type       string                     `json:"type"`
    Properties map[string]OllamaParamProp `json:"properties"`
    Required   []string                   `json:"required,omitempty"`
}

type OllamaParamProp struct {
    Type        string   `json:"type"`
    Description string   `json:"description"`
    Enum        []string `json:"enum,omitempty"`
}

Weather tool schema: in this tutorial, we have only one tool resident in the backend. Add the following tool definition to your file:

var WEATHER_TOOL = OllamaToolSchema{
    Type: "function",
    Function: OllamaToolFunction {
        Name:        "get_weather",
        Description: "Get current temperature",
        Parameters: OllamaFunctionParams{
            Type: "object",
            Properties: map[string]OllamaParamProp{
                "latitude": OllamaParamProp{
                    Type:        "string",
                    Description: "latitude of location of interest",
                },
                "longitude": OllamaParamProp{
                    Type:        "string",
                    Description: "longitude of location of interest",
                },
            },
            Required: []string{"latitude", "longitude"},
        },
    },
}

Weather tool function: we implement the get_weather tool as a getWeather() function that makes an API call to the free Open Meteo weather service. Add the following nested struct definition to hold Open Meteo’s return result. For this tutorial, we’re only interested in the latitude, longitude, and temperature returned by Open Meteo:

type OMeteoResponse struct {
    Lat     float64 `json:"latitude"`
    Lon     float64 `json:"longitude"`
    Current struct {
        Temp float64 `json:"temperature_2m"`
    } `json:"current"`
}

Here’s the definition of the getWeather() function:

func getWeather(argv []string) (*string, error) {
    // Open-Meteo API doc: https://open-meteo.com/en/docs#api_documentation
    response, err := http.DefaultClient.Get(fmt.Sprintf(
        "https://api.open-meteo.com/v1/forecast?latitude=%s&longitude=%s&current=temperature_2m&temperature_unit=fahrenheit",
        argv[0], argv[1],
    ))
    if err != nil {
        return nil, fmt.Errorf("Cannot connect to Open Meteo: %w", err)
    }
    defer func() {
        _ = response.Body.Close()
        http.DefaultClient.CloseIdleConnections()
    }()
    if response.StatusCode != http.StatusOK {
        return nil, fmt.Errorf("Open-meteo (%d): %s", response.StatusCode, response.Status)
    }
    var ometeoResponse OMeteoResponse
    if err = json.NewDecoder(response.Body).Decode(&ometeoResponse); err != nil {
        return nil, fmt.Errorf("Cannot decode Open Meteo's response: %w", err)
    }
    weather := fmt.Sprintf("Weather at lat: %f, lon: %f is %fºF",
        ometeoResponse.Lat, ometeoResponse.Lon, ometeoResponse.Current.Temp)
    return &weather, nil
}

The toolbox

Even though we have only one resident tool in this tutorial, we want a generalized architecture that can hold multiple tools and invoke the right tool dynamically. To that end, we’ve chosen to use a switch table (or jump table or, more fancily, service locator registry) as the data structure for our tool box. We implement the switch table as a dictionary. The “keys” in the dictionary are the names of the tools/functions. Each “value” is a record containing the tool’s definition/schema and a pointer to the function implementing the tool. To send a tool as part of a request to Ollama, we look up its schema in the switch table and copy it to the request. To invoke a tool called by Ollama in its response, we look up the tool’s function in the switch table and invoke the function.

Add the following type for a tool function and the record type containing a tool definition and the tool function:

type ToolFunction func(args []string) (*string, error)

type Tool struct {
    Schema    OllamaToolSchema
    Function  ToolFunction
    Arguments []string
}

Now create a switch-table toolbox and put the WEATHER_TOOL in it:

var TOOLBOX = map[string]Tool{
    "get_weather": {WEATHER_TOOL, getWeather, []string{"latitude", "longitude"}},
}

Tool use or function calling

Ollama tool call: Ollama’s JSON tool call comprises a JSON Object containing a nested JSON Object carrying the name of the function and the arguments to pass to it. Add these nested struct definitions representing Ollama’s tool call JSON to your file:

type OllamaToolCall struct {
    Function OllamaFunctionCall `json:"function"`
}

type OllamaFunctionCall struct {
    Name      string            `json:"name"`
    Arguments map[string]string `json:"arguments"`
}

Tool invocation: finally, here’s the tool invocation function. We call this function to execute any tool call we receive from Ollama response. It looks up the toolbox for the tool name. If the tool is resident, it runs it and returns the result, otherwise it returns a null.

func toolInvoke(function OllamaFunctionCall) (*string, error) {
    tool, ok := TOOLBOX[function.Name]

    if ok {
        var argv []string
        for _, label := range tool.Arguments {
            // get arguments in order, Dict doesn't preserve insertion order
            arg := function.Arguments[label]
            argv = append(argv, arg)
        }
        return tool.Function(argv)
    }
    return nil, nil
}

That concludes our toolbox definition. Save and exit the file.

`handlers`

Edit handlers.go:

server$ vi handlers.go

imports

If you don’t have code from the llmChat tutorial or llmPlay project in your code base, add the following imports to the import() block at the top of the file:

    "bufio"
    "encoding/json"
    "errors"
    "fmt"
    "regexp"
    "strings"

`structs`

Next add or update the following structs:

if update, add a tool-calls field to your OllamaMessage struct:

type OllamaMessage struct {
  Role    string `json:"role"`
  Content string `json:"content"`
  ToolCalls []OllamaToolCall `json:"tool_calls,omitempty"`
}

if update, add a tools field to your OllamaRequest:

type OllamaRequest struct {
  AppID    string    `json:"appID"`
  Model    string    `json:"model"`
  Messages []OllamaMessage `json:"messages"`
  Stream   bool      `json:"stream"`
  Tools    []OllamaToolSchema `json:"tools,omitempty"`
}

The OllamaResponse struct (unchanged if exists):

type OllamaResponse struct {
    Model     string  `json:"model"`
    CreatedAt string  `json:"created_at"`
    Message   OllamaMessage `json:"message"`
}

For the /weather testing API, add also the following struct:

type Location struct {
    Lat string `json:"lat"`
    Lon string `json:"lon"`
}

`weather`

Let’s implement the handler for the /weather API that we can use to test our getWeather() function later:

func weather(c echo.Context) error {
    var loc Location

    if err := c.Bind(&loc); err != nil {
        return logClientErr(c, http.StatusUnprocessableEntity, err)
    }

    temp, err := getWeather([]string{loc.Lat, loc.Lon})
    if err != nil {
        return logServerErr(c, err)
    }
    logOk(c)
    return c.JSON(http.StatusOK, temp)
}

`llmtools`

The underlying request/response handling of llmtools() is basically that of llmchat(), however with all the mods needed to support tool calling, it’s simpler to just start the llmtools() handler from scratch. We will name variables according to this scheme:

camelCase for language-level data objects,
snake_case for string version of data objects to be used with PostgreSQL or JSON, and, earlier,
ALL_CAPS for immutable global toolbox and tool definitions.

To store the client’s conversation context/history with Ollama in the PostgreSQL database, llmtools() first confirms that the client has sent an appID that can be used to tag its entries in the database. Here’s the signature of llmtools(). We check for the existence of appID and return an HTTP error if it is absent:

func llmtools(c echo.Context) error {
    err := errors.New("")
    var ollamaRequest OllamaRequest

    if err = c.Bind(&ollamaRequest); err != nil {
        return logClientErr(c, http.StatusUnprocessableEntity, err)
    }

    if len(ollamaRequest.AppID) == 0 {
        return logClientErr(c, http.StatusUnprocessableEntity, fmt.Errorf("invalid appID: %s", ollamaRequest.AppID))
    }

        // retrieve client's tool(s)

}

Our goal here is to prepend all prior conversations between the client and Ollama as context to the current prompt. The client’s appID allows us to identify its conversation with Ollama stored in the PostgreSQL database—similar to how MCP tags JSON-RPC 2.0 messages with a session ID. Once we confirm that the client has an appID, we retrieve any tool definitions attached to the ollamaRequest carrying the prompt. We will assemble these tools along with any tools the client may have previously sent to Ollama, attached to an earlier prompt, and any tools resident on chatterd and attach them all to the contextualized prompt request we will POST to Ollama. Replace // retrieve client's tool(s) with:

    // convert tools from client as JSON string (client_tools) and save to db;
    // prepare ollama_request for re-use to be sent to Ollama:
    // clear tools in request, to be populated later
    var client_tools []byte
    if ollamaRequest.Tools != nil {
        // has device tools
        // must marshal to string to store to postgres
        client_tools, _ = json.Marshal(ollamaRequest.Tools)

        // reset tools, to be populated with
        // accumulated tools below, without duplicates
        ollamaRequest.Tools = nil
    }

    // insert into DB

Then we insert the current prompt into the database, adding to the client’s conversation history with Ollama. As shown in the example in Tool definition JSON section, the client’s current prompt could comprise of multiple elements in the messages array of the ollamaRequest, but the tools will reside in a single tools array next to the messages array. When there are multiple elements in an ollamaRequest, we want to insert the tools only once. Below we have chosen to insert the tools only with the first element of the messages array. Replace the comment // insert into DB with the following code:

    // insert each message into the database
    // insert client_tools only with the first message:
    // reset it to empty after first message.
    for _, msg := range ollamaRequest.Messages {
        _, err = chatterDB.Exec(background, `INSERT INTO chatts (username, message, id, appid, toolschemas) VALUES ($1, $2, gen_random_uuid(), $3, $4)`,
            msg.Role, msg.Content, ollamaRequest.AppID, client_tools)
        if err != nil {
            return logServerErr(c, err)
        }

        // store device's tools only once
        client_tools = nil
    }

    // assemble resident tools

To prepare the full assemblage of tools to send to Ollama, we first attach all the tools resident on chatterd. Replace // assemble resident tools with:

    // append all of chatterd's resident tools to ollamaRequest
    for _, tool := range TOOLBOX {
        ollamaRequest.Tools = append(ollamaRequest.Tools, tool.Schema)
    }

    // reconstruct ollamaRequest

Then we retrieve the client’s conversation history, including the recently inserted, current prompt, as the last entry, and put each as a separate element in the ollamaRequest.messages array, taking care to accumulate any tool(s) present into ollamaRequest.tools array instead. Replace // reconstruct ollamaRequest with:

    // reconstruct ollamaRequest to be sent to Ollama:
    // - add context: retrieve all past messages by appID,
    //   incl. the one just received, and attach them to
    //   ollamaRequest
    // - convert each back to OllamaMessage and
    // - insert it into ollamaRequest
    // - add each message's clientTools to chatterd's resident tools
    //   already copied to ollama_request.tools.
    req := c.Request()
    reqCtx := req.Context()
    ollamaRequest.Messages = nil
    rows, err := chatterDB.Query(reqCtx, `SELECT username, message, toolcalls, toolschemas FROM chatts WHERE appid = $1 ORDER BY time ASC`, ollamaRequest.AppID)
    if err != nil {
        if rows != nil {
            rows.Close()
        }
        return logServerErr(c, err)
    }
    for rows.Next() {
        msg, err := OllamaMessageFromRow(rows, &ollamaRequest)
        if err != nil {
            rows.Close()
            return logServerErr(c, err)
        }
        ollamaRequest.Messages = append(ollamaRequest.Messages, *msg)
    }

    // NDJSON to SSE stream transformation

Go does not support static method. Instead, define a global function OllamaMessageFromRow outside your llmtools() function, for example right under, and also outside, the definition of type OllamaMessage struct {}, at the top of the file:

func OllamaMessageFromRow(rows pgx.Rows, ollamaRequest *OllamaRequest) (*OllamaMessage, error) {
    var msg OllamaMessage
    var toolcalls []byte
    var toolschemas []byte

    err := rows.Scan(&msg.Role, &msg.Content, &toolcalls, &toolschemas)
    if err != nil {
        return &msg, err
    }

    if toolcalls != nil {
        var toolCalls []OllamaToolCall
        // must unmarshal to type to append toolcalls
        _ = json.Unmarshal(toolcalls, &toolCalls)
        msg.ToolCalls = append(msg.ToolCalls, toolCalls...)
    }

    if toolschemas != nil {
        // has device tools
        var tools []OllamaToolSchema
        // must unmarshal to type to append device tools to ollamaRequest.tools
        _ = json.Unmarshal(toolschemas, &tools)
        ollamaRequest.Tools = append(ollamaRequest.Tools, tools...)
    }

    return &msg, nil
}

NDJSON to SSE stream transformation

As we know, Ollama response is in the form of an NDJSON stream, which we transform into a stream of SSE events to be sent to the client. We first prepare the response header with which to send the SSE stream. Then we declare an accumulator variable, full_response, to assemble the reply tokens Ollama streams to us. To accommodate resident-tool call, we use a flag, sendNewPrompt, to indicate to our stream generator whether:

to start a resident-tool call connection to Ollama and continue yielding results to the client or

to conclude streaming to the connection. While sendNewPrompt is true—it is initialized to true, we open a new POST connection to Ollama and send it the ollamaRequest message. Place the following below the // NDJSON to SSE stream transformation comment:

  res := c.Response()
  res.Header().Set(echo.HeaderContentType, "text/event-stream")
  res.Header().Set(echo.HeaderCacheControl, "no-cache")

  var full_response string
  sendNewPrompt := true

  for sendNewPrompt {
      sendNewPrompt = false // assume no resident-tool calls

      request, err := json.Marshal(&ollamaRequest) // convert the request to JSON
      if err != nil {
          err_msg, _ := json.Marshal(err.Error())
          _, _ = fmt.Fprintf(res, "event: error\ndata: { \"error\": %s }\n\n", string(err_msg))
          res.Flush()
          return err 
      }
      ollama_url := OLLAMA_BASE_URL.String() + "/chat"
      // Send request to Ollama
      client, _ := http.NewRequestWithContext(reqCtx, req.Method, ollama_url, bytes.NewReader(request))

      response, err := http.DefaultClient.Do(client)
      if err != nil {
          err_msg, _ := json.Marshal(err.Error())
          _, _ = fmt.Fprintf(res, "event: error\ndata: { \"error\": %s }\n\n", string(err_msg))
          res.Flush()
          return err 
      }
      defer func() {
          _ = response.Body.Close()
      }()

      // handle Ollama response

  } // for sendNewPrompt 

  logOk(c)
  return nil

We convert each NDJSON line to a language-level type, OllamaResponse in this case, with semantically meaningful structure and fields that we can more easily manipulate than a linear byte stream or string. If the conversion is unsuccessful and the Model property of the type is empty, we return an SSE error event and move on to the next NDJSON line. Otherwise, we append the content of this OllamaResponse.message to the full_response accumulator. Replace // handle Ollama response with:

        var tool_calls []byte
        var tool_result = ""

        scanner := bufio.NewScanner(response.Body)
        for scanner.Scan() {
            line := scanner.Text()

            var ollamaResponse OllamaResponse
            // deserialize each line into OllamaResponse
            if err = json.Unmarshal([]byte(line), &ollamaResponse); err == nil {
                if ollamaResponse.Model == "" {
                    // didn't receive an ollamaresponse, likely got an error message
                    _, _ = fmt.Fprintf(res, "event: error\ndata: %s\n\n",
                        strings.ReplaceAll(line, "\\\"", "'"))
                    res.Flush()
                    continue
                }

                // append response token to full assistant message
                full_response += ollamaResponse.Message.Content

                // check for tool call

            } else {
                err_msg, _ := json.Marshal(fmt.Sprintf("%s", err))
                _, _ = fmt.Fprintf(res, "event: error\ndata: { \"error\": %s }\n\n", string(err_msg))
                res.Flush()
            }
        }

        if err = scanner.Err(); err != nil {
            err_msg, _ := json.Marshal(fmt.Sprintf("%s", err))
            _, _ = fmt.Fprintf(res, "event: error\ndata: { \"error\": %s }\n\n", string(err_msg))
            res.Flush()
        }     
                    
        // insert full response into db

The tool call field in OllamaResponse is an array, even though it looks like Qwen3 on Ollama is presently limited to making only one tool call per HTTP round. We loop through the array and for each tool call, we try to call its function by calling toolInvoke() from our toolbox. If there is no tool call, we simply encode the full NDJSON line into an SSE Message event and yield it as an element of the SSE stream and move on to the next NDJSON line, as we do in llmchat. Replace // check for tool call with:

                // is there a tool call?
                if len(ollamaResponse.Message.ToolCalls) != 0 {
                    // convert toolCalls to JSON string (tool_calls) and save to db
                    tool_calls, _ = json.Marshal(ollamaResponse.Message.ToolCalls)

                    for _, toolCall := range ollamaResponse.Message.ToolCalls {
                        if toolCall.Function.Name == "" {
                            continue // LLM miscalled
                        }
                        toolResult, err := toolInvoke(toolCall.Function)

                        // handle tool result

                    } // for _, toolCall
                } else {
                    // no tool call, send NDJSON line as SSE data line
                    _, _ = fmt.Fprintf(res, "data: %s\n\n", line)
                    res.Flush()
                }

If the tool is resident, toolInvoke() returns the result of the tool call. There are three possible outcomes from the call to toolInvoke():

the tool is resident but the call was unsuccesfull and returns an error,
the tool is resident and the call was successful, or
the tool is non-resident.

If the result indicates that an error has occured, we are dealing with the first outcoe above. We simply report the error to the client and move on to the next NDJSON line. If there’s no error but toolInvoke() returns null result, this indicates that the tool is non resident. We forward the tool call to the client as a tool_calls SSE event. Otherwise, we prepare the result to be saved to PostgreSQL and return the result to Ollama. Replace // handle tool result with:

                        if toolResult != nil {
                            // outcome 2: tool call is resident

                            // convert toolResult to JSON string (tool_result)
                            // to be saved to db
                            if tool_result == "" {
                                tool_result = *toolResult
                            } else {
                                tool_result += " " + *toolResult
                            }

                            // create new OllamaMessage with tool result
                            // to be sent back to Ollama
                            var toolresultMsg = OllamaMessage{
                                Role:    "tool",
                                Content: *toolResult,
                            }
                            ollamaRequest.Messages = append(ollamaRequest.Messages, toolresultMsg)

                            // send result back to Ollama
                            sendNewPrompt = true
                        } else if err != nil {
                            // outcome 1: tool resident but had error
                            err_msg, _ := json.Marshal(err.Error())
                            _, _ = fmt.Fprintf(res, "event: error\ndata: { \"error\": %s }\n\n", string(err_msg))
                            res.Flush()
                        } else {
                            // outcome 3: tool non resident, forward
                            // to device as 'tool_calls' SSE event
                            _, _ = fmt.Fprintf(res, "event: tool_calls\ndata: %s\n\n", line)
                            res.Flush()
                        }

When we reach the end of the NDJSON stream, we insert the full Ollama response and any resident tool calls and their results into PostgreSQL database as the assistant’s reply. Any error in the insertion yields an SSE error event sent to the client. Replace // insert full response into db with:

        // save full response to db
        if full_response != "" {
            wsRegex := regexp.MustCompile("\\s+")

            // save full response, including tool call(s), to db,
            // to form part of next prompt's history
            _, err =
                chatterDB.Exec(background, `INSERT INTO chatts (username, message, id, appid, toolcalls) 
                VALUES ('assistant', $1, gen_random_uuid(), $2, $3)`,
                    wsRegex.ReplaceAllString(full_response, " "),
                    ollamaRequest.AppID, tool_calls)

            // if there were resident tool call(s), save result(s)
            if err == nil && sendNewPrompt {
                _, err =
                    chatterDB.Exec(background, `INSERT INTO chatts (username, message, id, appid) 
                VALUES ('tool', $1, gen_random_uuid(), $2)`,
                        tool_result, ollamaRequest.AppID)
            }

            if err != nil {
                err_msg, _ := json.Marshal(fmt.Sprintf("%s", err))
                _, _ = fmt.Fprintf(res, "event: error\ndata: { \"error\": %s }\n\n", string(err_msg))
                res.Flush()
            }
        }

We’re done with handlers.go! Save and exit the file.

`main.go` package

Edit main.go:

server$ vi main.go

Find the global variable router and add these routes right after the route for /llmprompt:

    {"POST", "/llmtools/", llmtools},
    {"GET", "/weather/", weather},

We’re done with main.go. Save and exit the file.

Build and test run

To build your server:

server$ go get   # -u  # to upgrade all packages to the latest version
server$ go build

Go is a compiled language, like C/C++ and unlike Python, which is an interpreted language. This means you must run go build each and every time you made changes to your code, for the changes to show up in your executable.

To run your server:

server$ sudo ./chatterd
# Hit ^C to end the test

Return to the Testing your /llmtools API section.

Prepared by Xin Jie ‘Joyce’ Liu, Chenglin Li, and Sugih Jamin

Last updated August 26th, 2025

Cover Page

Backend Page

Go with Echo

toolbox