nearle_daily/backend_fiesta
2
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Nearle Buddy answers a typed question

Browse Source 
Phase 2: the loop and the model gateway. The composer in the console has
said "Not connected yet" since it was built, because there was no
assistant endpoint anywhere. There is one now.

- utils/chat.go   the gateway, a sibling of embedding.go: one small
                  interface, a provider switch, the shared postJSON, no
                  framework. Agents name a TIER (fast/balanced/deep) and
                  config maps tier to model, so changing provider does not
                  touch an agent.
- services/assistantService.go  one loop for every agent. An agent is a
                  name, a tier, a prompt and an allow-list — data, not a
                  class — so a sixth is config rather than a subclass.
- the endpoint under /v1/web, inheriting middleware.WebAuth along with
  every other console route. The assistant reads the same data the console
  does and must read it as the same person.

What the model does not get to decide:

  whose data      the caller is built from the verified session in the
                  controller, never from the request body — there is no
                  tenant field to fill in. A test scripts the model calling
                  a tool with {"tenantid": 916} and asserts it ran for 1147.
  which tools     the registry enforces the agent's allow-list; a test
                  scripts a call to a tool the agent lacks and asserts the
                  handler never ran.
  when to stop    steps and tool calls are counted here. A model that keeps
                  calling tools is stopped by arithmetic, not by being
                  asked nicely.

Two quiet failures have tests of their own. A finish_reason of "length"
means the provider cut the reply off mid-sentence, which reads exactly
like a complete answer unless it is flagged. And a truncated tool result
reaches the model in words it will repeat — otherwise it describes a
capped list and an empty one identically.

A refused tool goes back as a message, not an error: a model told "that
tool needs a tenant" can explain it, where a model handed nothing says
"something went wrong".

Optional, like the embedder. Without ASSISTANT_PROVIDER the endpoint
answers "not switched on here", the composer stays disabled, and the tools
still work — they are ordinary Go functions, and only turning a sentence
into a tool call needs a model.

14 tests, against a scripted model rather than a live provider: these are
about what the loop refuses to let a model do, and that has to hold for
any model, including one behaving badly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
abhishek
2026-09-23 13:13:20 +05:30
parent bb5f40926f
commit 8e1549764b
9 changed files with 1193 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
config/config.go
View File

@@ -71,6 +71,9 @@ type Config struct {
S3 S3Config
MQTT MQTTConfig
Embedding EmbeddingConfig
// Assistant is the model behind Nearle Buddy. Empty provider = no typed
// questions; the tools still work.
Assistant AssistantConfig
// POSTokenSecret signs terminal sessions. Falls back to JWTSecret when
// unset, matching utils/postoken.go.
@@ -141,6 +144,54 @@ type EmbeddingConfig struct {
func (e EmbeddingConfig) Enabled() bool { return e.Provider != "" }
// AssistantConfig is the model behind Nearle Buddy.
//
// Optional, like the embedder. With no provider the assistant refuses typed
// questions and says so — the tools still work and still answer correctly,
// because they are ordinary Go functions; only the part that turns a sentence
// into a tool call is missing.
//
// ── Why three models and not one ────────────────────────────────────────────
//
// An agent names a TIER, never a model. "Which branch is underperforming?"
// and "why is the cancel rate high?" want different amounts of thinking, and
// wiring a model name into an agent means changing every agent to change
// provider. The tiers are the stable vocabulary; this map is the only place a
// model name appears.
//
// `ASSISTANT_MODEL` alone sets all three, which is the sane default for a
// deployment that has not thought about it yet.
type AssistantConfig struct {
Provider string // "openai" — any OpenAI-compatible endpoint
BaseURL string // default https://api.openai.com/v1
APIKey string
// Tier → model name. Empty falls back to Balanced, which falls back to
// ASSISTANT_MODEL.
Fast string
Balanced string
Deep string
}
func (a AssistantConfig) Enabled() bool { return a.Provider != "" && a.Balanced != "" }
// ModelFor resolves a tier to a model name, falling back rather than failing.
//
// A missing `fast` model should answer a cheap question with the balanced one,
// not refuse it. A deployment that sets one model gets one model everywhere.
func (a AssistantConfig) ModelFor(tier string) string {
switch tier {
case "fast":
if a.Fast != "" {
return a.Fast
}
case "deep":
if a.Deep != "" {
return a.Deep
}
}
return a.Balanced
}
// IsProduction is true under APP_ENV=production.
func (c *Config) IsProduction() bool { return c.AppEnv == EnvProduction }
@@ -197,6 +248,17 @@ func Load() (*Config, error) {
BaseURL: env("EMBEDDING_BASE_URL", ""),
},
Assistant: AssistantConfig{
Provider: strings.ToLower(env("ASSISTANT_PROVIDER", "")),
BaseURL: env("ASSISTANT_BASE_URL", ""),
APIKey: env("ASSISTANT_API_KEY", ""),
Fast: env("ASSISTANT_MODEL_FAST", ""),
// ASSISTANT_MODEL alone sets every tier, for a deployment that has
// not thought about tiers yet.
Balanced: env("ASSISTANT_MODEL_BALANCED", env("ASSISTANT_MODEL", "")),
Deep: env("ASSISTANT_MODEL_DEEP", ""),
},
POSTokenSecret: env("POS_TOKEN_SECRET", ""),
JWTSecret: env("JWT_SECRET_KEY", ""),
UserContextKey: env("USER_CONTEXT_KEY", "nearle"),

122
controllers/assistantController.go Normal file
View File

@@ -0,0 +1,122 @@
package controllers
import (
"errors"
"net/http"
"strings"
"nearle/middleware"
"nearle/services"
"nearle/services/tools"
"nearle/utils"
"github.com/gofiber/fiber/v2"
)
// Nearle Buddy's HTTP surface.
//
// POST /v1/web/assistant/ask a question → an answer, and what it ran
// GET /v1/web/assistant/status is this switched on here?
//
// ── Where the caller comes from ─────────────────────────────────────────────
//
// `middleware.WebAuth` parks the verified claims on the request, and this
// builds the tool caller from those and from nothing else. There is no tenant
// field on the request body — deliberately, so there is nothing for a model or
// a caller to fill in. The console asks "what is stuck?" and the server already
// knows whose shop that means.
type AssistantController struct {
assistant services.AssistantService
}
func NewAssistantController(assistant services.AssistantService) *AssistantController {
return &AssistantController{assistant: assistant}
}
type assistantAskRequest struct {
// Which agent to ask. The console sends the one matching the page the panel
// is sitting beside; empty means orders, the only one phase 2 ships.
Agent string `json:"agent"`
Question string `json:"question"`
}
// Status lets the console decide what to render before anybody types.
//
// The composer is disabled when this says no, which is the honest thing: a
// field that accepts text and then swallows it is worse than one that says it
// is not connected. The console has shown "Not connected yet" since it was
// built, and this is what finally answers that question at runtime rather than
// at build time.
func (ctl *AssistantController) Status(c *fiber.Ctx) error {
return c.Status(http.StatusOK).JSON(fiber.Map{
"code": http.StatusOK, "status": true, "message": "Success",
"details": fiber.Map{"available": ctl.assistant.Available()},
})
}
func (ctl *AssistantController) Ask(c *fiber.Ctx) error {
var req assistantAskRequest
if err := c.BodyParser(&req); err != nil {
return assistantRefuse(c, http.StatusBadRequest, "Invalid request body")
}
caller, ok := callerFrom(c)
if !ok {
// Reachable only while WEB_AUTH_REQUIRED is off, where an untokened
// request still reaches handlers. Every other endpoint answers such a
// request; this one must not. Reading a shop's orders through a REST
// call takes knowing the endpoints and the fields; through an
// assistant it takes one sentence, so this surface holds the higher
// bar from its first day rather than inheriting the rollout's.
return assistantRefuse(c, http.StatusUnauthorized, "Sign in again to use Nearle Buddy.")
}
agent := strings.TrimSpace(req.Agent)
if agent == "" {
agent = "orders"
}
ctx, cancel := services.WithTimeout(c.Context())
defer cancel()
answer, err := ctl.assistant.Ask(ctx, agent, req.Question, caller)
if err != nil {
// "Not switched on here" is a deployment fact, not a fault, and it gets
// its own status so the console can disable the composer rather than
// showing an error the person can do nothing about.
if errors.Is(err, utils.ErrChatNotConfigured) {
return c.Status(http.StatusOK).JSON(fiber.Map{
"code": http.StatusServiceUnavailable, "status": false,
"message": "Nearle Buddy is not switched on for this deployment.",
})
}
return assistantRefuse(c, http.StatusBadRequest, err.Error())
}
return c.Status(http.StatusOK).JSON(fiber.Map{
"code": http.StatusOK, "status": true, "message": "Success", "details": answer,
})
}
// callerFrom turns a verified session into a tool caller.
//
// The one place the two vocabularies meet. Staff (`issuperadmin`) carry no
// tenant, and the registry lets them through — but a tool that reads a shop's
// data refuses them until they have picked one, because "every tenant at once"
// is not an answer to "what is stuck?".
func callerFrom(c *fiber.Ctx) (tools.Caller, bool) {
claims, ok := middleware.WebClaimsFrom(c)
if !ok {
return tools.Caller{}, false
}
return tools.Caller{
Userid: claims.Userid,
Tenantid: claims.Tenantid,
Locationid: claims.Locationid,
Superadmin: claims.Superadmin,
}, true
}
func assistantRefuse(c *fiber.Ctx, code int, message string) error {
return c.Status(code).JSON(fiber.Map{"code": code, "status": false, "message": message})
}

12
facade/container.go
View File

@@ -25,6 +25,7 @@ type Facade struct {
LiveController *controllers.LiveController
CatalogueUploadController *controllers.CatalogueUploadController
ScanController *controllers.ScanController
AssistantController *controllers.AssistantController
// Tools is what Nearle Buddy is allowed to do.
//
@@ -43,7 +44,7 @@ type Facade struct {
// it may be nil if catalogue env vars are not configured, in which case
// catalogue endpoints will error at query time rather than at startup.
// embedder may be nil too: scan-to-order then matches on words alone.
func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder) *Facade {
func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder, chat utils.Chat) *Facade {
// User Module
userRepo := repositories.NewUserRepository(db)
@@ -143,6 +144,14 @@ func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder) *Faca
}
}
// Nearle Buddy. `chat` may be nil — a deployment with no model configured
// still gets the registry and the endpoint, and the endpoint answers "not
// switched on here" rather than a 500. The tools themselves are ordinary
// Go functions and work either way; only turning a sentence into a tool
// call needs a model.
assistantService := services.NewAssistantService(toolRegistry, chat, nil)
assistantController := controllers.NewAssistantController(assistantService)
return &Facade{
UserController: userController,
ProductController: productController,
@@ -158,6 +167,7 @@ func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder) *Faca
LiveController: liveController,
CatalogueUploadController: catalogueUploadController,
ScanController: scanController,
AssistantController: assistantController,
Tools: toolRegistry,
posService: posService,
}

16
main.go
View File

@@ -362,7 +362,21 @@ func main() {
log.Printf("scan: product search uses %s/%s", cfg.Embedding.Provider, cfg.Embedding.Model)
}
f := facade.NewFacade(db.DB, db.CatalogueDB, embedder)
// The model behind Nearle Buddy. Optional in the same way: without
// ASSISTANT_PROVIDER the tools still work and the panel says the assistant
// is not switched on, rather than the console showing a field that accepts
// text and swallows it.
chat, err := utils.NewChat(cfg.Assistant)
if err != nil {
log.Fatal("assistant provider:", err)
}
if chat == nil {
log.Println("assistant: ASSISTANT_PROVIDER not set, Nearle Buddy answers no typed questions")
} else {
log.Printf("assistant: %s, balanced tier is %s", cfg.Assistant.Provider, cfg.Assistant.ModelFor(utils.TierBalanced))
}
f := facade.NewFacade(db.DB, db.CatalogueDB, embedder, chat)
routes.RegisterRoutes(app, f)

20
routes/assistantroutes.go Normal file
View File

@@ -0,0 +1,20 @@
package routes
import (
"nearle/facade"
"github.com/gofiber/fiber/v2"
)
// Nearle Buddy. See controllers/assistantController.go for the two calls and
// services/assistantService.go for the loop behind them.
//
// Under `/v1/web`, so it inherits `middleware.WebAuth` along with every other
// console route — which is the point. The assistant reads the same data the
// console does, and it must read it as the same person.
func RegisterAssistantRoutes(api fiber.Router, f *facade.Facade) {
assistant := api.Group("/v1/web/assistant")
assistant.Get("/status", f.AssistantController.Status)
assistant.Post("/ask", f.AssistantController.Ask)
}

1
routes/routes.go
View File

@@ -39,4 +39,5 @@ func RegisterRoutes(app *fiber.App, f *facade.Facade) {
RegisterPosRoutes(api, f)
RegisterUploadRoutes(api, f)
RegisterScanRoutes(api, f)
RegisterAssistantRoutes(api, f)
}

300
services/assistantService.go Normal file
View File

@@ -0,0 +1,300 @@
package services
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"nearle/services/tools"
"nearle/utils"
)
// Nearle Buddy's loop.
//
// One loop runs every agent. An agent is a name, a tier, a system prompt and an
// allow-list — data, not a class — so a sixth agent is a config entry rather
// than a subclass, and the behaviour they all share cannot drift between them.
//
// The shape is the ordinary one: ask the model, run any tools it asked for,
// give it the results, ask again, stop when it answers in words. What matters
// is what the loop refuses to let the model decide.
//
// ── What the model does not get to choose ───────────────────────────────────
//
// - Whose data it reads. The caller comes from the verified session and is
// passed to the registry directly. No tool accepts a tenant argument.
// - Which tools exist. The allow-list is the agent's, enforced by the
// registry; a model asking for something else is refused, not obeyed.
// - When to stop. Steps and tool calls are counted here. A model that keeps
// calling tools is stopped by arithmetic, not by being asked nicely.
//
// ── A refused tool is a message, not an error ───────────────────────────────
//
// When the registry refuses a call, the refusal goes back to the model as the
// tool's result. A model told "that tool needs a tenant" can explain the
// problem to the person; a model handed a 500 says "something went wrong",
// which is true and useless. The refusal is still audited either way.
// Agent is one assistant, as data.
type Agent struct {
Name string
Tier string
// System is what the model is told about its job. Rules that MUST hold do
// not live here — a prompt is a request. This is for tone, scope and the
// habits that make an answer useful.
System string
Tools []string
// MaxSteps bounds the conversation: one step is one model round trip.
MaxSteps int
// MaxToolCalls bounds the work across the whole conversation, because a
// model can ask for several tools in a single step.
MaxToolCalls int
}
// DefaultAgents are the agents phase 2 ships with.
//
// One, deliberately. The loop, the gateway, the transport and the audit trail
// are the parts that are expensive to change later, and they are easier to get
// right against a single agent with a single tool than across five. The other
// four are a config change once this one is proven — which is phase 3, where
// they move to YAML and stop being Go at all.
var DefaultAgents = map[string]Agent{
"orders": {
Name: "orders",
Tier: utils.TierBalanced,
System: strings.TrimSpace(`
You are Nearle Buddy, helping a shopkeeper run their business from the Nearle console.
Answer from tool results and nothing else. Every number you state must have come
from a tool in this conversation. If no tool can answer, say so plainly and say
what you would need — never estimate, and never fill a gap from general knowledge
about retail.
When a tool returns no rows, that is an answer: say there are none. Do not
describe an empty result as a problem with the system.
When a result says it was truncated, say so in your reply. Do not describe a
capped list as the full picture.
Say what your answer covers — one branch or all of them — using the scope the
tool reports.
Be brief. A shopkeeper reading this is mid-shift: lead with the answer, then the
detail that makes it actionable. No preamble, no restating the question.`),
Tools: []string{"stuck_orders"},
MaxSteps: 4,
MaxToolCalls: 6,
},
}
// AssistantAnswer is what one question produced.
type AssistantAnswer struct {
Reply string `json:"reply"`
// What was actually run, in order. Returned to the console so an answer can
// show its working — Buddy states a conclusion, and this is how a person
// sees which numbers it came from.
Used []AssistantStep `json:"used,omitempty"`
Model string `json:"model,omitempty"`
// Where to go and check. Collected from the tools that answered.
Sources []string `json:"sources,omitempty"`
// True when the loop stopped on its own limits rather than because the
// model finished. The reply is still returned — a partial answer beats a
// spinner — but it is flagged rather than passed off as complete.
Incomplete bool `json:"incomplete,omitempty"`
}
// AssistantStep is one tool call, for the console to render.
type AssistantStep struct {
Tool string `json:"tool"`
// Refused calls are included on purpose. An answer that quietly dropped a
// refusal would look like the assistant chose not to look.
Outcome string `json:"outcome"`
Rows int `json:"rows,omitempty"`
Detail string `json:"detail,omitempty"`
Scope string `json:"scope,omitempty"`
}
// AssistantService answers a question.
type AssistantService interface {
Ask(ctx context.Context, agentName, question string, caller tools.Caller) (AssistantAnswer, error)
// Available reports whether typed questions work at all here.
Available() bool
}
type assistantService struct {
registry *tools.Registry
chat utils.Chat
agents map[string]Agent
}
func NewAssistantService(registry *tools.Registry, chat utils.Chat, agents map[string]Agent) AssistantService {
if agents == nil {
agents = DefaultAgents
}
return &assistantService{registry: registry, chat: chat, agents: agents}
}
func (s *assistantService) Available() bool { return s.chat != nil }
// maxQuestion bounds what a person can send.
//
// Not a safety control — it is a cost one. A pasted spreadsheet as a "question"
// is a large bill and a worse answer.
const maxQuestion = 4000
func (s *assistantService) Ask(ctx context.Context, agentName, question string, caller tools.Caller) (AssistantAnswer, error) {
question = strings.TrimSpace(question)
if question == "" {
return AssistantAnswer{}, fmt.Errorf("ask a question")
}
if len(question) > maxQuestion {
return AssistantAnswer{}, fmt.Errorf("that question is too long; keep it under %d characters", maxQuestion)
}
if s.chat == nil {
return AssistantAnswer{}, utils.ErrChatNotConfigured
}
agent, known := s.agents[agentName]
if !known {
return AssistantAnswer{}, fmt.Errorf("no assistant called %q", agentName)
}
allow := tools.Agent{Name: agent.Name, Tools: agent.Tools}
messages := []utils.Message{
{Role: utils.RoleSystem, Content: agent.System},
{Role: utils.RoleUser, Content: question},
}
answer := AssistantAnswer{Model: s.chat.ModelFor(agent.Tier)}
calls := 0
seenSource := map[string]bool{}
for step := 0; step < agent.MaxSteps; step++ {
reply, err := s.chat.Complete(ctx, utils.ChatRequest{
Tier: agent.Tier,
Messages: messages,
Tools: s.registry.Definitions(allow),
})
if err != nil {
return AssistantAnswer{}, err
}
answer.Model = reply.Model
if len(reply.ToolCalls) == 0 {
answer.Reply = strings.TrimSpace(reply.Content)
// `length` means the provider cut the reply off mid-sentence. A
// truncated answer reads exactly like a complete one unless it is
// flagged here.
if reply.StopReason == "length" {
answer.Incomplete = true
}
return answer, nil
}
// The assistant turn has to go back verbatim, tool calls and all, or
// the model has no record of what it asked for and asks again.
messages = append(messages, utils.Message{
Role: utils.RoleAssistant,
Content: reply.Content,
ToolCalls: reply.ToolCalls,
})
for _, call := range reply.ToolCalls {
if calls >= agent.MaxToolCalls {
answer.Incomplete = true
messages = append(messages, utils.Message{
Role: utils.RoleTool,
ToolCallID: call.ID,
Name: call.Name,
Content: "Refused: this conversation has already run its maximum number of tool calls. Answer with what you have and say it is partial.",
})
continue
}
calls++
result, err := s.registry.Call(ctx, allow, call.Name, call.Arguments, caller)
step := AssistantStep{Tool: call.Name, Outcome: tools.OutcomeOK, Rows: result.Count, Scope: result.Scope}
if err != nil {
step.Outcome = tools.OutcomeRefused
step.Detail = err.Error()
}
answer.Used = append(answer.Used, step)
if result.Source != "" && !seenSource[result.Source] {
seenSource[result.Source] = true
answer.Sources = append(answer.Sources, result.Source)
}
messages = append(messages, utils.Message{
Role: utils.RoleTool,
ToolCallID: call.ID,
Name: call.Name,
Content: toolMessage(result, err),
})
}
}
// Out of steps with the model still working. Ask once for what it has
// rather than returning nothing: a partial answer beats a blank panel, and
// `Incomplete` is what stops it being passed off as the whole story.
answer.Incomplete = true
messages = append(messages, utils.Message{
Role: utils.RoleUser,
Content: "Answer now with what you already have, and say plainly that you ran out of steps before finishing.",
})
reply, err := s.chat.Complete(ctx, utils.ChatRequest{Tier: agent.Tier, Messages: messages})
if err != nil {
return answer, err
}
answer.Reply = strings.TrimSpace(reply.Content)
return answer, nil
}
// toolMessage is what the model is told a tool returned.
//
// A refusal is reported as text, not as a failure: a model told "that tool
// needs a tenant" can explain it to the person, where a model handed nothing
// says "something went wrong".
//
// The rows go back as JSON because that is what the model reads most reliably,
// and `note` rides alongside them rather than inside, so an instruction about
// truncation cannot be mistaken for data.
func toolMessage(result tools.Result, err error) string {
if err != nil {
return "Refused: " + err.Error()
}
payload := map[string]any{
"rows": result.Rows,
"count": result.Count,
}
if result.Scope != "" {
payload["covers"] = result.Scope
}
if result.Truncated {
payload["truncated"] = true
}
if result.Note != "" {
payload["note"] = result.Note
}
encoded, marshalErr := json.Marshal(payload)
if marshalErr != nil {
return fmt.Sprintf("Refused: the result could not be encoded: %v", marshalErr)
}
return string(encoded)
}
// assistantTimeout bounds one question end to end.
//
// Generous, because a deep question makes several round trips, and short enough
// that a wedged provider does not hold a console connection open all afternoon.
const assistantTimeout = 90 * time.Second
// WithTimeout is the bound the HTTP layer applies. Here rather than in the
// controller so every caller of Ask — HTTP today, MCP later — gets the same one.
func WithTimeout(ctx context.Context) (context.Context, context.CancelFunc) {
return context.WithTimeout(ctx, assistantTimeout)
}

377
services/assistant_test.go Normal file
View File

@@ -0,0 +1,377 @@
package services
import (
"context"
"errors"
"strings"
"testing"
"nearle/services/tools"
"nearle/utils"
)
// The loop, against a scripted model.
//
// A fake rather than a live provider on purpose: these are about what the loop
// REFUSES to let a model do, and that has to hold for any model, including one
// behaving badly. A test that needed a network would only ever prove what one
// model happened to do that afternoon.
type scriptedChat struct {
replies []utils.ChatReply
err error
// Every request the loop made, so the tests can inspect what the model was
// actually shown — the tool list especially.
seen []utils.ChatRequest
}
func (s *scriptedChat) ModelFor(string) string { return "scripted-model" }
func (s *scriptedChat) Complete(_ context.Context, req utils.ChatRequest) (utils.ChatReply, error) {
s.seen = append(s.seen, req)
if s.err != nil {
return utils.ChatReply{}, s.err
}
if len(s.replies) == 0 {
return utils.ChatReply{Content: "nothing further", Model: "scripted-model"}, nil
}
reply := s.replies[0]
s.replies = s.replies[1:]
if reply.Model == "" {
reply.Model = "scripted-model"
}
return reply, nil
}
// recordingTool answers with fixed rows and remembers the caller it ran for.
func recordingTool(name string, result tools.Result, seen *tools.Caller) tools.Tool {
return tools.Tool{
Name: name,
Description: "a tool, for testing",
Scope: tools.ScopeRead,
Schema: tools.Schema{},
Handler: func(_ context.Context, req tools.Request) (tools.Result, error) {
if seen != nil {
*seen = req.Caller
}
return result, nil
},
}
}
func newAssistant(t *testing.T, chat utils.Chat, toolset ...tools.Tool) AssistantService {
t.Helper()
registry := tools.New(nil)
names := make([]string, 0, len(toolset))
for _, tool := range toolset {
if err := registry.Register(tool); err != nil {
t.Fatalf("registering: %v", err)
}
names = append(names, tool.Name)
}
agents := map[string]Agent{"orders": {
Name: "orders", Tier: utils.TierBalanced, System: "be brief",
Tools: names, MaxSteps: 4, MaxToolCalls: 6,
}}
return NewAssistantService(registry, chat, agents)
}
var merchant = tools.Caller{Userid: 904, Tenantid: 1147}
func toolCall(id, name string, args map[string]any) utils.ChatReply {
return utils.ChatReply{ToolCalls: []utils.ToolCall{{ID: id, Name: name, Arguments: args}}}
}
/* ── The happy path ────────────────────────────────────────────────────── */
func TestATypedQuestionRoutesToAToolAndComesBackAsWords(t *testing.T) {
// Phase 2's whole point.
chat := &scriptedChat{replies: []utils.ChatReply{
toolCall("c1", "stuck", nil),
{Content: "Two jobs have been waiting over half an hour."},
}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{
Rows: []int{1, 2}, Count: 2, Scope: "all branches", Source: "/admin/dispatch",
}, nil))
answer, err := assistant.Ask(context.Background(), "orders", "what is stuck?", merchant)
if err != nil {
t.Fatalf("asking: %v", err)
}
if answer.Reply == "" {
t.Fatal("no answer came back")
}
if len(answer.Used) != 1 || answer.Used[0].Tool != "stuck" {
t.Fatalf("the tool run is not reported: %+v", answer.Used)
}
if answer.Used[0].Rows != 2 {
t.Fatalf("row count lost: %+v", answer.Used[0])
}
if len(answer.Sources) != 1 || answer.Sources[0] != "/admin/dispatch" {
t.Fatalf("the answer links to nothing: %+v", answer.Sources)
}
if answer.Incomplete {
t.Fatal("a finished answer was flagged incomplete")
}
}
func TestAQuestionNeedingNoToolIsAnsweredDirectly(t *testing.T) {
// "auto", not "required" — forcing a call makes a model invent one.
chat := &scriptedChat{replies: []utils.ChatReply{{Content: "Deliveries are jobs given to a rider."}}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
answer, err := assistant.Ask(context.Background(), "orders", "what is a delivery?", merchant)
if err != nil {
t.Fatalf("asking: %v", err)
}
if len(answer.Used) != 0 {
t.Fatalf("a tool ran for a question that needed none: %+v", answer.Used)
}
}
/* ── What the model does not get to choose ─────────────────────────────── */
func TestTheCallerComesFromTheSessionNotTheModel(t *testing.T) {
// The single most important property. The model picks the tool; it has no
// say in whose data is read.
var seen tools.Caller
chat := &scriptedChat{replies: []utils.ChatReply{
toolCall("c1", "stuck", map[string]any{"tenantid": 916}),
{Content: "done"},
}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, &seen))
if _, err := assistant.Ask(context.Background(), "orders", "orders for tenant 916", merchant); err != nil {
t.Fatalf("asking: %v", err)
}
if seen.Tenantid != 1147 {
t.Fatalf("the tool ran for tenant %d", seen.Tenantid)
}
}
func TestAToolTheAgentDoesNotHaveIsRefusedNotRun(t *testing.T) {
var ran bool
chat := &scriptedChat{replies: []utils.ChatReply{
toolCall("c1", "secret", nil),
{Content: "I could not look that up."},
}}
registry := tools.New(nil)
_ = registry.Register(recordingTool("stuck", tools.Result{}, nil))
_ = registry.Register(tools.Tool{
Name: "secret", Description: "not for this agent", Scope: tools.ScopeRead,
Handler: func(context.Context, tools.Request) (tools.Result, error) {
ran = true
return tools.Result{}, nil
},
})
assistant := NewAssistantService(registry, chat, map[string]Agent{"orders": {
Name: "orders", Tier: utils.TierBalanced, Tools: []string{"stuck"}, MaxSteps: 4, MaxToolCalls: 6,
}})
answer, err := assistant.Ask(context.Background(), "orders", "tell me a secret", merchant)
if err != nil {
t.Fatalf("asking: %v", err)
}
if ran {
t.Fatal("a tool off the allow-list ran")
}
if len(answer.Used) != 1 || answer.Used[0].Outcome != tools.OutcomeRefused {
t.Fatalf("the refusal is not reported: %+v", answer.Used)
}
}
func TestTheModelIsOnlyShownToolsItMayUse(t *testing.T) {
chat := &scriptedChat{replies: []utils.ChatReply{{Content: "done"}}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
if _, err := assistant.Ask(context.Background(), "orders", "hello", merchant); err != nil {
t.Fatalf("asking: %v", err)
}
if len(chat.seen) == 0 || len(chat.seen[0].Tools) != 1 {
t.Fatalf("the model was shown the wrong tools: %+v", chat.seen)
}
}
/* ── A refusal is a message, not a crash ───────────────────────────────── */
func TestARefusedToolIsExplainedToTheModel(t *testing.T) {
// A model told "that tool needs a tenant" can explain it; a model handed
// nothing says "something went wrong".
chat := &scriptedChat{replies: []utils.ChatReply{
toolCall("c1", "broken", nil),
{Content: "I could not read that."},
}}
registry := tools.New(nil)
_ = registry.Register(tools.Tool{
Name: "broken", Description: "fails", Scope: tools.ScopeRead,
Handler: func(context.Context, tools.Request) (tools.Result, error) {
return tools.Result{}, errors.New("the database is down")
},
})
assistant := NewAssistantService(registry, chat, map[string]Agent{"orders": {
Name: "orders", Tier: utils.TierBalanced, Tools: []string{"broken"}, MaxSteps: 4, MaxToolCalls: 6,
}})
answer, err := assistant.Ask(context.Background(), "orders", "what is stuck?", merchant)
if err != nil {
t.Fatalf("a failing tool broke the whole question: %v", err)
}
if answer.Reply == "" {
t.Fatal("no answer after a tool failure")
}
// The model must have been TOLD, not just had the call dropped.
var told bool
for _, req := range chat.seen {
for _, m := range req.Messages {
if m.Role == utils.RoleTool && strings.Contains(m.Content, "database is down") {
told = true
}
}
}
if !told {
t.Fatal("the model was never told why the tool failed")
}
}
/* ── Limits are arithmetic, not a polite request ───────────────────────── */
func TestAModelThatKeepsCallingToolsIsStopped(t *testing.T) {
// Asking a model to stop is a request. This is the thing that actually
// stops it.
chat := &scriptedChat{replies: []utils.ChatReply{
toolCall("c1", "stuck", nil),
toolCall("c2", "stuck", nil),
toolCall("c3", "stuck", nil),
toolCall("c4", "stuck", nil),
toolCall("c5", "stuck", nil),
toolCall("c6", "stuck", nil),
toolCall("c7", "stuck", nil),
}}
registry := tools.New(nil)
_ = registry.Register(recordingTool("stuck", tools.Result{}, nil))
assistant := NewAssistantService(registry, chat, map[string]Agent{"orders": {
Name: "orders", Tier: utils.TierBalanced, Tools: []string{"stuck"}, MaxSteps: 3, MaxToolCalls: 2,
}})
answer, err := assistant.Ask(context.Background(), "orders", "loop forever", merchant)
if err != nil {
t.Fatalf("asking: %v", err)
}
if !answer.Incomplete {
t.Fatal("the loop ran out of room and did not say so")
}
ran := 0
for _, step := range answer.Used {
if step.Outcome == tools.OutcomeOK {
ran++
}
}
if ran > 2 {
t.Fatalf("the tool-call cap was ignored: %d ran", ran)
}
}
func TestATruncatedReplyIsFlagged(t *testing.T) {
// `length` means the provider cut the answer off mid-sentence, and that
// reads exactly like a complete answer unless it is flagged.
chat := &scriptedChat{replies: []utils.ChatReply{{Content: "The branches that are under", StopReason: "length"}}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
answer, err := assistant.Ask(context.Background(), "orders", "compare my branches", merchant)
if err != nil {
t.Fatalf("asking: %v", err)
}
if !answer.Incomplete {
t.Fatal("a reply cut off mid-sentence was reported as complete")
}
}
/* ── Degrading, and refusing ───────────────────────────────────────────── */
func TestWithNoModelConfiguredTheAssistantSaysSo(t *testing.T) {
assistant := newAssistant(t, nil, recordingTool("stuck", tools.Result{}, nil))
if assistant.Available() {
t.Fatal("reported available with no model")
}
_, err := assistant.Ask(context.Background(), "orders", "what is stuck?", merchant)
if !errors.Is(err, utils.ErrChatNotConfigured) {
t.Fatalf("expected a clear 'not configured', got: %v", err)
}
}
func TestAnUnknownAgentIsRefused(t *testing.T) {
chat := &scriptedChat{}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
if _, err := assistant.Ask(context.Background(), "invented", "hello", merchant); err == nil {
t.Fatal("a question was answered by an agent that does not exist")
}
}
func TestAnEmptyQuestionIsRefusedBeforeTheModelIsPaid(t *testing.T) {
chat := &scriptedChat{}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
if _, err := assistant.Ask(context.Background(), "orders", " ", merchant); err == nil {
t.Fatal("an empty question reached the model")
}
if len(chat.seen) != 0 {
t.Fatal("the model was called for an empty question")
}
}
func TestAPastedSpreadsheetIsNotAQuestion(t *testing.T) {
chat := &scriptedChat{}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
if _, err := assistant.Ask(context.Background(), "orders", strings.Repeat("x", maxQuestion+1), merchant); err == nil {
t.Fatal("an oversized question was sent to the model")
}
if len(chat.seen) != 0 {
t.Fatal("the model was called for an oversized question")
}
}
/* ── What the console is told ──────────────────────────────────────────── */
func TestTheAnswerNamesTheModelThatProducedIt(t *testing.T) {
// An answer nobody can attribute to a model cannot be reproduced when it
// turns out to be wrong.
chat := &scriptedChat{replies: []utils.ChatReply{{Content: "done", Model: "some-model-v2"}}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{}, nil))
answer, _ := assistant.Ask(context.Background(), "orders", "hello", merchant)
if answer.Model != "some-model-v2" {
t.Fatalf("the model is not named: %q", answer.Model)
}
}
func TestTruncationReachesTheModelInWords(t *testing.T) {
// An empty result and a capped one look identical to a model, and it will
// describe both as "none".
chat := &scriptedChat{replies: []utils.ChatReply{
toolCall("c1", "stuck", nil),
{Content: "done"},
}}
assistant := newAssistant(t, chat, recordingTool("stuck", tools.Result{
Rows: []int{1}, Count: 60, Truncated: true, Note: "60 jobs are waiting; the 50 longest are listed.",
}, nil))
if _, err := assistant.Ask(context.Background(), "orders", "what is stuck?", merchant); err != nil {
t.Fatalf("asking: %v", err)
}
var told bool
for _, req := range chat.seen {
for _, m := range req.Messages {
if m.Role == utils.RoleTool && strings.Contains(m.Content, "60 jobs are waiting") {
told = true
}
}
}
if !told {
t.Fatal("the model was not told the list was capped")
}
}

285
utils/chat.go Normal file
View File

@@ -0,0 +1,285 @@
package utils
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"time"
"nearle/config"
)
// The model gateway.
//
// A sibling of `embedding.go`, and deliberately the same shape: one small
// interface, a provider switch behind a factory, the shared `postJSON`, and a
// `Model()` so what produced an answer can be recorded. No framework — the
// backend talks to models over plain HTTP already, and the OpenAI chat
// completions format is what Groq, Ollama, vLLM, LM Studio, Together and Azure
// all accept, so one client reaches all of them.
//
// ── Agents name a tier, never a model ───────────────────────────────────────
//
// A tier is a promise about how much thinking a question deserves. Wiring a
// model name into an agent means editing every agent to change provider, and
// means an agent's YAML going stale the day a model is retired. The mapping
// lives in config and nowhere else.
//
// ── Degrades rather than fails ──────────────────────────────────────────────
//
// `ErrChatNotConfigured` is what a deployment with no provider gets, and the
// assistant answers "typed questions are not switched on here" instead of a
// 500. The tools keep working either way: they are ordinary Go functions, and
// only the part that turns a sentence into a tool call is missing.
// Tier is how much thinking a question deserves.
const (
// TierFast is a chip answer: one tool, no reasoning.
TierFast = "fast"
// TierBalanced is the default, and what most typed questions get.
TierBalanced = "balanced"
// TierDeep is multi-step reasoning across several tools.
TierDeep = "deep"
)
// Roles in a conversation, as the wire format spells them.
const (
RoleSystem = "system"
RoleUser = "user"
RoleAssistant = "assistant"
RoleTool = "tool"
)
// ErrChatNotConfigured is what callers see when no provider is set.
var ErrChatNotConfigured = errors.New("no assistant model is configured")
// ToolCall is the model asking for a tool to be run.
//
// `Arguments` is decoded here rather than passed along as the raw string the
// wire carries, so a model that emits malformed JSON is caught at the edge —
// one error, in one place, instead of every caller parsing it again.
type ToolCall struct {
ID string
Name string
Arguments map[string]any
}
// Message is one turn.
type Message struct {
Role string
Content string
// Set on an assistant turn that asked for tools.
ToolCalls []ToolCall
// Set on a tool turn, naming the call it answers. Without it the model
// cannot tell which result belongs to which request when it asked for two.
ToolCallID string
// The tool's name, which some providers want on the tool turn as well.
Name string
}
// ChatRequest is one round trip.
type ChatRequest struct {
Tier string
Messages []Message
// Tool definitions as the registry describes them: name, description,
// input_schema. Converted to the provider's shape here so nothing above
// this file knows what that shape is.
Tools []map[string]any
MaxTokens int
}
// ChatReply is what came back.
type ChatReply struct {
Content string
ToolCalls []ToolCall
// Which model actually answered. Recorded on every audit row: an answer
// nobody can attribute to a model cannot be reproduced when it is wrong.
Model string
// The provider's own word for why it stopped — `stop`, `tool_calls`,
// `length`. `length` is the one that matters: a truncated answer reads as a
// complete one unless somebody looks.
StopReason string
}
// Chat is the whole interface. One method, like the embedder.
type Chat interface {
Complete(ctx context.Context, req ChatRequest) (ChatReply, error)
// ModelFor names the model a tier resolves to, for the audit trail.
ModelFor(tier string) string
}
const chatTimeout = 60 * time.Second
// NewChat builds the gateway, or nil when none is configured.
//
// Nil rather than an error for the unconfigured case, matching `NewEmbedder`:
// "no model here" is a deployment choice, not a fault, and the caller checks
// for nil exactly as it does for the embedder.
func NewChat(cfg config.AssistantConfig) (Chat, error) {
if !cfg.Enabled() {
return nil, nil
}
switch cfg.Provider {
case "openai", "groq", "ollama", "together", "compatible":
base := strings.TrimRight(cfg.BaseURL, "/")
if base == "" {
base = "https://api.openai.com/v1"
}
return &openAIChat{cfg: cfg, base: base, client: &http.Client{Timeout: chatTimeout}}, nil
}
return nil, fmt.Errorf("assistant provider %q is not supported", cfg.Provider)
}
// ── OpenAI-compatible chat completions ──────────────────────────────────────
type openAIChat struct {
cfg config.AssistantConfig
base string
client *http.Client
}
func (c *openAIChat) ModelFor(tier string) string { return c.cfg.ModelFor(tier) }
// wire types, kept unexported: nothing above this file should know that a tool
// call arrives with its arguments as a string.
type wireToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
Function struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
} `json:"function"`
}
type wireMessage struct {
Role string `json:"role"`
Content string `json:"content,omitempty"`
ToolCalls []wireToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
Name string `json:"name,omitempty"`
}
func (c *openAIChat) Complete(ctx context.Context, req ChatRequest) (ChatReply, error) {
model := c.cfg.ModelFor(req.Tier)
if model == "" {
return ChatReply{}, ErrChatNotConfigured
}
messages := make([]wireMessage, 0, len(req.Messages))
for _, m := range req.Messages {
wire := wireMessage{Role: m.Role, Content: m.Content, ToolCallID: m.ToolCallID, Name: m.Name}
for _, call := range m.ToolCalls {
raw, err := json.Marshal(call.Arguments)
if err != nil {
return ChatReply{}, fmt.Errorf("assistant: encoding a tool call: %w", err)
}
wt := wireToolCall{ID: call.ID, Type: "function"}
wt.Function.Name = call.Name
wt.Function.Arguments = string(raw)
wire.ToolCalls = append(wire.ToolCalls, wt)
}
messages = append(messages, wire)
}
body := map[string]any{
"model": model,
"messages": messages,
}
if req.MaxTokens > 0 {
body["max_tokens"] = req.MaxTokens
}
if len(req.Tools) > 0 {
body["tools"] = toolsForWire(req.Tools)
// "auto", never "required": some questions are answered from what is
// already in the conversation, and forcing a call makes the model
// invent one to satisfy the demand.
body["tool_choice"] = "auto"
}
var out struct {
Choices []struct {
Message wireMessage `json:"message"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Model string `json:"model"`
Error *struct {
Message string `json:"message"`
} `json:"error"`
}
auth := ""
if c.cfg.APIKey != "" {
auth = "Bearer " + c.cfg.APIKey
}
if err := postJSON(ctx, c.client, c.base+"/chat/completions", auth, body, &out); err != nil {
return ChatReply{}, err
}
if out.Error != nil {
return ChatReply{}, fmt.Errorf("assistant: %s", out.Error.Message)
}
if len(out.Choices) == 0 {
return ChatReply{}, errors.New("assistant: the model returned no choices")
}
choice := out.Choices[0]
reply := ChatReply{
Content: choice.Message.Content,
Model: firstNonEmpty(out.Model, model),
StopReason: choice.FinishReason,
}
for _, call := range choice.Message.ToolCalls {
args := map[string]any{}
// An empty argument string is a call with no arguments, which is
// ordinary — `{}` and `""` both mean the same thing here.
if trimmed := strings.TrimSpace(call.Function.Arguments); trimmed != "" {
if err := json.Unmarshal([]byte(trimmed), &args); err != nil {
// Caught at the edge rather than passed on. A model emitting
// malformed JSON is a fact about this round trip, and the loop
// can retry or give up — but nothing downstream should have to
// parse it a second time.
return ChatReply{}, fmt.Errorf("assistant: the model sent unreadable arguments for %s: %w", call.Function.Name, err)
}
}
reply.ToolCalls = append(reply.ToolCalls, ToolCall{
ID: call.ID,
Name: call.Function.Name,
Arguments: args,
})
}
return reply, nil
}
// toolsForWire converts the registry's description into the provider's shape.
//
// The registry speaks `{name, description, input_schema}` because that is what
// MCP uses and what reads clearly. This is the one place that knows OpenAI
// wants it wrapped in a `function` object — so a second provider with a
// different shape is a change here and nowhere else.
func toolsForWire(defs []map[string]any) []map[string]any {
out := make([]map[string]any, 0, len(defs))
for _, def := range defs {
out = append(out, map[string]any{
"type": "function",
"function": map[string]any{
"name": def["name"],
"description": def["description"],
"parameters": def["input_schema"],
},
})
}
return out
}
func firstNonEmpty(values ...string) string {
for _, value := range values {
if value != "" {
return value
}
}
return ""
}