Enterprise AI Chatbot & Agent Orchestrator

Deep Dive: Agent Architecture & Tool Orchestration

Production Status (The Real Talk): Architected, built, and thoroughly battle-tested in staging. However, it is currently parked in legendary corporate deployment limbo due to classic enterprise office politics & roadmap reshuffling (yes, the code works; no, the steering committee hasn't agreed on who clicks "Deploy" or whose quarterly OKR gets the glory). Status: Fully functional Go orchestrator, awaiting bureaucratic clearance. A generalized open-source template and architectural blueprint are shared below!

Project: Enterprise AI Assistant and Tool Layer for Multi-Service Platforms Role: Backend / platform engineering — orchestrator, tool registry, security gates, automation hooks Stack: Go, PostgreSQL, OpenAPI-derived tools, multi-provider LLM adapter, OAuth/MCP (read path), scheduled routines

In enterprise operations, business and operational teams need answers immediately—whether reconciling yesterday’s accounts, verifying entity statuses, or checking operational records across distributed systems. A single “chat API” cannot answer those questions safely: data lives behind several REST microservices, permissions differ by role, and “today” on the wall clock is not today in the accounting books.

This project is an agent orchestrator that sits between staff (and, on a separate surface, external AI clients) and downstream services. It does not own ledgers or entity records directly. It plans reads, executes only allowlisted GETs under the caller’s credentials, forces tenant scope in code, and turns JSON into structured answers—with a fallback path when the model is unavailable.

Multi-Service Topology

The enterprise backend is composed of discrete services:

[ Staff question in plain text (+ active tenant) ]
                        ↓
[ Agent Orchestrator (Go) — single binary ]
   ├── Session gate: token valid + user allowed on tenant
   ├── Business-date resolution (reconciliation anchor, not server clock)
   ├── Intent / planner path OR keyword handler fallback
   ├── Tool allowlist (OpenAPI-generated reads, admin/export/internal stripped)
   ├── Tenant injection on every downstream call (model cannot widen scope)
   └── Parallel GET execution → user JWT forwarded as downstream identity header
                        ↓
[ Internal REST APIs — RBAC enforced per route ]
                        ↓
[ Presenter: short text + typed cards (table, chart, stats) ]
                        ↓
[ Verified answer + persisted turn & tool audit ]

Parallel tracks (same security philosophy, different front door):
   • Scheduled / conditional routines — grant + short-lived delegated token at run time
   • MCP read-only tool server — OAuth bearer, no in-house planner, same read filter

Design choice: Treat the LLM as a planner and presenter, not as the system of record. Numbers come from API responses; scope and dates are server invariants.

Critical Problem: “Today,” Time Zones, and Missing Query Params

When queries used relative language—_“yesterday,” “last week,” “which accounts renewed today”_—naive LLM-orchestrated API calls failed in predictable ways:

• •The model assumed UTC or server local time, not the organization’s operational business date.
• •OpenAPI tools often require paired date fields or explicit filters; omitted params yielded empty reports or 400s the model could not recover from.
• •Letting the model freely set tenantId would violate one-tenant-at-a-time isolation rules.

Resolution: A three-layer contract—

Illustrative orchestration (portfolio pseudocode—not production):

package orchestrator

import (
	"context"
	"time"
)

type ReadRequest struct {
	TenantID      string
	UserToken     string
	BusinessDate  time.Time // reconciliation "today"
	LocationTZ    *time.Location
	PlannedReads  []PlannedToolCall
}

func (e *Engine) ExecuteReads(ctx context.Context, req ReadRequest) ([]ReadResult, error) {
	for i := range req.PlannedReads {
		call := &req.PlannedReads[i]
		if !e.allowlist.IsPermittedRead(call.ToolName) {
			return nil, ErrToolNotInPolicy
		}
		// Invariant: scope and dates are server-owned
		call.Args = e.forceTenantScope(call.Args, req.TenantID)
		call.Args = e.fillRequiredDates(call.Args, req.BusinessDate, req.LocationTZ)
	}
	return e.runParallelGETs(ctx, req.UserToken, req.PlannedReads)
}

Outcome: Relative questions map to the same windows operational teams expect on reports, without trusting the model to guess ISO boundaries or tenant ids.

Critical Problem: Security Without Prompt Religion

Interactive chat benefits from prompts (“refuse writes,” “mask card numbers,” narrow scope). External MCP clients bring their own models and prompts—so MCP and automation paths rely only on:

• •OAuth consent with an explicit tenant on the grant
• •Opaque access token → token exchange → ~120s delegated user JWT used once per downstream call
• •Same read allowlist as chat; no write tools in v1 MCP
• •Separate audit rows attributing user, client, grant, tool, and outcome

Lesson for the portfolio: Split “AI product surface” (planner, cards, confirmations) from “AI tool surface” (protocol + allowlist + injection). Shared filters; not shared trust in instructions.

Tool Orchestration Model

Hundreds of generated tools are registered from sibling OpenAPI specs. At runtime:

Registry (all generated tools)
        ↓ filter at startup
Allowlisted GET reads only (~300+ tools, minus admin/internal/export)
        ↓ planner or MCP tools/list
Subset chosen per turn OR full list exposed to MCP client
        ↓ dispatcher
Forced tenantId + date/param completion → parallel execution
        ↓
Presenter (chat) OR passthrough JSON (MCP)

Multi-step operational workflows (approvals, entity assignment, status changes) use an agent loop: reads until context is sufficient, then in-chat confirmation flows or schema-driven write forms—writes never bypass the user’s token and downstream RBAC. Background routines reuse the same engine with a reviewed definition, grant-based tokens, and an allowlisted autonomous write path where product requires it.

What I Learned

• •Enterprise agents are state machines with guardrails. The LLM proposes tool names and parameters; invariants (tenant scope, business date, allowlist, compliance patterns) belong in Go.
• •OpenAPI as the tool contract scales better than hand-written integrations—if paired with a strict filter and CI regen from service specs.
• •One permission system. The orchestrator does not maintain a shadow RBAC matrix; 403 from upstream becomes a denied or error card—simpler audits, sharper UX when roles are wrong.
• •Two HTTP personalities in one process (chat streaming vs MCP streams) avoid timeout and trust-boundary mistakes.
• •Automation credentials are grants, not service accounts. Background runs exchange revocable secrets for short-lived tokens with live permission re-check—same pattern as MCP, different UX.

What’s Next (and what shipped adjacent to this work)

Architectural Highlights

• •Built a Go orchestrator between a Next.js workspace panel and multiple microservices, with streaming SSE and card-based answers.
• •Solved business-date and timezone anchoring so relative user language maps to correct report windows.
• •Designed MCP + OAuth token exchange so external AI tools inherit the same read allowlist and RBAC as the in-app chat agent—without forwarding long-lived browser JWTs.
• •Introduced routines as validated JSON pipelines with grant-based background execution and audit parity with interactive chat.

Architecture Flow (Mermaid)

Related Architecture Deep Dives

• •Inside an Enterprise AI Orchestrator: Multi-Service LLMs, Automation, and Zero-Trust Access
• •Building a Read-Only MCP Tool Server: Zero-Trust Enterprise Data Access for Claude, Cursor, and Beyond