~kris/9p

llm9p

ref: 42c2e6958db4e870f21ce0b60b7522975cd8757f llm9p/internal d---------
42c2e695 — pdfinn 5 months ago
fix: extract tool args and fix tool result history for OpenAI path

Mirror of infernode llmclient fixes for the Go llm9p server:

1. Call extractToolArgs on OpenAI tool_call arguments before building
   TOOL: lines. The Anthropic path already did this; the OpenAI path
   passed raw {"args":"value"} JSON through to the agent layer.

2. Handle user-role messages with StructuredContent (tool results)
   in buildChatMessages. Previously these were emitted as plain
   {"role":"user","content":"tool results submitted"}, losing the
   actual results and breaking role alternation. Now expanded into
   individual {"role":"tool"} messages via rebuildToolResultMessages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fd213ef6 — pdfinn 5 months ago
feat(openai): fallback text tool-call parser for non-Anthropic models

When Ollama/Qwen models generate tool calls as text instead of structured
API responses, parse <function=>, <tool_call>, and <|tool_call|> formats
from the content and promote them to proper STOP:tool_use/TOOL: wire
format. Validates tool names against definitions, rejects hallucinated tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1dc34eeb — pdfinn 6 months ago
feat(llm): add thinking control for OpenAI/Ollama backend

Implements unified thinking interface for the OpenAI-compatible backend
(Ollama, gpt-oss, etc.) matching the existing Claude token-budget semantics:
  0           → think: false  (disabled)
  1–10000     → think: true,  think_level: "low"
  10001–20000 → think: true,  think_level: "medium"
  20001+ / -1 → think: true,  think_level: "high"

Bypasses go-openai library for AskWithRequest to send Ollama-specific
options field; implements SSE parsing with bufio.Scanner for the streaming
path and direct JSON decode for the blocking path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
b96d9137 — pdfinn 6 months ago
fix(llm): seed session model from backend and persist sessions across connections

Two bugs found during gpt-oss/Ollama operational test:

1. DefaultSessionDefaults() hardcoded the Claude model ID, causing
   llm9p to send "claude-sonnet-4-5-20250929" to Ollama which has no
   such model. NewSessionManager now seeds defaults.Model from
   apiClient.Model() so the session inherits the backend's configured
   model (e.g. "gpt-oss:20b").

2. Sessions started with refs=0 and were auto-deleted the moment the
   first 9P fid using them was clunked. The plan9port 9p CLI tool
   opens a new connection per command, so a session created by
   "9p read new" was gone before the next "9p write N/ask" could use
   it. Sessions now start with refs=1 (the session holds a reference
   to itself) and persist until explicitly closed via "ctl close".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
c94202ec — pdfinn 6 months ago
fix(llm): harden tool-use history to prevent cascading failures

- Record tool_results in history before the API call so orphaned
  tool_use blocks can't corrupt the session if the call fails
- Detect and auto-recover from tool_use/tool_result history mismatches
  by resetting the session and retrying
- Add synthetic assistant error message on AskWithToolResults failure
  to keep role-alternation valid
- Replace manual JSON escaping in buildToolResultsJSON with json.Marshal
- Update mock AskWithRequest signatures to return AskResponse

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
11a3967d — P. D. Finn KD9WEH 6 months ago
Merge pull request #1 from NERVsystems/claude/local-llm-feasibility-gVLhq

Add OpenAI-compatible local LLM backend support
2eac753a — Claude 6 months ago
Add OpenAI-compatible backend for local LLM support (GPT-OSS)

Implement OpenAIClient backend that speaks the OpenAI Chat Completions
API (/v1/chat/completions), enabling llm9p to work with any local model
server: Ollama, vLLM, llama-server, LocalAI, or LM Studio.

Primary target is GPT-OSS (OpenAI's open-weight MoE models), but any
model served via these platforms works. The backend supports:
- Blocking and streaming chat completions
- Tool/function calling with STOP:/TOOL: formatting
- Token counting from API usage (with estimation fallback)
- Conversation history, system prompts, temperature control
- Stateless AskWithRequest for session isolation

Usage:
  ./llm9p -backend openai -openai-url http://localhost:11434/v1 -model gpt-oss:20b

Also fixes pre-existing stale mock backends in test files (AskWithRequest
signature was out of date with the Backend interface).

https://claude.ai/code/session_017qVVZUUhfCCvNkMYmXDZAa
822b318b — pdfinn 6 months ago
fix(llmfs): clean up sessions on disconnect to prevent memory leak

Each spawned subagent creates an LLM session via /n/llm/new, but sessions
were never automatically freed: BaseFile.Close() was a no-op, and connection
drop handling only removed the client from the map without touching sessions.
Repeated spawn calls caused sessions to accumulate indefinitely, consuming
memory for full conversation histories.

Changes:
- Add refs int32 (atomic) to Session struct
- Add IncRef(id) / DecRef(id) to SessionManager; DecRef calls Close when
  refs reach zero, freeing the session and all its conversation history
- Add sessionRefFile / sessionRefDir wrappers (session_ref.go) that call
  IncRef on Open and DecRef on Close (once.Do guards against double-decrement)
- SessionsDir.Lookup wraps the returned SessionDir in sessionRefDir, so
  child file lookups via sessionRefDir.Lookup also yield sessionRefFile objects
- Fix handleConn disconnect path to call file.Close() on all unclosed fids
  before removing the client, ensuring sessions are freed on network drop

Typical flow: subagent exits → NEWFD closes all FDs → 9P connection drops →
handleConn deferred cleanup calls Close on remaining fids → sessionRefFile.Close
→ sm.DecRef → refs=0 → sm.Close(id) → session deleted from map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4191d4e7 — pdfinn 6 months ago
fix(llmfs): correct model alias IDs to match actual Anthropic API

Previous commit used claude-sonnet-4-6 and claude-opus-4-6 which don't
exist in the Anthropic API. IDs verified against anthropic-sdk-go v1.19.0:

  haiku  → claude-haiku-4-5-20251001    (unchanged, was correct)
  sonnet → claude-sonnet-4-5-20250929   (was claude-sonnet-4-6, 404)
  opus   → claude-opus-4-5-20251101     (was claude-opus-4-6, 404)

Also fix default from claude-sonnet-4-6 (invalid) to claude-sonnet-4-5-20250929.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
27f32368 — pdfinn 6 months ago
fix(llmfs): expand model aliases and update default model ID

Add a modelAliases map in session_settings.go so that short names
written to /n/llm/N/model are expanded to full Anthropic model IDs:

  haiku  → claude-haiku-4-5-20251001
  sonnet → claude-sonnet-4-6
  opus   → claude-opus-4-6

This fixes spawn subagents which write short names like "haiku" to the
model file — the Anthropic API rejects bare aliases as invalid model IDs.

Also update the default model from the stale claude-sonnet-4-20250514
to claude-sonnet-4-6 to match current API availability.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bad7a7b7 — pdfinn 6 months ago
feat(llmfs): async Write + per-session streaming for live token delivery

Enable lucibridge to stream LLM tokens into the Lucifer conversation
zone as they are generated, without waiting for the full response.

Changes:
- session.go: add BeginGeneration/EndGeneration/SendChunk/GetStreamCh/
  WaitDone methods on Session; streamCh (cap 256) carries raw text chunks
  during generation; doneCh signals completion; EndGeneration closes
  channels but does NOT nil streamCh (late readers still see closed chan)
- session_ask.go: Write() is now async — calls BeginGeneration(), spawns
  goroutine, returns immediately; Read() calls WaitDone() before accessing
  LastResponse so pread blocks until generation completes
- session_stream.go (new): /n/llm/N/stream file; Read() blocks on <-ch
  returning each text chunk as it arrives; returns EOF when generation
  is done or no generation is active (channel nil or closed)
- session_dir.go: register stream file in Children() and Lookup()
- client.go: AskWithRequest() branches on req.StreamFunc != nil to use
  SSE Messages.NewStreaming() path; text_delta events forwarded to
  StreamFunc; session.Ask() sets StreamFunc=session.SendChunk when
  GetStreamCh() is non-nil
- server.go: add detailed walk debug logging (names, types, failures)
  controlled by existing -debug flag

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
35c20f3e — pdfinn 6 months ago
fix(client): replace empty text content block with placeholder

When the LLM returns an end_turn with no text content after a tool
call, the session stores Message{Content:"", StructuredContent:""}.
On the next user turn, buildMessageParam() hit the plain-text branch
and called NewTextBlock(""), which the Anthropic API rejects with:
  400: "messages: text content blocks must be non-empty"

Replace empty Content with "..." before building the text block.
This preserves the alternating user/assistant message structure
required by the API without introducing invalid empty blocks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
0afa911a — pdfinn 6 months ago
feat(llmfs): native Anthropic tool_use protocol support

Add structured tool_use protocol to llm9p, enabling Veltro to use
Claude's native JSON tool invocation instead of text-based parsing.

New types (backend.go):
- AskResponse: carries Response, StructuredJSON, and Tokens — replaces
  the old (string, int, error) return from AskWithRequest
- ToolDef: tool definition passed to Anthropic tools API
- ToolResult: tool execution result for submission to the LLM
- Backend.AskWithRequest() now returns (AskResponse, error)

client.go:
- Message.StructuredContent: stores JSON content blocks for correct
  history replay of tool_use and tool_result turns
- AskWithRequest(): when ToolDefs non-nil, passes tools to API and
  returns STOP:/TOOL: formatted response for Limbo parsing
  Format: "STOP:tool_use\nTOOL:<id>:<name>:<args>\n<text>" or
          "STOP:end_turn\n<text>" or plain text (no tools)
- AskWithToolResults(): submits tool results as a new user turn
- Helpers: buildMessageParam(), buildToolParams(), extractToolArgs(),
  jsonEscapeString()

session.go:
- Session.tools field + SetTools/Tools methods
- Session.AddStructuredMessage() for storing structured content blocks
- AskRequest extended with ToolDefs and ToolResults fields
- SessionManager.Ask(): includes tools, stores structured JSON in history
- SessionManager.AskWithToolResults(): new method for tool result turns
- Fix Compact() for new AskResponse return type
- Helpers: extractTextContent(), buildToolResultsJSON()

cli_client.go: update AskWithRequest() to return AskResponse (no tools
support; StructuredJSON always empty)

session_tools.go (new): /n/llm/{id}/tools write-only file
- Write JSON array of ToolDef to enable native tool_use protocol
- Empty write clears tools (returns session to text-only mode)

session_ask.go:
- Detect TOOL_RESULTS\n prefix in Write() → parseToolResults() → AskWithToolResults()
- TOOL_RESULTS format: "TOOL_RESULTS\n<id>\n<content>\n---\n..."

session_dir.go: add tools file to Children() and Lookup()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
f2d8604a — pdfinn 6 months ago
fix(llm): strip CLAUDECODE env var before spawning claude subprocess

The claude CLI refuses to run when CLAUDECODE is set in the environment,
as it detects a nested Claude Code session. When llm9p is launched from
within Claude Code, all subprocesses inherit this variable and every LLM
call fails with "Cannot be launched inside another Claude Code session".

Added claudeEnv() helper that filters CLAUDECODE from os.Environ() before
passing the environment to cmd. Replaced all five cmd.Environ() call sites
in cli_client.go (Ask, Compact, StartStream, AskWithHistory, AskWithRequest).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
58bb4b6e — pdfinn 6 months ago
test(llm9p): add tests for per-session compact and usage files

- session_compact_test.go (llm): Tests for Session.EstimatedContextTokens,
  SessionManager.Compact (not found, too short, replaces messages, resets
  tokens), SessionManager.ContextLimit and EstimatedContextTokens.

- session_compact.go (llmfs): Fix Stat().Length for SessionCompactFile
  (was 0 from BaseFile default; now returns fixed read-msg length).

- session_compact_test.go (llmfs): Tests for SessionCompactFile (read,
  read EOF, write no-op short history, write compacts, stat), for
  SessionUsageFile (format, EOF, read-only write, stat, dynamic content),
  and SessionDir.Children/Lookup wiring for compact and usage.

52 tests total, all pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
6da61fd4 — pdfinn 6 months ago
feat(llm9p): per-session compact and usage files for context window management

Add automatic context window compaction support to the per-session 9P API:

- session.go: Add Session.EstimatedContextTokens() (4 chars/token heuristic
  over current messages — more accurate than cumulative totalTokens for
  threshold decisions). Add SessionManager.Compact(ctx, id) which summarises
  the conversation via AskWithRequest then replaces session.messages with a
  compact 2-message exchange. Add SessionManager.EstimatedContextTokens(id)
  and SessionManager.ContextLimit() (200K for all Claude models).

- session_compact.go: New /n/llm/N/compact file. Write any content to
  trigger Compact() for that session. Follows the SessionModelFile pattern.

- session_usage.go: New /n/llm/N/usage file. Read returns
  "estimated_tokens/200000\n". Follows the SessionModelFile pattern.

- session_dir.go: Wire compact and usage into Children() and Lookup().

Also land two pre-existing uncommitted fixes:
- cli_client.go: Accept result messages with empty Result field
- protocol.go: Increase MaxMessageSize 8192→65536 for large system prompts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
a3dc06aa — pdfinn 7 months ago
feat(llm9p): Implement clone-based session architecture

Replace per-fid session model with Plan 9 clone pattern:
- Reading /n/llm/new creates a session and returns its ID
- Each session gets its own directory: /n/llm/<id>/
- Per-session files: ask, ctl, model, system, thinking, context, metrics
- AskWithRequest method for stateless CSP-style LLM calls
- Session settings (model, temperature, thinking) are per-session
- Remove old ask.go, context.go in favor of session-scoped files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ed43a61b — pdfinn 7 months ago
feat(llm9p): Add per-fid session isolation and prefill support

- Add SessionManager for per-fid conversation isolation
- Each 9P fid now gets its own conversation history
- Add FidAwareFile interface for files needing fid context
- Add /n/llm/prefill file for assistant response prefill
- Prefill helps keep model in character (e.g., "[Veltro]")
- Update ask, new, context files to use session manager
- Fix context contamination between parent and subagent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
fa382007 — pdfinn 7 months ago
feat(llm): Add extended thinking support and usage tracking

- Add thinking token control via /n/llm/thinking file (max/off/number)
- CLI backend sets MAX_THINKING_TOKENS env var for Claude CLI
- Default to max thinking (31999 tokens) for CLI backend
- Add /n/llm/usage file for token usage monitoring
- Add /n/llm/compact file for conversation summarization
- Extend Backend interface with ThinkingTokens, TotalTokens, ContextLimit, Compact
- Add true streaming support for CLI backend with line-by-line output
- Update example file with thinking documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
4e8ff274 — pdfinn 7 months ago
feat: Add system prompt file for persistent persona configuration

- Add system file (read/write) to set system prompt
- System prompt persists across conversation resets
- Add SystemPrompt() and SetSystemPrompt() to Backend interface
- Update both API and CLI clients to support dedicated system prompt
- Update documentation and examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Next