~kris/9p

llm9p

ref: 2eac753a4d03c6eb26f9196b204498e3cba7af8a llm9p/internal/protocol d---------
2eac753a — Claude 6 months ago
Add OpenAI-compatible backend for local LLM support (GPT-OSS)

Implement OpenAIClient backend that speaks the OpenAI Chat Completions
API (/v1/chat/completions), enabling llm9p to work with any local model
server: Ollama, vLLM, llama-server, LocalAI, or LM Studio.

Primary target is GPT-OSS (OpenAI's open-weight MoE models), but any
model served via these platforms works. The backend supports:
- Blocking and streaming chat completions
- Tool/function calling with STOP:/TOOL: formatting
- Token counting from API usage (with estimation fallback)
- Conversation history, system prompts, temperature control
- Stateless AskWithRequest for session isolation

Usage:
  ./llm9p -backend openai -openai-url http://localhost:11434/v1 -model gpt-oss:20b

Also fixes pre-existing stale mock backends in test files (AskWithRequest
signature was out of date with the Backend interface).

https://claude.ai/code/session_017qVVZUUhfCCvNkMYmXDZAa
822b318b — pdfinn 6 months ago
fix(llmfs): clean up sessions on disconnect to prevent memory leak

Each spawned subagent creates an LLM session via /n/llm/new, but sessions
were never automatically freed: BaseFile.Close() was a no-op, and connection
drop handling only removed the client from the map without touching sessions.
Repeated spawn calls caused sessions to accumulate indefinitely, consuming
memory for full conversation histories.

Changes:
- Add refs int32 (atomic) to Session struct
- Add IncRef(id) / DecRef(id) to SessionManager; DecRef calls Close when
  refs reach zero, freeing the session and all its conversation history
- Add sessionRefFile / sessionRefDir wrappers (session_ref.go) that call
  IncRef on Open and DecRef on Close (once.Do guards against double-decrement)
- SessionsDir.Lookup wraps the returned SessionDir in sessionRefDir, so
  child file lookups via sessionRefDir.Lookup also yield sessionRefFile objects
- Fix handleConn disconnect path to call file.Close() on all unclosed fids
  before removing the client, ensuring sessions are freed on network drop

Typical flow: subagent exits → NEWFD closes all FDs → 9P connection drops →
handleConn deferred cleanup calls Close on remaining fids → sessionRefFile.Close
→ sm.DecRef → refs=0 → sm.Close(id) → session deleted from map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bad7a7b7 — pdfinn 6 months ago
feat(llmfs): async Write + per-session streaming for live token delivery

Enable lucibridge to stream LLM tokens into the Lucifer conversation
zone as they are generated, without waiting for the full response.

Changes:
- session.go: add BeginGeneration/EndGeneration/SendChunk/GetStreamCh/
  WaitDone methods on Session; streamCh (cap 256) carries raw text chunks
  during generation; doneCh signals completion; EndGeneration closes
  channels but does NOT nil streamCh (late readers still see closed chan)
- session_ask.go: Write() is now async — calls BeginGeneration(), spawns
  goroutine, returns immediately; Read() calls WaitDone() before accessing
  LastResponse so pread blocks until generation completes
- session_stream.go (new): /n/llm/N/stream file; Read() blocks on <-ch
  returning each text chunk as it arrives; returns EOF when generation
  is done or no generation is active (channel nil or closed)
- session_dir.go: register stream file in Children() and Lookup()
- client.go: AskWithRequest() branches on req.StreamFunc != nil to use
  SSE Messages.NewStreaming() path; text_delta events forwarded to
  StreamFunc; session.Ask() sets StreamFunc=session.SendChunk when
  GetStreamCh() is non-nil
- server.go: add detailed walk debug logging (names, types, failures)
  controlled by existing -debug flag

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
6da61fd4 — pdfinn 6 months ago
feat(llm9p): per-session compact and usage files for context window management

Add automatic context window compaction support to the per-session 9P API:

- session.go: Add Session.EstimatedContextTokens() (4 chars/token heuristic
  over current messages — more accurate than cumulative totalTokens for
  threshold decisions). Add SessionManager.Compact(ctx, id) which summarises
  the conversation via AskWithRequest then replaces session.messages with a
  compact 2-message exchange. Add SessionManager.EstimatedContextTokens(id)
  and SessionManager.ContextLimit() (200K for all Claude models).

- session_compact.go: New /n/llm/N/compact file. Write any content to
  trigger Compact() for that session. Follows the SessionModelFile pattern.

- session_usage.go: New /n/llm/N/usage file. Read returns
  "estimated_tokens/200000\n". Follows the SessionModelFile pattern.

- session_dir.go: Wire compact and usage into Children() and Lookup().

Also land two pre-existing uncommitted fixes:
- cli_client.go: Accept result messages with empty Result field
- protocol.go: Increase MaxMessageSize 8192→65536 for large system prompts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
a3dc06aa — pdfinn 7 months ago
feat(llm9p): Implement clone-based session architecture

Replace per-fid session model with Plan 9 clone pattern:
- Reading /n/llm/new creates a session and returns its ID
- Each session gets its own directory: /n/llm/<id>/
- Per-session files: ask, ctl, model, system, thinking, context, metrics
- AskWithRequest method for stateless CSP-style LLM calls
- Session settings (model, temperature, thinking) are per-session
- Remove old ask.go, context.go in favor of session-scoped files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ed43a61b — pdfinn 7 months ago
feat(llm9p): Add per-fid session isolation and prefill support

- Add SessionManager for per-fid conversation isolation
- Each 9P fid now gets its own conversation history
- Add FidAwareFile interface for files needing fid context
- Add /n/llm/prefill file for assistant response prefill
- Prefill helps keep model in character (e.g., "[Veltro]")
- Update ask, new, context files to use session manager
- Fix context contamination between parent and subagent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
68199d2a — pdfinn 7 months ago
feat: Initial implementation of llm9p - LLM as 9P filesystem

Exposes Claude as a 9P filesystem, enabling interaction through
standard file operations:

- ask: write prompt, read response (shim pattern)
- model: read/write current model name
- temperature: read/write sampling temperature
- tokens: read-only token count from last response
- new: write to reset conversation
- context: read JSON history, write to add system message
- _example: usage documentation
- stream/chunk: blocking read for streaming responses

Includes:
- Full 9P2000 protocol implementation (stdlib only)
- Anthropic SDK integration with conversation state
- Streaming support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>