~kris/9p

llm9p

ref: 42c2e6958db4e870f21ce0b60b7522975cd8757f llm9p/internal/llm/openai_client.go -rw-r--r-- 30.8 KiB
42c2e695 — pdfinn 5 months ago
fix: extract tool args and fix tool result history for OpenAI path

Mirror of infernode llmclient fixes for the Go llm9p server:

1. Call extractToolArgs on OpenAI tool_call arguments before building
   TOOL: lines. The Anthropic path already did this; the OpenAI path
   passed raw {"args":"value"} JSON through to the agent layer.

2. Handle user-role messages with StructuredContent (tool results)
   in buildChatMessages. Previously these were emitted as plain
   {"role":"user","content":"tool results submitted"}, losing the
   actual results and breaking role alternation. Now expanded into
   individual {"role":"tool"} messages via rebuildToolResultMessages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fd213ef6 — pdfinn 5 months ago
feat(openai): fallback text tool-call parser for non-Anthropic models

When Ollama/Qwen models generate tool calls as text instead of structured
API responses, parse <function=>, <tool_call>, and <|tool_call|> formats
from the content and promote them to proper STOP:tool_use/TOOL: wire
format. Validates tool names against definitions, rejects hallucinated tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1dc34eeb — pdfinn 6 months ago
feat(llm): add thinking control for OpenAI/Ollama backend

Implements unified thinking interface for the OpenAI-compatible backend
(Ollama, gpt-oss, etc.) matching the existing Claude token-budget semantics:
  0           → think: false  (disabled)
  1–10000     → think: true,  think_level: "low"
  10001–20000 → think: true,  think_level: "medium"
  20001+ / -1 → think: true,  think_level: "high"

Bypasses go-openai library for AskWithRequest to send Ollama-specific
options field; implements SSE parsing with bufio.Scanner for the streaming
path and direct JSON decode for the blocking path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
11a3967d — P. D. Finn KD9WEH 6 months ago
Merge pull request #1 from NERVsystems/claude/local-llm-feasibility-gVLhq

Add OpenAI-compatible local LLM backend support
2eac753a — Claude 6 months ago
Add OpenAI-compatible backend for local LLM support (GPT-OSS)

Implement OpenAIClient backend that speaks the OpenAI Chat Completions
API (/v1/chat/completions), enabling llm9p to work with any local model
server: Ollama, vLLM, llama-server, LocalAI, or LM Studio.

Primary target is GPT-OSS (OpenAI's open-weight MoE models), but any
model served via these platforms works. The backend supports:
- Blocking and streaming chat completions
- Tool/function calling with STOP:/TOOL: formatting
- Token counting from API usage (with estimation fallback)
- Conversation history, system prompts, temperature control
- Stateless AskWithRequest for session isolation

Usage:
  ./llm9p -backend openai -openai-url http://localhost:11434/v1 -model gpt-oss:20b

Also fixes pre-existing stale mock backends in test files (AskWithRequest
signature was out of date with the Backend interface).

https://claude.ai/code/session_017qVVZUUhfCCvNkMYmXDZAa