~kris/9p

llm9p

20386977c1adea41e728d4b639fe51f5186cef0d — pdfinn 7 months ago 0f0dfed
feat: Add stream/ask file to trigger streaming requests

- Add StreamAskFile to start streaming via write to stream/ask
- Read chunks from stream/chunk as they arrive
- Update documentation with streaming examples and verified tests
- Update _example file with correct streaming instructions

Streaming workflow:
1. Write prompt to stream/ask to start streaming
2. Read from stream/chunk to get chunks (blocks until available)
3. EOF returned when stream completes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
5 files changed, 92 insertions(+), 7 deletions(-)

M CLAUDE.md
M README.md
M internal/llmfs/example.go
M internal/llmfs/root.go
M internal/llmfs/stream.go
M CLAUDE.md => CLAUDE.md +9 -0
@@ 313,6 313,15 @@ The following scenarios have been tested and verified working:
- [x] `echo 'prompt' > /n/llm/ask` followed by `cat /n/llm/ask` - Full LLM interaction works
- [x] LLM correctly identifies client as Inferno OS when asked

### Streaming (plan9port)
- [x] `ls stream` - Lists `ask` and `chunk` files
- [x] `echo "prompt" | 9p write stream/ask` - Starts streaming request
- [x] `9p read stream/chunk` - Returns streamed chunks
- [x] Multiple chunks received for longer responses
- [x] EOF returned when stream completes
- [x] Short response ("Write a haiku") streams correctly
- [x] Long response ("Count 1 to 20") streams all content

## Future Enhancements

- [ ] Multiple conversation support (via subdirectories)

M README.md => README.md +33 -2
@@ 150,22 150,53 @@ cat /mnt/llm/_example
├── context          # Read: conversation history; Write: add system message
├── _example         # Read-only: usage examples
└── stream/          # Streaming interface
    └── chunk        # Read blocks until next chunk, EOF on completion
    ├── ask          # Write-only: starts a streaming request
    └── chunk        # Read-only: blocks until next chunk, EOF on completion
```

### File Behaviors

| File | Read | Write |
|------|------|-------|
| `ask` | Returns last LLM response | Sends prompt to LLM, stores response |
| `ask` | Returns last LLM response | Sends prompt to LLM (sync), stores response |
| `model` | Returns current model name | Sets model for subsequent requests |
| `temperature` | Returns current temperature | Sets temperature (0.0-2.0) |
| `tokens` | Returns last response token count | Permission denied |
| `new` | Permission denied | Any write resets conversation state |
| `context` | Returns JSON conversation history | Appends system message to context |
| `_example` | Returns usage examples | Permission denied |
| `stream/ask` | Permission denied | Starts a streaming request |
| `stream/chunk` | Blocks until next chunk, returns it | Permission denied |

## Streaming

For long responses, use the streaming interface to see output as it's generated:

```bash
# Start a streaming request (using 9p tool)
echo "Write a poem about the moon" | 9p -a localhost:5640 write stream/ask &

# Read chunks as they arrive
while chunk=$(9p -a localhost:5640 read stream/chunk 2>/dev/null); do
  [ -z "$chunk" ] && break
  printf "%s" "$chunk"
done
```

With a mounted filesystem:

```bash
# Start streaming in background
echo "Explain quantum computing" > /mnt/llm/stream/ask &

# Read chunks
while read -r chunk < /mnt/llm/stream/chunk 2>/dev/null; do
  printf "%s" "$chunk"
done
```

**Note:** Start reading chunks immediately after writing to `stream/ask`. If you wait too long, the stream may complete and you'll get EOF.

## Shell Scripting

```bash

M internal/llmfs/example.go => internal/llmfs/example.go +8 -5
@@ 25,9 25,11 @@ Conversation Management:
Token Usage:
  cat tokens                     # View tokens from last response

Streaming (Advanced):
  echo "Tell me a story" > ask   # Start generating
  cat stream/chunk               # Read chunks as they arrive (blocks)
Streaming:
  echo "Tell me a story" > stream/ask  # Start streaming request
  cat stream/chunk                      # Read next chunk (blocks until available)
  # Keep reading stream/chunk until EOF for full response
  # Note: Read chunks immediately after writing to stream/ask

Shell Scripting:
  #!/bin/sh


@@ 46,14 48,15 @@ Environment:
  ANTHROPIC_API_KEY must be set when starting the server

Files:
  ask          Read/write: prompt goes in, response comes out
  ask          Read/write: prompt goes in, response comes out (sync)
  model        Read/write: current model name
  temperature  Read/write: sampling temperature (0.0-2.0)
  tokens       Read-only: token count from last response
  new          Write-only: any write resets conversation
  context      Read: JSON history; Write: add system message
  _example     Read-only: this help text
  stream/chunk Read-only: streaming chunks (blocking)
  stream/ask   Write-only: starts a streaming request
  stream/chunk Read-only: returns next chunk (blocks), EOF when done
`

// NewExampleFile creates the _example file with usage examples

M internal/llmfs/root.go => internal/llmfs/root.go +1 -0
@@ 21,6 21,7 @@ func NewRoot(client *llm.Client) protocol.Dir {

	// Add stream directory
	streamDir := protocol.NewStaticDir("stream")
	streamDir.AddChild(NewStreamAskFile(client))
	streamDir.AddChild(NewChunkFile(client))
	root.AddChild(streamDir)


M internal/llmfs/stream.go => internal/llmfs/stream.go +41 -0
@@ 1,7 1,9 @@
package llmfs

import (
	"context"
	"io"
	"strings"

	"github.com/NERVsystems/llm9p/internal/llm"
	"github.com/NERVsystems/llm9p/internal/protocol"


@@ 51,3 53,42 @@ func (f *ChunkFile) Stat() protocol.Stat {
	s.Length = 0
	return s
}

// StreamAskFile starts a streaming request
// Write a prompt to start streaming, then read chunks from stream/chunk
type StreamAskFile struct {
	*protocol.BaseFile
	client *llm.Client
}

// NewStreamAskFile creates the stream/ask file
func NewStreamAskFile(client *llm.Client) *StreamAskFile {
	return &StreamAskFile{
		BaseFile: protocol.NewBaseFile("ask", 0222), // write-only
		client:   client,
	}
}

func (f *StreamAskFile) Read(p []byte, offset int64) (int, error) {
	return 0, protocol.ErrPermission
}

func (f *StreamAskFile) Write(p []byte, offset int64) (int, error) {
	prompt := strings.TrimSpace(string(p))
	if prompt == "" {
		return len(p), nil
	}

	// Start streaming - chunks will be available via stream/chunk
	err := f.client.StartStream(context.Background(), prompt)
	if err != nil {
		// Return error to indicate stream failed to start
		return 0, err
	}

	return len(p), nil
}

func (f *StreamAskFile) Stat() protocol.Stat {
	return f.BaseFile.Stat()
}