ION606
221 lines
10 KiB
Markdown
221 lines
10 KiB
Markdown
# ML Stack — Local AI Orchestration Toolkit
|
||
|
||
This repository packages a complete self-hosted assistant stack around Open WebUI plus several companion services: a scheduler that can trigger chats and workflows, a docker-backed code runner, a Roku remote tool server, Nextcloud file access, SearxNG metasearch, and a headless browser UI for deep-research sessions. Everything is wired together through `docker-compose.yml` so the stack can be brought up on a single host.
|
||
|
||
_Last updated: 2025-10-03_
|
||
|
||
---
|
||
|
||
## A (Few) Notes
|
||
|
||
1. ports are currently exposed on most services for development purposes (e.g. 12253 for the scheduler), remove these in production or consider adding a proxy
|
||
|
||
2. **ALL DATA IS STORED IN VOLUMES!!!** This means if you do `docker compose down -v` your data **WILL** dissapear. Consider mounting a persistant directory to avoid this
|
||
|
||
3. Before starting the cluster, check if you need the different components (e.g. Nextcloud Tool Server). They are set to restart on failiure and will throw if missing env vars/credentials, which will loop endlessly
|
||
|
||
4. If you do not use cloudflared for tunneling, please adjust the CORS policies accordingly, and consider adding a reverse proxy to either your local machine or the compose
|
||
|
||
5. The code runner and scheduler both mount the host Docker socket. Ensure the host user/group IDs match the compose configuration (`DOCKER_GID` build arg defaults to 977) so containers can operate without root. This will be replaced when I enentually migrate this to a kubernetes cluster
|
||
|
||
6. When adjusting `NEXTCLOUD_ACCESS_DIRS`, remember to restart `ollama-nextcloud` so the regex list is reloaded
|
||
|
||
---
|
||
|
||
## Stack At A Glance
|
||
|
||
| Compose service | Directory / build context | External ports | Primary role |
|
||
|-----------------|---------------------------|----------------|--------------|
|
||
| `open-webui` | (image: `ghcr.io/open-webui/open-webui:main`) | `4000 -> 8080` | Chat UI, agent orchestration, embedded knowledge base & RAG powered by Postgres |
|
||
| `postgres` | – | – | Persistence for Open WebUI (users, KB, events) |
|
||
| `searxng` | `searxng.yml` | `4001 -> 8080` (debug only) | Private SearxNG instance used for live web search tools |
|
||
| `coderunner` | `coderunner/` | – (internal `8787`) | Bun service that executes pure source code inside sandboxed Docker containers |
|
||
| `openwebui_tools` | `tools/` | – (internal `1331`) | Python Roku remote API exposed as an OpenAPI tool server |
|
||
| `browser` | `browser/` | `7788 -> 7788` | Playwright Chromium UI for autonomous browsing / research |
|
||
| `schedules-api` | `scheduler/` | `12253 -> 12253` | Cron-style job scheduler that can open chats, call templates, and upload files |
|
||
| `ollama-nextcloud` | `nextcloud/` | `13284 -> 1111` | Nextcloud WebDAV proxy with caching and access controls |
|
||
|
||
Volumes declared in compose: `open-webui`, `pgdata`, `searxng_data`, `webui_data`, `schedule_data`, and `nextcloud_data`
|
||
|
||
|
||
> [!CAUTION]
|
||
> PLEASE I BEG OF YOU REMEMBER TO BACK THESE UP/USE A LOCAL DIRECTORY.
|
||
> IF YOU DO NOT AND REMOVE OR PRUNE THE VOLUMES YOU WILL LOSE *ALL* DATA
|
||
|
||
---
|
||
|
||
## Service Details
|
||
|
||
### Open WebUI (`open-webui`)
|
||
- Runs the latest `ghcr.io/open-webui/open-webui:main` image with Postgres backing for durable data (`open-webui` and `pgdata` volumes)
|
||
|
||
- `.env` enables the login form, optional API keys (not currently used), and forwards identifying headers so downstream tools know which user initiated a request
|
||
|
||
- Depends on the tool containers (`openwebui_tools`, `coderunner`, `schedules-api`, `ollama-nextcloud`) via internal networking; discover their OpenAPI docs from inside the UI to register tools
|
||
|
||
### Postgres (`postgres`)
|
||
> [!IMPORTANT]
|
||
> If you plan on exposing ports on this service, please move the inline credentials to the `.env` file
|
||
|
||
- Standard `postgres:latest` image. Credentials are set inline in compose for local development
|
||
|
||
- Health-checked with `pg_isready`; the data volume `pgdata` stores Open WebUI metadata
|
||
|
||
### SearxNG (`searxng`)
|
||
- Private SearxNG deployment for agent web search tasks with HTML/JSON outputs enabled
|
||
|
||
- Mounts `searxng.yml` and persists internal data to `searxng_data`. External port 4001 is exposed only for local debugging and should be removed in production
|
||
|
||
### Code Runner (`coderunner`)
|
||
- Bun-based HTTP server that accepts pure source code plus optional extra files, then runs the workload in a throwaway Docker container pinned to an allow-listed base image per language
|
||
|
||
- Enforces strict limits (`--network=none`, read-only root FS, tmpfs workdir, CPU/memory caps, dropped capabilities). Supported Languages:
|
||
- `python`
|
||
- `node`
|
||
- `bun`
|
||
- `bash`
|
||
- `ruby`
|
||
- `go`
|
||
- `rust`
|
||
- `java`
|
||
- `c`
|
||
- `cpp`
|
||
- Exposes `GET /openapi.json` and `POST /execute` inside the internal network (`http://coderunner:8787`). Requires the host Docker socket to spawn child sandboxes; the compose file mounts it read-only with matching group ID.
|
||
|
||
### Roku Tool Server (`openwebui_tools`)
|
||
- Lightweight Python HTTP server that proxies Roku remote commands
|
||
|
||
- Reads `ROKU_IP` from `.env`; returns helpful errors when the IP is missing or the device is offline
|
||
|
||
- Serves `GET /roku/openapi.json` for automatic tool registration and handles `GET /roku/{command}` requests. Supported command list matches the enum in `spec/roku.openapi.json` (navigation, inputs, power, volume, remote finder)
|
||
|
||
### Browser Research UI (`browser`)
|
||
- Builds the upstream `browser-use/web-ui` project, installs Chromium plus dependencies, and launches the UI on port 7788
|
||
|
||
- Runs as an unprivileged user (uid 1000) with dedicated tmpfs directories and a `webui_data` volume for persisted history/state
|
||
|
||
- Configure resolution, telemetry, and default LLM via `browser/.env` or container environment variables
|
||
|
||
- The browser-use docs can be found at https://docs.browser-use.com/
|
||
|
||
### Scheduler API (`schedules-api`)
|
||
- Bun/Node cron worker that lets you schedule Open WebUI chats or template-driven jobs using authenticated user tokens
|
||
|
||
- Persists schedule definitions to `schedule_data` (JSON payload) and can store uploaded supporting files under the same volume
|
||
|
||
- Reads workflow templates from the bundled `scheduler/templates.json`. To inject custom templates, mount a host file or populate the root-level `templates.json/` directory and update the compose volume mapping
|
||
|
||
- Key endpoints (documented in `scheduler/openapi.json`):
|
||
- `GET /openapi.json`: tool contract.
|
||
- `POST /api/schedules`: create or replace a schedule (cron or one-shot ISO timestamp). Validates feature flags, attachments, and template references
|
||
- `GET /api/schedules`: list schedules scoped to the calling user (identified via Open WebUI bearer token)
|
||
- `DELETE /api/schedules/{name}`: remove a schedule the user owns
|
||
- Includes a static UI in `scheduler/public/` for manual interaction. Uses `node-cron` to avoid overlapping executions; failed jobs clean themselves up
|
||
|
||
### Nextcloud Files Tool (`ollama-nextcloud`)
|
||
- Express + WebDAV proxy that exposes a simple JSON API for browsing, downloading, and uploading files stored in Nextcloud
|
||
|
||
- Environment variables (configured in `.env`):
|
||
- `NEXTCLOUD_APP_ID` / `NEXTCLOUD_APP_PASS` / `NEXTCLOUD_WEBDAV_ADDR`: service credentials
|
||
- `NEXTCLOUD_ACCESS_DIRS`: JSON array of regex strings that whitelist readable paths (e.g. `["^/Notes", "^/School"]`). When unset, the tool has full access
|
||
|
||
- Cached downloads are stored under `/tmp` using an embedded SQLite index (`cache.ts`). The server keeps ETags in sync and reuses cached bytes when possible unless `bypasscache` is requested
|
||
|
||
- Major endpoints (see `nextcloud/openapi.json`):
|
||
- `GET /openapi.json`: discovery document for tool registration.
|
||
- `POST /file`: fetch a file. Automatically caches and returns metadata + content-type.
|
||
- `POST /dir`: list directory contents (shallow or recursive).
|
||
- `PUT /file`: upload via multipart form-data (optional recursive dir creation, never overwrites existing files).
|
||
|
||
### Cloudflared Tunnel Config
|
||
- `cloudflared-tunnel-config.yml` maps friendly hostnames to the local services (Ollama, Open WebUI, tool servers). Use it as a blueprint when exposing the stack through Cloudflare Tunnels.
|
||
|
||
---
|
||
|
||
## Configuration (`.env`)
|
||
|
||
```env
|
||
ROKU_IP=
|
||
|
||
WEBUI_URL=
|
||
|
||
# use built-in login form (username/password)
|
||
ENABLE_LOGIN_FORM="true"
|
||
|
||
# forward identity on outbound model requests (if you're going to use openAI/external LLM)
|
||
ENABLE_FORWARD_USER_INFO_HEADERS="true"
|
||
|
||
# allow user api keys for the scheduler calling OWUI’s
|
||
ENABLE_API_KEY_AUTH="true"
|
||
|
||
NEXTCLOUD_APP_ID=
|
||
NEXTCLOUD_APP_PASS=
|
||
NEXTCLOUD_WEBDAV_ADDR=
|
||
NEXTCLOUD_ACCESS_DIRS=
|
||
```
|
||
|
||
---
|
||
|
||
## Running the Stack
|
||
|
||
1. Install Docker and Docker
|
||
|
||
2. Populate `.env` with the correct Roku and Nextcloud settings plus any Open WebUI options
|
||
|
||
3. Build images (pull base layers and bake GID overrides where needed):
|
||
```sh
|
||
docker compose build --pull
|
||
```
|
||
|
||
4. Launch everything:
|
||
```sh
|
||
docker compose up -d
|
||
```
|
||
|
||
5. Open WebUI is available on http://localhost:4000 (use credentials from the UI setup). The supporting services are reachable on the ports listed above or through the internal Docker network
|
||
|
||
To inspect logs for a specific service:
|
||
|
||
```sh
|
||
docker compose logs -f coderunner
|
||
```
|
||
|
||
Bring the stack down (volumes persist):
|
||
|
||
```sh
|
||
docker compose down
|
||
```
|
||
|
||
---
|
||
|
||
## Registering Tool Servers in Open WebUI
|
||
|
||
Inside Open WebUI (Settings --> Tools --> Add tool server), point to the internal URLs:
|
||
- Code runner: `http://coderunner:8787/openapi.json`
|
||
- Scheduler: `http://schedules-api:12253/openapi.json`
|
||
- Nextcloud files: `http://ollama-nextcloud:1111/openapi.json`
|
||
- Roku remote: `http://openwebui_tools:1331/roku/openapi.json`
|
||
|
||
These should be fully internal in the docker network. If you expose them consider using a reverse proxy/authentication
|
||
|
||
---
|
||
|
||
## Data, Volumes, and Shared Paths
|
||
|
||
- `open-webui` volume: Open WebUI application state (uploads, knowledge base, configs)
|
||
- `pgdata` volume: Postgres cluster data directory
|
||
- `searxng_data` volume: SearxNG runtime files
|
||
- `webui_data` volume: browser-use web UI session data
|
||
- `schedule_data` volume: scheduler persisted schedules and stored file attachments
|
||
- `nextcloud_data` volume: temp storage for cached Nextcloud content
|
||
|
||
> [!IMPORTANT]
|
||
> Back up the volumes you care about before upgrading images
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
The repository and reference code are released under Apache-2.0 (see `LICENSE`).
|
||
|