# AbstractCore — Full LLM Context

> AbstractCore is a Python library that provides a unified `create_llm(...)` API over multiple LLM backends (cloud + local). Optional subsystems (tools, embeddings, media handling, glyph compression, server) are enabled via explicit install extras; the default install is intentionally small.

Last updated: 2026-02-04  
Package version: 2.11.2 (`abstractcore/utils/version.py`)

## What to install

### Core (lightweight default)

```bash
pip install abstractcore
```

Core deps are intentionally minimal (`pydantic`, `httpx`). Anything heavy/optional must live behind an extra.

### Providers

```bash
pip install "abstractcore[openai]"
pip install "abstractcore[anthropic]"
pip install "abstractcore[huggingface]"  # heavy (transformers/torch)
pip install "abstractcore[mlx]"          # heavy (Apple Silicon)
pip install "abstractcore[vllm]"         # heavy (GPU server integration)
```

Notes:
- Providers like `ollama`, `lmstudio`, `openrouter`, and `openai-compatible` use the core dependencies only (no provider-specific extra required).

### Optional features

```bash
pip install "abstractcore[tools]"       # built-in web tools (requests/bs4/lxml, DDG)
pip install "abstractcore[media]"       # images + PDF/Office extraction (Pillow, PyMuPDF4LLM, unstructured, ...)
pip install "abstractcore[compression]" # glyph visual-text compression (Pillow renderer)
pip install "abstractcore[embeddings]"  # local embeddings (sentence-transformers, numpy)
pip install "abstractcore[tokens]"      # precise token counting (tiktoken)
pip install "abstractcore[server]"      # FastAPI OpenAI-compatible gateway (+ endpoint)
```

### Turnkey installs (pick one)

```bash
pip install "abstractcore[all-apple]"    # macOS/Apple Silicon (includes MLX, excludes vLLM)
pip install "abstractcore[all-non-mlx]"  # Linux/Windows/Intel Mac (excludes MLX and vLLM)
pip install "abstractcore[all-gpu]"      # Linux NVIDIA GPU (includes vLLM, excludes MLX)
```

Notes:
- Quote extras in shells like zsh: `"abstractcore[media]"`.
- `abstractcore[all]` exists but may not install cleanly everywhere (MLX + vLLM together).

## Key docs (source of truth)

- `README.md`: overview, install matrix, quickstart
- `docs/getting-started.md`: 5-minute intro (`create_llm`, `generate`)
- `docs/README.md`: docs index and reading paths
- `docs/api.md`: user-facing public API map
- `docs/api-reference.md`: complete Python API reference (includes events)
- `docs/faq.md`: common setup questions and gotchas
- `CHANGELOG.md`: version history and upgrade notes
- `SECURITY.md`: responsible vulnerability reporting

Notes:
- `docs/reports/`, `docs/research/`, and `docs/archive/` are useful background but are not canonical user docs.

## Key entrypoints (how users interact)

### Python API

```python
from abstractcore import create_llm, BasicSession, tool

llm = create_llm("openai", model="gpt-4o-mini")  # example model; use one you have access to
resp = llm.generate("Hello!")
print(resp.content)
```

### Console scripts (see `pyproject.toml`)

- `abstractcore` / `abstractcore-config`: config CLI (API keys, defaults)
- `abstractcore-chat`: interactive REPL
- `summarizer`, `extractor`, `judge`, `intent`, `deepsearch`: task-specific CLI apps
- `abstractcore-endpoint`: single-model `/v1` server (see `abstractcore/endpoint/app.py`)

Server modules:
- Multi-provider gateway: `python -m abstractcore.server.app` (see `abstractcore/server/app.py`)
- Single-model endpoint: `abstractcore-endpoint --provider ... --model ...` (see `abstractcore/endpoint/app.py`)

## Packaging / optional-dependency rules (important)

AbstractCore is designed so that:
- `pip install abstractcore` stays small and imports cleanly.
- Heavy deps are opt-in via extras.

Rules to keep it that way:

1. **Do not import optional deps in default import paths**
   - Keep `abstractcore/__init__.py` lightweight.
   - Avoid top-level imports of provider SDKs (`openai`, `anthropic`), embeddings libs (`sentence_transformers`), media libs (`Pillow`, `pymupdf4llm`, `unstructured`), or web libs (`requests`, `bs4`).

2. **Use lazy imports / best-effort optional imports**
   - Import optional modules inside functions/methods where they’re used.
   - When a dependency is missing, raise a clear error with a quoted extra, e.g. `pip install "abstractcore[media]"`.

3. **Keep optional subsystems modular**
   - Embeddings live under `abstractcore/embeddings/` and require `abstractcore[embeddings]`.
   - Media/PDF/Office processing lives under `abstractcore/media/` and requires `abstractcore[media]`.
   - Built-in web tools that need requests/bs4 live in `abstractcore/tools/common_tools.py` and require `abstractcore[tools]`.
   - Glyph compression lives under `abstractcore/compression/` and requires `abstractcore[compression]`.

4. **Docs and user-facing error messages**
   - Always quote extras: `pip install "abstractcore[server]"` (zsh-safe).
   - Don’t claim hard performance numbers unless they’re reproducible and labeled as such.

## Repo map (where to edit)

Core:
- `abstractcore/__init__.py`: public exports (keep minimal)
- `abstractcore/core/`: interfaces, factory, session, types
- `abstractcore/providers/`: provider implementations + shared base (`base.py`)
- `abstractcore/providers/registry.py`: provider registry + install hints

Features:
- `abstractcore/tools/`: tool system (decorator, parser, registry, handler)
- `abstractcore/media/`: media pipeline (capabilities, processors, auto handler)
- `abstractcore/compression/`: glyph compression (renderer + orchestrator)
- `abstractcore/embeddings/`: embedding manager + model configs

Server:
- `abstractcore/server/`: multi-provider OpenAI-compatible gateway (FastAPI)
- `abstractcore/endpoint/`: single-model endpoint server (FastAPI)

Docs:
- `README.md`: public overview + install matrix
- `docs/`: canonical docs (index at `docs/README.md`)
- `docs/reports/`: non-authoritative engineering notes
- `docs/research/`: non-authoritative experiments
- `docs/archive/`: historical/superseded docs

Tests:
- `tests/`: pytest suite (`pytest.ini`)

Packaging:
- `pyproject.toml`: deps/extras/scripts (keep default deps minimal)
- Version: `abstractcore/utils/version.py`

## How the big pieces work (high level)

### Provider IDs (factory names)

Common provider IDs used by `create_llm(provider=..., model=...)`:
- Cloud: `openai`, `anthropic`, `openrouter`
- Local/self-hosted: `ollama`, `lmstudio`, `mlx`, `huggingface`, `vllm`, `openai-compatible`

### Tool calling

- Default mode is **passthrough**: AbstractCore parses tool calls and returns them in `GenerateResponse.tool_calls`; the host/runtime executes tools.
- Tools are created with `@tool` (`abstractcore/tools/core.py`). The decorator is dependency-free.
- Built-in web tools (`fetch_url`, `web_search`, …) live in `abstractcore/tools/common_tools.py` and require `abstractcore[tools]`.
- Prompted tool catalogs include short `when_to_use` hints for small tool sets and a few high-impact tools (edit/write/execute + web triage tools) to improve tool choice without bloating prompts.

### Structured output

- Use `response_model=YourPydanticModel` in `generate(...)`.
- The provider layer selects the best available strategy (native when supported; otherwise prompted validation + retry).

### Media input (images/audio/video + documents)

- Use `media=[...]` on `generate(...)` to attach images/audio/video/documents.
- Input modality behavior is policy-driven and fails loudly by default unless an explicit fallback is configured:
  - audio: `audio_policy` (`native_only|speech_to_text|auto|caption`)
  - video: `video_policy` (`native_only|frames_caption|auto`)
  - images: optional vision fallback (caption → inject short observations; config-driven)
- Advanced PDF/Office extraction requires `abstractcore[media]` (PyMuPDF4LLM + unstructured).

### Capability plugins (voice/audio/vision)

- `AbstractCoreInterface` exposes lazy facades:
  - `llm.capabilities` (plugin registry + `status()`)
  - `llm.voice` / `llm.audio` / `llm.vision` (convenience proxies)
- Plugins are discovered via Python entry points (`abstractcore.capabilities_plugins`).
- Optional packages register backends:
  - install `abstractvoice` → `core.voice` + `core.audio` (TTS/STT)
  - install `abstractvision` → `core.vision` (generative vision)

Server note:
- AbstractCore Server can optionally expose `/v1/images/*` (requires `abstractvision`) and `/v1/audio/*` (requires an audio/voice plugin, typically `abstractvoice`).

### Glyph visual-text compression

- Optional, experimental: render long text into images and process with a vision-capable model.
- Requires `abstractcore[compression]` (Pillow renderer).
- Optional (experimental): direct PDF→image conversion uses `pdf2image` (+ system Poppler) and is not part of the default extras.

### Embeddings

- Use `from abstractcore.embeddings import EmbeddingManager`.
- Requires `abstractcore[embeddings]`.

## Development commands

Repo dev install:

```bash
pip install -e ".[dev]"
```

Run tests:

```bash
pytest -q
pytest tests/structured -q
```

If you need provider-specific tests, install the relevant extra as well (e.g. `pip install -e ".[huggingface]"`).

Benchmarks:
- Web triage tool footprint: `python examples/skim_tools_benchmark.py --help`