AbstractCore Capabilities

This document clearly explains what AbstractCore can and cannot do, helping you understand when to use it and when to look elsewhere.

What AbstractCore IS

AbstractCore is production-ready LLM infrastructure. It provides a unified, reliable interface to language models with essential features built-in.

Core Philosophy

Infrastructure

Not application logic

Reliability

Over features

Simplicity

Over complexity

Provider Agnostic

Works everywhere

Optional capability plugins (voice/audio/vision)

AbstractCore stays dependency-light by default. Deterministic modality APIs (STT/TTS, generative vision) live in optional packages and are exposed through the capability plugin layer:

Install abstractvoice → llm.voice / llm.audio (TTS/STT)
Install abstractvision → llm.vision (text→image, image→image, …)

pip install abstractvoice
pip install abstractvision

from abstractcore import create_llm

llm = create_llm("openai", model="gpt-4o-mini")  # example; pick a provider/model you have access to
print(llm.capabilities.status())  # availability + selected backend ids + install hints

# Voice/audio
wav_bytes = llm.voice.tts("Hello", format="wav")
text = llm.audio.transcribe("speech.wav")

# Vision (requires vision_base_url / ABSTRACTVISION_BASE_URL)
# png_bytes = llm.vision.t2i("a red square")

See Audio & Voice, Vision Capabilities, and Server.

✅ What AbstractCore Does Exceptionally Well

1. Universal LLM Provider Interface

What it does: Provides identical APIs across all major LLM providers.

# Same code works with any provider
def ask_llm(provider_name, question):
    llm = create_llm(provider_name, model="default")
    return llm.generate(question)

# All of these work identically
ask_llm("openai", "What is Python?")
ask_llm("anthropic", "What is Python?")
ask_llm("ollama", "What is Python?")

Key benefit: Identical API across all providers eliminates provider-specific code.

2. Production-Grade Reliability

What it does: Handles failures gracefully with retry logic, circuit breakers, and comprehensive error handling.

Automatic retries with exponential backoff for rate limits and network errors
Circuit breakers prevent cascade failures when providers go down
Smart error classification - retries recoverable errors, fails fast on auth errors
Event system for monitoring and alerting

Key benefit: Includes error handling, retries, and monitoring required for production use.

3. Universal Tool Calling

What it does: Tools work consistently across ALL providers, even those without native tool support.

from abstractcore import create_llm, tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a city."""
    return f"Weather in {city}: 72°F, sunny"

# Works with providers that have native tool support
openai_response = openai_llm.generate("Weather in Paris?", tools=[get_weather])

# Also works with providers that don't (via intelligent prompting)
ollama_response = ollama_llm.generate("Weather in Paris?", tools=[get_weather])

Key benefit: Works with providers that lack native tool support through intelligent prompting.

4. Centralized Configuration System

What it does: Single source of truth for all settings with clear priority hierarchy.

# One-time configuration
abstractcore --status
abstractcore --set-global-default ollama/qwen3:4b-instruct
abstractcore --set-app-default summarizer openai gpt-4o-mini
abstractcore --set-api-key openai sk-your-key-here
abstractcore --download-vision-model

# Now use anywhere without repeating config
summarizer document.pdf  # Uses configured defaults
python -m abstractcore.utils.cli --prompt "Hello"  # Uses CLI default

Key benefit: Configuration hierarchy (Explicit > App-Specific > Global > Default) reduces repetitive parameter passing.

5. Deterministic Generation

What it does: Provides consistent, reproducible outputs across all providers using seed parameters.

# Deterministic outputs with seed + temperature=0
llm = create_llm("openai", model="gpt-4o-mini", seed=42, temperature=0.0)

# These will produce identical outputs
response1 = llm.generate("Write exactly 3 words about coding")
response2 = llm.generate("Write exactly 3 words about coding")
# response1.content == response2.content (identical outputs)

# Works across providers with same interface
ollama_llm = create_llm("ollama", model="qwen3:4b-instruct", seed=42, temperature=0.0)
mlx_llm = create_llm("mlx", model="mlx-community/Qwen3-4B-4bit", seed=42, temperature=0.0)

Provider Support:

✅ Native SEED: OpenAI, Ollama, LMStudio, MLX, HuggingFace
⚠️ Warning: Anthropic (doesn't support seed but issues warning; use temperature=0.0 for consistency)

Key benefit: Consistent seed parameter support enables reproducible outputs for testing and debugging.

6. Universal Media Handling System

What it does: Attach files to your LLM requests with simple @filename syntax or media=[] parameter. Images and documents are handled automatically; audio/video inputs are supported via explicit policies (no silent semantic changes).

# Python API - works across ALL providers
llm = create_llm("openai", model="gpt-4o")
response = llm.generate(
    "Compare the chart with the data and summarize the document",
    media=["chart.png", "data.csv", "report.pdf"]
)

# CLI - same simplicity
# python -m abstractcore.utils.cli --prompt "Analyze @report.pdf and @chart.png"

# Supported: Images (PNG, JPEG, GIF, WEBP, BMP, TIFF)
#            Documents (PDF, DOCX, XLSX, PPTX)
#            Data (CSV, TSV, TXT, MD, JSON)
#            Audio (MP3, WAV, M4A, OGG, FLAC, AAC) via audio_policy (e.g. speech_to_text)
#            Video (MP4, MOV, MKV, WEBM, AVI, WMV) via video_policy (frames sampling)

Key benefit: Same code works across all providers with automatic format detection and processing, plus explicit policy controls for audio/video.

Audio is policy-driven by default: attaching an audio file will fail unless you opt into a fallback. For speech audio, use audio_policy="speech_to_text" (requires pip install abstractvoice). See Audio & Voice (STT/TTS).

7. Vision Capabilities with Intelligent Fallback

What it does: Image analysis across all providers with automatic resolution optimization. Text-only models can process images through transparent vision fallback.

# Works with vision-capable models
llm = create_llm("openai", model="gpt-4o")
response = llm.generate(
    "What's in this image?",
    media=["photo.jpg"]
)

# Also works with text-only models via vision fallback
text_llm = create_llm("lmstudio", model="qwen/qwen3-next-80b")  # No native vision
response = text_llm.generate(
    "Analyze this image",
    media=["complex_scene.jpg"]
)
# Transparent: vision model analyzes → text model processes description

Key benefit: Automatic image optimization and vision fallback enables image processing with text-only models.

8. Tool-Call Syntax Rewriting

What it does: Preserve/rewrite tool-call markup in assistant content when a downstream consumer needs text-based tool parsing.

llm = create_llm("openai", model="gpt-4o-mini")

# Qwen3 tags
response = llm.generate("Weather in Paris?", tools=tools, tool_call_tags="qwen3")
# Output: <|tool_call|>{"name": "get_weather", ...}

# LLaMA3 tags
response = llm.generate("Weather in Paris?", tools=tools, tool_call_tags="llama3")
# Output: {"name": "get_weather", ...}

# XML tags
response = llm.generate("Weather in Paris?", tools=tools, tool_call_tags="xml")
# Output: {"name": "get_weather", ...}

# Default (recommended): tool markup is stripped from content; use response.tool_calls instead.

Key benefit: Lets you interop with systems that only parse tool calls from text without changing tool definitions.

HTTP server note: If you consume tool calls via the OpenAI-compatible server, prefer agent_format (see Server Guide).

9. Structured Output with Automatic Retry

What it does: Gets typed Python objects from LLMs with automatic validation and retry on failures.

from pydantic import BaseModel

class Product(BaseModel):
    name: str
    price: float

# Automatically retries with error feedback if validation fails
product = llm.generate(
    "Extract: Gaming laptop for $1200",
    response_model=Product
)

Key benefit: Automatic validation and retry reduces manual error handling.

10. MCP Tool Discovery (Model Context Protocol)

What it does: Discover external tool schemas from an MCP server and inject them into any LLM call (MCP is a tool protocol, not a provider).

from abstractcore import create_llm
from abstractcore.mcp import McpClient, McpToolSource

llm = create_llm("openai", model="gpt-4o-mini")

client = McpClient(url="http://localhost:3000/mcp")  # example MCP endpoint
tool_specs = McpToolSource(server_id="local", client=client).list_tool_specs()

response = llm.generate("Use an external tool if needed.", tools=tool_specs)
print(response.tool_calls)  # names like: mcp::local::tool_name

Key benefit: Standardizes tool discovery across ecosystems while keeping execution explicit and host-owned.

Docs: MCP Tools

11. Communication Tools (Email + WhatsApp)

What it does: Optional side-effecting tools for email (SMTP/IMAP) and WhatsApp (Twilio) that you can expose to agents behind an approval boundary.

from abstractcore.tools.comms_tools import send_email, list_emails, read_email, send_whatsapp_message

# Pass tools like any other tool set (execution remains host/runtime-owned by default)
response = llm.generate("Send a daily digest email.", tools=[send_email])

Key benefit: Practical “real-world actions” with secrets resolved via env vars at execution time.

Docs: Communication Tools

❌ What AbstractCore Does NOT Do

Understanding limitations is crucial for choosing the right tool.

1. RAG Pipelines (Use Specialized Tools)

What AbstractCore provides: Vector embeddings via EmbeddingManager

What it doesn't provide: Document chunking, vector databases, retrieval strategies

# AbstractCore gives you this
from abstractcore.embeddings import EmbeddingManager
embedder = EmbeddingManager()
similarity = embedder.compute_similarity("query", "document")

# You need to build this yourself
def rag_pipeline(query, documents):
    # 1. Chunk documents - YOU implement
    # 2. Store in vector DB - YOU implement
    # 3. Retrieve relevant chunks - YOU implement
    # 4. Construct prompt - YOU implement
    return llm.generate(prompt)

Better alternatives:

LlamaIndex - Full RAG framework
LangChain - RAG components and chains

2. Complex Agent Workflows (Use Agent Frameworks)

What AbstractCore provides: Single LLM calls with tool execution

What it doesn't provide: Multi-step agent reasoning, planning, memory persistence

# AbstractCore is great for this
response = llm.generate("What's 2+2?", tools=[calculator_tool])

# AbstractCore is NOT for this
def complex_agent():
    # 1. Plan multi-step solution - NOT provided
    # 2. Execute steps with memory - NOT provided
    # 3. Reflect and re-plan - NOT provided
    # 4. Persist agent state - NOT provided
    pass

Better alternatives:

AbstractAgent - Built on AbstractCore
LangGraph - Agent orchestration
AutoGPT - Autonomous agents

When to Choose AbstractCore

✅ Choose AbstractCore When You Need:

Reliable LLM Infrastructure
- Production-ready error handling and retry logic
- Consistent interface across different providers
- Built-in monitoring and observability
Provider Flexibility
- Easy switching between OpenAI, Anthropic, Ollama, etc.
- Provider-agnostic code that runs anywhere
- Local and cloud provider support
Universal Tool Calling
- Tools that work across ALL providers
- Consistent tool execution regardless of native support
- Event-driven tool control and monitoring
Centralized Configuration
- Single source of truth for all settings
- Clear priority hierarchy (Explicit > App > Global > Default)
- No repetitive parameter passing
Universal Media Handling
- Attach images, PDFs, Office docs, data files
- Same API across all providers
- Automatic processing and formatting
Deterministic Generation
- Reproducible outputs with seed parameters
- Empirically verified across all providers
- Essential for testing and debugging
Vision Capabilities
- Image analysis across all providers
- Automatic resolution optimization
- Vision fallback for text-only models

❌ Don't Choose AbstractCore When You Need:

Full RAG Frameworks → Use LlamaIndex or LangChain
Complex Agent Workflows → Use AbstractAgent or LangGraph
Advanced Memory Systems → Use AbstractMemory or Mem0
Prompt Template Management → Use Jinja2 or LangChain Prompts
Model Training/Fine-tuning → Use Transformers or Axolotl
Multi-Agent Systems → Use CrewAI or AutoGen