← Back to all products
$59
LLM Application Framework
Build LLM-powered apps with prompt management, RAG pipelines, guardrails, evaluation harnesses, and cost tracking.
PythonYAMLMarkdownJSONLLM
Product Preview
Try the interactive reader and demo tools below, or get the full product with all content unlocked.
Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free SampleFile Structure 40 files
llm-application-framework/
├── LICENSE
├── README.md
├── configs/
│ ├── app_config.yaml
│ └── guardrails_config.yaml
├── free-sample.zip
├── guide/
│ ├── 01-app-assembly.md
│ └── 02-guardrails-guide.md
├── guides/
│ ├── app-assembly.md
│ └── guardrails-guide.md
├── index.html
├── interactive.html
├── prompts/
│ ├── few_shot_examples.yaml
│ └── system_prompts.yaml
├── requirements.txt
├── src/
│ └── llm_framework/
│ ├── __init__.py
│ ├── __pycache__/
│ │ ├── __init__.cpython-312.pyc
│ │ ├── app_builder.cpython-312.pyc
│ │ ├── cost_tracker.cpython-312.pyc
│ │ ├── evaluation.cpython-312.pyc
│ │ ├── guardrails.cpython-312.pyc
│ │ ├── mock_client.cpython-312.pyc
│ │ ├── prompt_registry.cpython-312.pyc
│ │ └── rag_pipeline.cpython-312.pyc
│ ├── app_builder.py
│ ├── cost_tracker.py
│ ├── evaluation.py
│ ├── guardrails.py
│ ├── mock_client.py
│ ├── prompt_registry.py
│ └── rag_pipeline.py
└── tests/
├── __pycache__/
│ ├── test_guardrails.cpython-312-pytest-9.0.2.pyc
│ ├── test_guardrails.cpython-312.pyc
│ ├── test_prompt_registry.cpython-312-pytest-9.0.2.pyc
│ ├── test_prompt_registry.cpython-312.pyc
│ ├── test_rag.cpython-312-pytest-9.0.2.pyc
│ └── test_rag.cpython-312.pyc
├── test_guardrails.py
├── test_prompt_registry.py
└── test_rag.py
Documentation Preview README excerpt
LLM Application Framework
Build production LLM applications with prompt management, RAG pipelines, guardrails, evaluation harnesses, and cost tracking — using a mock LLM client that runs without API keys.
Assemble LLM apps the way an ML engineer would: modular pipeline stages, versioned prompts, measurable output quality, and explicit cost control. Every component works standalone or composes into a full application pipeline.
What You Get
- Mock LLM client — deterministic LLM simulator for testing without API keys, with token counting, streaming simulation, and error injection for resilience testing
- Prompt registry — versioned prompt templates with
{{variable}}substitution, validation, fingerprinting, bulk loading from YAML, and composition (system + few-shot + context + query) - RAG pipeline — document chunking with configurable overlap, in-memory vector store with cosine similarity, token-budget-aware context assembly, and source citation
- Guardrails — input validation (prompt injection detection, content length, topic filtering) and output validation (PII detection/redaction, format compliance) composable into chains
- Evaluation harness — ROUGE-L, BLEU, word overlap, factual consistency scoring, answer relevance, format compliance, output length checks, and A/B comparison
- Cost tracker — per-model pricing, running cost accumulation, budget alerts with callbacks, cost breakdown by model and prompt, and monthly cost estimation
- App builder — pipeline orchestrator that assembles stages (guardrails → retrieval → prompt assembly → generation → output validation → cost tracking) into a complete application
- Pre-built prompts — YAML prompt templates for QA, summarization, classification, entity extraction, code generation, chain-of-thought reasoning, and conversational assistants
- Few-shot example sets — ready-to-use examples for sentiment classification, intent detection, entity extraction, summarization, and code review
File Tree
llm-application-framework/
├── README.md
├── LICENSE
├── requirements.txt
├── src/
│ └── llm_framework/
│ ├── __init__.py # Package exports
│ ├── mock_client.py # Deterministic LLM simulator
│ ├── prompt_registry.py # Prompt templates and versioning
│ ├── rag_pipeline.py # Document retrieval and context assembly
│ ├── guardrails.py # Input/output validation and PII detection
│ ├── evaluation.py # Output quality scoring (ROUGE-L, BLEU, etc.)
│ ├── cost_tracker.py # Token usage and API cost tracking
│ └── app_builder.py # Pipeline orchestrator
├── prompts/
│ ├── system_prompts.yaml # Reusable prompt templates
│ └── few_shot_examples.yaml # In-context learning examples
├── configs/
│ ├── app_config.yaml # Application configuration
│ └── guardrails_config.yaml # Guardrail rules and thresholds
├── tests/
│ ├── test_prompt_registry.py # Template rendering and registry tests
│ ├── test_guardrails.py # PII, injection, and chain tests
│ └── test_rag.py # Chunking, vector store, pipeline tests
└── guides/
├── app-assembly.md # Step-by-step app building guide
└── guardrails-guide.md # Guardrails configuration and tuning
Quick Start
1. Install Dependencies
pip install -r requirements.txt
... continues with setup instructions, usage examples, and more.
Code Sample .py preview
src/llm_framework/app_builder.py
"""
App Builder — LLM Application Framework
===========================================
Orchestrate the assembly of LLM applications from components:
prompts, retrieval, guardrails, generation, evaluation, and cost
tracking. This is the top-level module that ties everything together.
The app builder follows the pipeline pattern: each request flows
through a sequence of stages, and each stage can be configured
independently. The key insight is that LLM apps are NOT just
"send prompt → get response". Real apps need:
Input validation → Context retrieval → Prompt assembly →
LLM generation → Output validation → Cost tracking → Logging
This module makes that pipeline explicit and configurable.
By Datanest Digital — [email protected]
"""
from __future__ import annotations
import json
import logging
import time
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
# Import sibling modules
from llm_framework.mock_client import ChatMessage, CompletionResponse, MockLLMClient
from llm_framework.prompt_registry import PromptRegistry, PromptTemplate, compose_prompt
from llm_framework.guardrails import GuardrailChain, GuardrailResult, GuardrailAction
from llm_framework.cost_tracker import CostTracker
from llm_framework.rag_pipeline import RAGPipeline, RAGResponse
# ---------------------------------------------------------------------------
# App request/response types
# ---------------------------------------------------------------------------
@dataclass
class AppRequest:
"""An incoming request to the LLM application.
Attributes:
query: User's input/question.
user_id: Identifier for the requesting user (for rate limiting, logging).
# ... 473 more lines ...