← Back to all products

LLM Application Framework

$59

Build LLM-powered apps with prompt management, RAG pipelines, guardrails, evaluation harnesses, and cost tracking.

40 files
PythonYAMLMarkdownJSONLLM

Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free Sample

File Structure 40 files

llm-application-framework/ ├── LICENSE ├── README.md ├── configs/ │ ├── app_config.yaml │ └── guardrails_config.yaml ├── free-sample.zip ├── guide/ │ ├── 01-app-assembly.md │ └── 02-guardrails-guide.md ├── guides/ │ ├── app-assembly.md │ └── guardrails-guide.md ├── index.html ├── interactive.html ├── prompts/ │ ├── few_shot_examples.yaml │ └── system_prompts.yaml ├── requirements.txt ├── src/ │ └── llm_framework/ │ ├── __init__.py │ ├── __pycache__/ │ │ ├── __init__.cpython-312.pyc │ │ ├── app_builder.cpython-312.pyc │ │ ├── cost_tracker.cpython-312.pyc │ │ ├── evaluation.cpython-312.pyc │ │ ├── guardrails.cpython-312.pyc │ │ ├── mock_client.cpython-312.pyc │ │ ├── prompt_registry.cpython-312.pyc │ │ └── rag_pipeline.cpython-312.pyc │ ├── app_builder.py │ ├── cost_tracker.py │ ├── evaluation.py │ ├── guardrails.py │ ├── mock_client.py │ ├── prompt_registry.py │ └── rag_pipeline.py └── tests/ ├── __pycache__/ │ ├── test_guardrails.cpython-312-pytest-9.0.2.pyc │ ├── test_guardrails.cpython-312.pyc │ ├── test_prompt_registry.cpython-312-pytest-9.0.2.pyc │ ├── test_prompt_registry.cpython-312.pyc │ ├── test_rag.cpython-312-pytest-9.0.2.pyc │ └── test_rag.cpython-312.pyc ├── test_guardrails.py ├── test_prompt_registry.py └── test_rag.py

Documentation Preview README excerpt

LLM Application Framework

Build production LLM applications with prompt management, RAG pipelines, guardrails, evaluation harnesses, and cost tracking — using a mock LLM client that runs without API keys.

Assemble LLM apps the way an ML engineer would: modular pipeline stages, versioned prompts, measurable output quality, and explicit cost control. Every component works standalone or composes into a full application pipeline.

What You Get

  • Mock LLM client — deterministic LLM simulator for testing without API keys, with token counting, streaming simulation, and error injection for resilience testing
  • Prompt registry — versioned prompt templates with {{variable}} substitution, validation, fingerprinting, bulk loading from YAML, and composition (system + few-shot + context + query)
  • RAG pipeline — document chunking with configurable overlap, in-memory vector store with cosine similarity, token-budget-aware context assembly, and source citation
  • Guardrails — input validation (prompt injection detection, content length, topic filtering) and output validation (PII detection/redaction, format compliance) composable into chains
  • Evaluation harness — ROUGE-L, BLEU, word overlap, factual consistency scoring, answer relevance, format compliance, output length checks, and A/B comparison
  • Cost tracker — per-model pricing, running cost accumulation, budget alerts with callbacks, cost breakdown by model and prompt, and monthly cost estimation
  • App builder — pipeline orchestrator that assembles stages (guardrails → retrieval → prompt assembly → generation → output validation → cost tracking) into a complete application
  • Pre-built prompts — YAML prompt templates for QA, summarization, classification, entity extraction, code generation, chain-of-thought reasoning, and conversational assistants
  • Few-shot example sets — ready-to-use examples for sentiment classification, intent detection, entity extraction, summarization, and code review

File Tree


llm-application-framework/
├── README.md
├── LICENSE
├── requirements.txt
├── src/
│   └── llm_framework/
│       ├── __init__.py              # Package exports
│       ├── mock_client.py           # Deterministic LLM simulator
│       ├── prompt_registry.py       # Prompt templates and versioning
│       ├── rag_pipeline.py          # Document retrieval and context assembly
│       ├── guardrails.py            # Input/output validation and PII detection
│       ├── evaluation.py            # Output quality scoring (ROUGE-L, BLEU, etc.)
│       ├── cost_tracker.py          # Token usage and API cost tracking
│       └── app_builder.py           # Pipeline orchestrator
├── prompts/
│   ├── system_prompts.yaml          # Reusable prompt templates
│   └── few_shot_examples.yaml       # In-context learning examples
├── configs/
│   ├── app_config.yaml              # Application configuration
│   └── guardrails_config.yaml       # Guardrail rules and thresholds
├── tests/
│   ├── test_prompt_registry.py      # Template rendering and registry tests
│   ├── test_guardrails.py           # PII, injection, and chain tests
│   └── test_rag.py                  # Chunking, vector store, pipeline tests
└── guides/
    ├── app-assembly.md              # Step-by-step app building guide
    └── guardrails-guide.md          # Guardrails configuration and tuning

Quick Start

1. Install Dependencies


pip install -r requirements.txt

... continues with setup instructions, usage examples, and more.

Code Sample .py preview

src/llm_framework/app_builder.py """ App Builder — LLM Application Framework =========================================== Orchestrate the assembly of LLM applications from components: prompts, retrieval, guardrails, generation, evaluation, and cost tracking. This is the top-level module that ties everything together. The app builder follows the pipeline pattern: each request flows through a sequence of stages, and each stage can be configured independently. The key insight is that LLM apps are NOT just "send prompt → get response". Real apps need: Input validation → Context retrieval → Prompt assembly → LLM generation → Output validation → Cost tracking → Logging This module makes that pipeline explicit and configurable. By Datanest Digital — [email protected] """ from __future__ import annotations import json import logging import time from dataclasses import dataclass, field from typing import Any, Callable, Dict, List, Optional, Tuple logger = logging.getLogger(__name__) # Import sibling modules from llm_framework.mock_client import ChatMessage, CompletionResponse, MockLLMClient from llm_framework.prompt_registry import PromptRegistry, PromptTemplate, compose_prompt from llm_framework.guardrails import GuardrailChain, GuardrailResult, GuardrailAction from llm_framework.cost_tracker import CostTracker from llm_framework.rag_pipeline import RAGPipeline, RAGResponse # --------------------------------------------------------------------------- # App request/response types # --------------------------------------------------------------------------- @dataclass class AppRequest: """An incoming request to the LLM application. Attributes: query: User's input/question. user_id: Identifier for the requesting user (for rate limiting, logging). # ... 473 more lines ...