← Back to all products

Hyperparameter Tuning Kit

$29

Optuna/Ray Tune configs, search space definitions, pruning strategies, and distributed tuning setups.

18 filesv1.0.0
PythonMarkdownJSON

Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free Sample

File Structure 18 files

hyperparameter-tuning-kit/ ├── LICENSE ├── README.md ├── examples/ │ ├── optuna_sklearn.py │ ├── random_search_baseline.py │ └── ray_tune_torch.py ├── guides/ │ ├── hpo-guide.md │ └── search-space-design.md ├── src/ │ └── hpo/ │ ├── __init__.py │ ├── analysis.py │ ├── objective.py │ ├── optuna_search.py │ ├── pruning.py │ ├── ray_tune_search.py │ └── search_spaces.py └── tests/ ├── test_objective.py ├── test_pruning.py └── test_search_spaces.py

Documentation Preview README excerpt

Hyperparameter Tuning Kit

A practical hyperparameter optimization toolkit built around a single idea: define

your search space once and run it on both Optuna and Ray Tune without

rewriting anything. Ships with cross-validated objectives, median/percentile

pruning, ASHA/PBT scheduling, multi-objective support, and backend-agnostic result

analysis.

The core (search spaces, pruning, objectives, analysis) depends only on the Python

standard library — it imports and its full test suite runs with no third-party

packages at all. Optuna and Ray Tune are imported lazily, so you install only the

backend you actually use.

Features

  • One declarative search space, two backends. SearchSpace compiles to the

Optuna suggest API and to a Ray Tune param_space dict — no drift between them.

  • Typed parameters — FloatParameter, IntParameter, CategoricalParameter

with log scaling, step grids, and validation that rejects impossible ranges.

  • Cross-validated objectives — Objective wraps stdlib K-fold CV with a

per-fold pruning hook and first-class multi-objective support.

  • Pruning in plain Python — MedianPruner, PercentilePruner,

ThresholdPruner, and EarlyStopping, with startup/warmup/interval gates you can

unit-test and reason about (and convert to real Optuna pruners on demand).

  • Optuna runner — samplers (TPE, CMA-ES, QMC, random, grid), pruning, SQLite

persistence, resume, and multi-objective studies via a declarative StudyConfig.

  • Ray Tune runner — ASHA, Population Based Training, and median stopping via a

declarative TuneConfig, plus per-trial resources.

  • Result analysis — best/top-k trials, a fast parameter-importance proxy,

Pareto fronts, convergence history, and CSV/JSON export over a single

TrialRecord type that works regardless of backend.

  • A zero-dependency example that runs the moment you unzip the kit.

What's included


hyperparameter-tuning-kit/
├── src/hpo/
│   ├── search_spaces.py     # SearchSpace + typed params + RandomTrial (stdlib)
│   ├── pruning.py           # median/percentile/threshold pruners + early stopping
│   ├── objective.py         # scorers, K-fold CV, Objective (multi-objective)
│   ├── analysis.py          # best/top-k, importance, Pareto, export (stdlib)
│   ├── optuna_search.py     # StudyConfig + run_study (lazy `optuna`)
│   └── ray_tune_search.py   # TuneConfig + run_tune, ASHA/PBT (lazy `ray`)
├── examples/
│   ├── random_search_baseline.py  # runs on stdlib alone — start here
│   ├── optuna_sklearn.py          # tunes a GradientBoostingClassifier with Optuna
│   └── ray_tune_torch.py          # tunes a PyTorch MLP with Ray Tune + ASHA
├── guides/
│   ├── hpo-guide.md               # samplers, pruning, schedulers, budgets, pitfalls
│   └── search-space-design.md     # log scales, ranges, conditionals, worked spaces
├── tests/
│   ├── test_search_spaces.py
│   ├── test_pruning.py
│   └── test_objective.py
├── README.md
└── LICENSE

Quick start

... continues with setup instructions, usage examples, and more.

Code Sample .py preview

src/hpo/analysis.py """Trial analysis: best-trial extraction, importance, Pareto fronts, and export. After a study finishes you need to answer four questions: *which configuration won?*, *which hyperparameters actually mattered?*, *for multi-objective runs, what is the trade-off frontier?*, and *how do I get all of this out as CSV/JSON for a report?* This module answers them over a backend-agnostic :class:`TrialRecord`, so the same code works whether the trials came from Optuna, Ray Tune, or the bundled random-search loop. :func:`from_optuna_study` converts a real Optuna study into ``TrialRecord`` objects (lazy import); everything else is pure standard library. The importance estimator here is a fast, explainable variance-reduction proxy (a one-split regression stump for numeric params, eta-squared grouping for categoricals) -- not Optuna's fANOVA. It is meant for quick "what should I widen or freeze next?" decisions, and it is deterministic and dependency-free. """ from __future__ import annotations import csv import json import math import statistics from dataclasses import dataclass, field from pathlib import Path from typing import Any, Iterable, Mapping, Sequence __all__ = [ "COMPLETE", "PRUNED", "FAIL", "TrialRecord", "best_trial", "top_k", "optimization_history", "param_importance", "pareto_front", "summary", "export_csv", "export_json", "from_optuna_study", ] COMPLETE = "COMPLETE" PRUNED = "PRUNED" FAIL = "FAIL" @dataclass # ... 323 more lines ...