Every tool for people who work in prompts.

Agent & Prompt Frameworks

A library for building LLM applications that also auto-tunes them, optimizing prompt text and few-shot demonstrations against a training set. It aims to make prompt tuning as routine as model training.

Agent & Prompt Frameworks

A Python framework for role-playing agent crews, where each agent is defined by a role, goal and backstory that become its prompt. Tasks are assigned to agents and executed sequentially or hierarchically.

Evaluation & Testing

A pytest-style evaluation framework for LLM output with ready-made metrics for hallucination, answer relevancy, faithfulness and task completion. Tests run locally and can be pushed to the Confident AI platform.

Agent & Prompt Frameworks

A Python framework that replaces hand-written prompt strings with declarative modules and signatures, then compiles them against your data. Optimizers search for the wording and few-shot examples that score best on a metric you define.

Agent & Prompt Frameworks

The official implementation of a paper that applies evolutionary algorithms to prompt search, using a language model to perform crossover and mutation on candidate prompts. Populations are scored on a dev set each generation.

Evaluation & Testing

A vulnerability scanner for language models that probes a target with hundreds of attack prompts covering jailbreaks, prompt injection, data leakage and toxic output, then reports which probes succeeded.

Agent & Prompt Frameworks

An optimizer that improves prompts and other text artifacts through reflective evolution, reading execution traces and natural-language feedback to propose edits. It integrates with DSPy and other pipelines.

Agent & Prompt Frameworks

A templating language for steering language models token by token, interleaving generation with control flow, regex and grammar constraints. Output structure is enforced during decoding rather than checked afterwards.

Agent & Prompt Frameworks

deepset's orchestration framework for retrieval and agent pipelines built from composable components. Prompt builders are pipeline nodes, so prompt text is versioned with the rest of the pipeline definition.

Agent & Prompt Frameworks

A library for building LLM applications that also auto-tunes them, optimizing prompt text and few-shot demonstrations against a training set. It aims to make prompt tuning as routine as model training.

Agent & Prompt Frameworks

A Python framework for role-playing agent crews, where each agent is defined by a role, goal and backstory that become its prompt. Tasks are assigned to agents and executed sequentially or hierarchically.

Evaluation & Testing

A pytest-style evaluation framework for LLM output with ready-made metrics for hallucination, answer relevancy, faithfulness and task completion. Tests run locally and can be pushed to the Confident AI platform.

Agent & Prompt Frameworks

A Python framework that replaces hand-written prompt strings with declarative modules and signatures, then compiles them against your data. Optimizers search for the wording and few-shot examples that score best on a metric you define.

Agent & Prompt Frameworks

The official implementation of a paper that applies evolutionary algorithms to prompt search, using a language model to perform crossover and mutation on candidate prompts. Populations are scored on a dev set each generation.

Evaluation & Testing

A vulnerability scanner for language models that probes a target with hundreds of attack prompts covering jailbreaks, prompt injection, data leakage and toxic output, then reports which probes succeeded.

Agent & Prompt Frameworks

An optimizer that improves prompts and other text artifacts through reflective evolution, reading execution traces and natural-language feedback to propose edits. It integrates with DSPy and other pipelines.

Agent & Prompt Frameworks

A templating language for steering language models token by token, interleaving generation with control flow, regex and grammar constraints. Output structure is enforced during decoding rather than checked afterwards.

Agent & Prompt Frameworks

deepset's orchestration framework for retrieval and agent pipelines built from composable components. Prompt builders are pipeline nodes, so prompt text is versioned with the rest of the pipeline definition.