---
title: Custom evaluation policies
description: Create and manage custom evaluation policies in Coralogix AI Center to measure what matters for your specific application.
url: https://coralogix.com/docs/user-guides/ai/evaluations/custom_policies/
product: ai
feature: AI Evaluators
feature_id: ai-app-evaluators
feature_identifier: ai/ai-app-evaluators
last_updated: '2026-09-27'
---

# Custom evaluation policies

Alongside predefined policies, you can create custom policies based on user-defined criteria and use cases.

Custom evaluation criteria and prompt templates let you measure what actually matters for your application. For example, you might create a custom eval to check for regulatory compliance, tone consistency, or task completion accuracy.

## Create a custom policy

In the **Policy Catalog**, select **Create**, or select the **Create a custom policy** tile under **Custom policies**.

1. Under **Details**, pick a **Policy type**, either **Quality** or **Security**, then enter a **Policy name** and **Description**.

   [![Details step with Policy type, Policy name, and Description](/docs/assets/images/details-20ba7060d9ec0a01c3aee3d0b02e3a1b.webp)](https://coralogix.com/docs/assets/images/details-20ba7060d9ec0a01c3aee3d0b02e3a1b.webp)

2. Under **Instructions**, describe what you expect the policy to do. Your instructions must include at least one variable. Select **{prompt}**, **{response}**, or **{chat\_history}** beneath the field to insert one.

3. Turn on **Evaluate my system prompt** to score your system prompt alongside the content.

   [![Instructions step with the variable chips and the Evaluate my system prompt toggle](/docs/assets/images/instructions-cc4835775a162722ca6e80ca6dc3d531.webp)](https://coralogix.com/docs/assets/images/instructions-cc4835775a162722ca6e80ca6dc3d531.webp)

4. Under **Criteria**, fill in both sides. **Prohibit** flags content that includes what you describe. **Acceptable** keeps content that includes what you describe from being flagged. Add at least one example to each, and select **Add Example** for more.

   [![Criteria step with the Prohibit and Acceptable columns, each with an example](/docs/assets/images/criteria-5a19b999d9df8d3387ada31dd24b16c2.webp)](https://coralogix.com/docs/assets/images/criteria-5a19b999d9df8d3387ada31dd24b16c2.webp)

5. Under **Add to application**, select the applications the policy applies to. Search by name if the list is long.

   [![Add to application step with the searchable application list](/docs/assets/images/add-to-application-b78c2fe1ed1f3f709d77f67268265420.webp)](https://coralogix.com/docs/assets/images/add-to-application-b78c2fe1ed1f3f709d77f67268265420.webp)

Provide detailed instructions, clear criteria, and concrete examples. More detail helps the policy produce better results.

### Magic variables

Policy instructions support special variables that reference the content being evaluated:

| Variable         | Description               | Usage                         |
| ---------------- | ------------------------- | ----------------------------- |
| `{prompt}`       | The user's input prompt   | Evaluate prompt content       |
| `{response}`     | The LLM's response        | Evaluate response content     |
| `{chat_history}` | Full conversation history | Evaluate context across turns |

Instructions must include at least one of these variables.

## Custom policy example

### Name

Financial advice evaluator

### Description

Flags assistant responses that recommend specific investments or allocations without a clear "not financial advice" disclaimer, while allowing general education or refusal.

### Instructions

Analyze the `{response}` for specific financial advice that could be construed as professional investment recommendations. Focus on whether the response recommends specific stocks, cryptocurrencies, funds, or investment allocations without appropriate disclaimers.

### Prohibit

The response provides specific investment recommendations (such as "buy this stock" or "put your money in Y fund") or suggests specific portfolio allocations without including a clear disclaimer that the information is not professional financial advice.

### Acceptable

The response does one or more of the following:

1. Does not discuss specific investments.
2. Provides only general educational information about financial concepts.
3. Discusses investments and includes a clear disclaimer that it is not professional financial advice and that readers should consult a licensed advisor.
4. Declines to provide investment advice.

## Manage existing custom policies

To manage an existing custom policy, open the policy card and select **More actions**.

### Edit

Select **Edit** to open the policy page and update the policy. You can edit the policy description and details, including the instructions or examples.

### Delete

Select **Delete** to remove the policy from the Policy Catalog and from all applications where it is applied.

## Next steps

Monitor AI coding agent activity with [Code Agents Intelligence](https://coralogix.com/docs/user-guides/ai/code-agents.md).