ILM & Reindexing Automation Runbooks

Elasticsearch ILM Policy JSON Template for Beginners

You need a single, copy-paste Index Lifecycle Management policy that carries a time-series index from active ingest through cheaper tiers and finally into deletion — this page is that template, annotated field by field, applied with the v8 client, and verified so you can prove it took effect rather than hope it did.

Where this fits in the ILM lifecycle

An ILM policy is a declarative document that names a set of phases (hot, warm, cold, delete) and the actions each phase runs. Elasticsearch stores it as a single cluster-level object and its always-on coordinator walks every managed index through those phases on its own schedule. A beginner’s first policy usually fails not because the JSON is malformed but because a field means something subtler than it looks — min_age is measured from the wrong moment, or shrink demands a precondition that was never set. This template is the safe baseline the parent guide on securing ILM policies with RBAC refers to as the version-controlled artifact a scoped service account is allowed to apply: pin these fields, keep the document under source control, and a well-bounded token can deploy it to every cluster without drift.

The phase progression the template encodes is the same hot-warm-cold architecture that governs where shards physically live, and its first action depends on the rollover conditions that decide when a hot index is cut. Once you understand this template you can graduate to authoring policies programmatically, covered in building custom ILM policies via the API.

hot → warm → cold → delete: the template's phase progression and min_age clockFour phase cards left to right. Hot, min_age 0ms: rollover 1d · 50gb · 10M docs, set_priority 100. Warm, min_age 2d: forcemerge 1 segment, shrink to 1 shard, allocate data:warm, set_priority 50. Cold, min_age 7d: allocate data:cold, replicas 0, set_priority 10. Delete, min_age 30d (drawn dashed as the terminal phase): delete the index. Below the cards a left-to-right time axis anchored at the rollover event marks the four min_age thresholds at 0ms, 2d, 7d and 30d, with the caption that min_age counts from rollover, not index creation.One policy, four phases — hot → warm → cold → deleteHOTmin_age 0msrollover1d · 50gb · 10Mset_priority 100recover writes firstWARMmin_age 2dforcemerge 1 segshrink → 1 shardallocate data:warmset_priority 50COLDmin_age 7dallocate data:coldreplicas 0set_priority 10cheapest tierDELETEmin_age 30ddelete { }index removedfrom the deployment⟳ rollover0ms2d7d30dEverymin_ageis measured from the rollover event — not from index creation.

Prerequisites

  • Elasticsearch 8.x with the ILM coordinator running — confirm GET _ilm/status reports "operation_mode": "RUNNING" before you expect any phase to advance.
  • elasticsearch-py v8.0+ if you apply the policy from Python; this page uses the v8 surface (ilm.put_lifecycle, ilm.explain_lifecycle, keyword-argument bodies), not the legacy body= pattern.
  • Node data-tier roles assigned (data_hot, data_warm, data_cold) so the allocate action has real tiers to route to — a policy that references a tier with no matching nodes stalls instead of migrating.
  • An index template that attaches the policy through index.lifecycle.name and names a rollover_alias, so a new index is managed the moment it is created.
  • A token with cluster manage_ilm to create the policy — index-level manage alone returns 403 on PUT _ilm/policy.

The policy template

The document below enforces a deterministic hot → warm → cold → delete progression. Each field is annotated with the behaviour it controls. JSON has no native comments, so strip the // lines before sending the request, or send the clean version with the Python helper further down.

PUT _ilm/policy/log-analytics-lifecycle
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_primary_shard_size": "50gb",
            "max_docs": 10000000
          },
          "set_priority": { "priority": 100 }
        }
      },
      "warm": {
        "min_age": "2d",
        "actions": {
          "forcemerge": { "max_num_segments": 1 },
          "shrink": { "number_of_shards": 1 },
          "set_priority": { "priority": 50 },
          "allocate": {
            "require": { "data": "warm" },
            "number_of_replicas": 1
          }
        }
      },
      "cold": {
        "min_age": "7d",
        "actions": {
          "set_priority": { "priority": 10 },
          "allocate": {
            "require": { "data": "cold" },
            "number_of_replicas": 0
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Field-by-field notes that trip up first deployments

  • Prefer max_primary_shard_size over max_size. max_size evaluates the whole index including replicas, so a replica-count change can trigger a premature rollover. max_primary_shard_size measures a single primary and stays deterministic regardless of replica topology — it is the field that keeps shard sizing predictable.
  • min_age is relative to the rollover, not index creation. For a rollover-managed index, each phase’s min_age counts from the moment the index rolled over, not from when documents first landed. A warm min_age of 2d means “two days after this index stopped being the write target,” which is why a busy index can sit in hot far longer than two days.
  • set_priority prevents recovery storms. Without explicit priorities, warm and cold shards compete equally with hot shards when a node restarts, starving live ingest. Descending priorities (100 → 50 → 10) make Elasticsearch recover the tier that serves writes first.
  • shrink and allocate have hard preconditions. shrink needs the index read-only and its primaries collocated on one node (ILM sets the write block automatically); allocate needs nodes that actually carry the data: warm/data: cold attribute. Miss either and the phase parks in an ERROR step.

Applying the template from Python v8+

Applying the policy from code keeps it idempotent and version-controlled — the same call creates or updates the policy in place. Note the v8 method is ilm.put_lifecycle(name=..., policy=...); there is no legacy body= wrapper.

import logging
from elasticsearch import Elasticsearch, ApiError

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger("ilm_bootstrap")

es = Elasticsearch(
    "https://es-prod-node-01:9200",
    api_key="YOUR_API_KEY",     # a token scoped to manage_ilm, never a superuser
    verify_certs=True,
)

POLICY = {
    "phases": {
        "hot": {
            "min_age": "0ms",
            "actions": {
                "rollover": {
                    "max_age": "1d",
                    "max_primary_shard_size": "50gb",
                    "max_docs": 10_000_000,
                },
                "set_priority": {"priority": 100},
            },
        },
        "warm": {
            "min_age": "2d",
            "actions": {
                "forcemerge": {"max_num_segments": 1},
                "shrink": {"number_of_shards": 1},
                "set_priority": {"priority": 50},
                "allocate": {"require": {"data": "warm"}, "number_of_replicas": 1},
            },
        },
        "cold": {
            "min_age": "7d",
            "actions": {
                "set_priority": {"priority": 10},
                "allocate": {"require": {"data": "cold"}, "number_of_replicas": 0},
            },
        },
        "delete": {"min_age": "30d", "actions": {"delete": {}}},
    }
}

try:
    # put_lifecycle is create-or-replace: safe to re-run from CI on every deploy.
    es.ilm.put_lifecycle(name="log-analytics-lifecycle", policy=POLICY)
    logger.info("Policy 'log-analytics-lifecycle' applied.")
except ApiError as exc:
    # A 403 here almost always means the token lacks cluster manage_ilm.
    logger.error("Failed to apply policy: %s %s", exc.meta.status, exc.body)
    raise

Verification

Applying a policy proves only that the JSON was accepted, not that any index is following it. First read the stored policy back to confirm the phases persisted exactly as intended:

GET _ilm/policy/log-analytics-lifecycle

Then check an index the policy actually governs — this is the authoritative signal. GET <index>/_ilm/explain reports which phase, action, and step the index currently occupies, and "managed": true confirms the policy attached at all:

GET logs-analytics-000001/_ilm/explain

A healthy index sitting in the hot phase, waiting for its first rollover condition, returns:

{
  "indices": {
    "logs-analytics-000001": {
      "index": "logs-analytics-000001",
      "managed": true,
      "policy": "log-analytics-lifecycle",
      "phase": "hot",
      "action": "rollover",
      "step": "check-rollover-ready"
    }
  }
}

"managed": false is a finding, not a pass: the index inherited template settings but never had index.lifecycle.name applied, so every phase transition is silently ignored. A step of ERROR means a phase precondition failed — read failed_step and step_info.reason to see which one, a workflow covered in depth under monitoring ILM execution and error states.

Gotchas and edge cases

  • The allocate tier must exist before the phase runs. "require": { "data": "warm" } only works if nodes carry that attribute. On a single-tier cluster with no data_warm nodes, the warm phase parks in check-allocation forever. Either assign the node roles or replace allocate with a fallback routing strategy that keeps shards placed instead of stalling.
  • shrink needs number_of_shards to divide the current count. You cannot shrink 5 primaries to 2 — the target must be a factor of the source (5→1 is valid, 5→2 is not). A non-divisor target fails the shrink step with an illegal_argument_exception.
  • number_of_replicas: 0 in cold trades safety for cost. Dropping to zero replicas halves storage but means a single node failure loses those shards permanently. Only set it where the data is reproducible or already snapshotted; otherwise keep at least one replica in cold.
  • The delete min_age is not a retention SLA on its own. Because min_age counts from rollover, an index that rolls late deletes late. If a compliance window is measured from ingest time, add headroom or monitor time-in-phase rather than trusting the 30d value to map exactly to calendar retention.

Frequently Asked Questions

Does min_age count from when the index was created?

No — for a rollover-managed index each phase's min_age is measured from the moment the index rolled over (stopped being the write target), not from index creation or first document. That is why a high-volume index can sit in the hot phase well past its warm min_age: the clock only starts once rollover fires. If you need age measured from creation, do not use rollover in the hot phase.

Can I apply this policy to an index that already has data?

Yes. Set index.lifecycle.name on the existing index (or on the template it was created from) and ILM adopts it on the next poll. Confirm with GET <index>/_ilm/explain that "managed" is true. Note that phases evaluate from that point forward; an old index may jump straight to a later phase if its age already exceeds the earlier phases' min_age values.

Why does my policy validate but no index ever moves?

Two usual causes. First, the index is not actually managed — _ilm/explain shows "managed": false because index.lifecycle.name was never applied. Second, the coordinator is stopped — GET _ilm/status returns something other than RUNNING, so nothing advances cluster-wide. Rule out both before debugging the policy JSON itself.

Do I need every phase, or can I start with just hot and delete?

You can omit any phase. A minimal policy with only hot (rollover) and delete is a perfectly valid starting point for logs that do not need tiering. Add warm and cold later when you have the node roles to support them — ILM simply skips phases that are not defined.

← Back to Securing ILM Policies with RBAC · ILM Architecture & Fundamentals