ILM & Reindexing Automation Runbooks

Auditing Indices with No ILM Policy Attached

Find the indices that slipped through with no index.lifecycle.name set — the ones ILM ignores completely, that will never roll over, migrate to a colder tier, or delete, and that grow unchecked until a disk watermark blocks writes.

An unmanaged index is the quietest form of policy drift. Nothing errors, no step wedges, no alert fires — the index simply sits outside the lifecycle state machine while everyone assumes retention is handled. This audit is one of the three signals in detecting ILM policy drift, and it exists because a reindex, a hand-created index, or a template gap can drop an index out of the versioned, idempotent lifecycle contract that ILM policy design and lifecycle synchronization is built to guarantee. The working example remains the metrics-* pattern that should be governed by the metrics-ilm policy.

Prerequisites

  • Elasticsearch 8.x; a canonical policy (metrics-ilm) already applied and a template intended to attach it.
  • elasticsearch-py v8.0+ — the scan uses client.indices.get_settings with filter_path.
  • monitor and view_index_metadata to scan; manage on metrics-* to attach a policy to any unmanaged index you find.
  • An understanding that data-stream backing indices are managed through the stream, not by a direct setting on each backing index.

Implementation

The absence you are hunting for is the settings key index.lifecycle.name. Any metrics-* index whose settings lack it is unmanaged. Pull just the relevant keys with filter_path so the payload stays small even on a large pattern:

GET metrics-*/_settings?filter_path=**.settings.index.lifecycle.name,**.settings.index.provided_name

Indices that appear in the full index list but are missing from this filtered response have no lifecycle name — that gap is the unmanaged set. The Python scan below reconciles the full list against the filtered settings and returns the unmanaged names, skipping dot-prefixed system and hidden indices.

import logging
from elasticsearch import Elasticsearch, ApiError

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger(__name__)


def find_unmanaged(es: Elasticsearch, index_pattern: str = "metrics-*") -> list[dict]:
    """Return metrics-* indices that carry no index.lifecycle.name setting."""
    settings = es.indices.get_settings(
        index=index_pattern,
        filter_path="**.settings.index.lifecycle.name,**.settings.index.provided_name",
        expand_wildcards="open",
    )
    # Full membership list so indices missing entirely from `settings` still surface.
    all_indices = es.indices.get(index=index_pattern, filter_path="*").keys()

    unmanaged = []
    for name in all_indices:
        if name.startswith("."):
            continue  # skip system / hidden indices (.security, .ds-*, monitoring)
        idx = settings.get(name, {}).get("settings", {}).get("index", {})
        if not idx.get("lifecycle", {}).get("name"):
            unmanaged.append({"index": name, "provided_name": idx.get("provided_name", name)})
    return unmanaged


def attach_policy(es: Elasticsearch, index: str, policy_name: str = "metrics-ilm") -> None:
    """Bring a single unmanaged index under management. Do NOT set rollover_alias here
    unless this index is genuinely the write index behind that alias."""
    try:
        es.indices.put_settings(
            index=index,
            settings={"index.lifecycle.name": policy_name},
        )
        logger.info("Attached '%s' to index '%s'.", policy_name, index)
    except ApiError as exc:
        logger.error("Attach failed on '%s' (%s): %s", index, exc.status_code, exc.info)
        raise


if __name__ == "__main__":
    es = Elasticsearch(
        "https://es-cluster-01:9200",
        api_key="YOUR_BASE64_ENCODED_API_KEY",
        verify_certs=True,
    )
    orphans = find_unmanaged(es, "metrics-*")
    logger.info("Unmanaged indices: %s", [o["index"] for o in orphans])
    # Attach a policy to each. Review the list before writing in production.
    for entry in orphans:
        attach_policy(es, entry["index"], "metrics-ilm")

Attaching via put_settings fixes the existing orphans immediately. To stop new ones being created, fix the template so future indices inherit index.lifecycle.name at creation — but understand that a template change only helps indices created after it lands. The two fixes are complementary: patch the template to prevent recurrence, and put_settings (or reindex) to rescue what already exists. Bootstrapping the template correctly is covered in bootstrapping index templates and data streams.

Verification

Re-scan and confirm the audit returns an empty list, then confirm each rescued index reports managed: true and a real phase through the explain API:

GET metrics-000007/_ilm/explain?only_managed=true
{
  "indices": {
    "metrics-000007": {
      "index": "metrics-000007",
      "managed": true,
      "policy": "metrics-ilm",
      "phase": "hot",
      "action": "complete",
      "step": "complete"
    }
  }
}

managed: true with a named policy is the proof the attach took. An index that still reports managed: false after a put_settings either had the wrong name written or is a dot-prefixed index the scan (correctly) skipped.

Gotchas and edge cases

  • Fixing the template only helps new indices. A composable template with index.lifecycle.name set applies at index creation — it never retroactively attaches to indices that already exist. Existing orphans always need an explicit put_settings (or a reindex into a managed destination). Patch both, or you will keep finding old unmanaged indices after “fixing” the template.
  • Do not set rollover_alias on a non-write index. Attaching index.lifecycle.rollover_alias to an index that is not the current write index behind that alias makes ILM try to roll an index it should not, producing an illegal_argument_exception on the rollover step. When rescuing a plain unmanaged index, set only index.lifecycle.name; leave the rollover alias to the genuine write index.
  • Skip dot-prefixed indices. System, hidden, and monitoring indices (.security, .ds-*, .monitoring-*) are managed by Elasticsearch itself or by their parent data stream. Flagging them as “unmanaged” produces noise and, worse, tempts you to attach a user policy to something you should never touch.
  • A data stream’s backing indices are managed via the stream. The .ds- backing indices behind a data stream inherit their lifecycle from the stream’s template, not from a per-index index.lifecycle.name you set by hand. Do not audit or attach them individually — manage the data stream, and the backing indices follow.

FAQ

Why did my index end up with no lifecycle policy in the first place?

The three usual causes are a reindex that recreated the index without re-attaching the policy in the destination settings, a hand-created index that never went through the template, or a template that was overwritten or never set index.lifecycle.name. In every case ILM has no record of the index and silently ignores it — which is exactly why a periodic unmanaged scan is worth running.

Can I attach a policy to an existing index without reindexing?

Yes. PUT metrics-000007/_settings {"index.lifecycle.name": "metrics-ilm"} brings an existing index under management in place — no reindex required. Reindexing is only necessary when you also need to change something a live setting cannot alter, such as the shard count or a mapping. For pure “it has no policy” drift, put_settings is the direct fix.

Should I attach the rollover alias when rescuing an unmanaged index?

Only if that index is genuinely the write index behind the alias. Setting index.lifecycle.rollover_alias on an old, non-write index makes ILM attempt a rollover it cannot complete, failing with an alias-mismatch error on the rollover step. For a plain unmanaged index that just needs retention, attach only index.lifecycle.name; the rollover alias belongs to the single current write index.

← Back to Detecting ILM Policy Drift · ILM Policy Design & Lifecycle Synchronization