Configuring Snapshot Lifecycle Management (SLM) Policies #

A snapshot that runs only when someone remembers to trigger it is not a backup strategy — it is a hope. For years the answer was a cron entry calling curator or a bespoke shell script that shelled into curl, and every one of those scripts eventually rotted: a credential expired, a host was decommissioned, or the retention arithmetic silently stopped pruning and the repository bill quietly tripled. Snapshot Lifecycle Management moves that whole responsibility inside the deployment, where it is coordinated by the elected master, survives node restarts, and is described by a single declarative document you can version-control. This page is about writing that document correctly: choosing a schedule that fires when you expect, naming each snapshot so it is unique and legible, scoping exactly which indices are captured, and setting a retention window that prunes aggressively without ever deleting your last good copy. It sits under Snapshot Lifecycle Management & Searchable Snapshots, the broader treatment of how snapshots, repositories, and the frozen tier fit together.

The single subtlety that trips up almost every first SLM deployment is that a policy describes two independent scheduled events, not one. The schedule field decides when a snapshot is created; a completely separate cluster-wide setting, slm.retention_schedule, decides when expired snapshots are deleted. A policy can take a snapshot every hour and still only prune once a day, because pruning is not the policy’s job — it is the retention run’s job, and the retention run serves every policy in the deployment at once. Internalize that split early and the rest of SLM configuration becomes straightforward; miss it, and you will spend an afternoon confused about why a repository keeps growing hours after the snapshots supposedly aged out.

Prerequisites #

• An Elasticsearch 8.x cluster with a healthy elected master — SLM is coordinated by the master node, so a deployment with an unstable master will run schedules erratically.
• A registered and verified snapshot repository. The policy references a repository by name and never validates it, so register and verify it first; the full workflow lives in managing snapshot repositories.
• elasticsearch-py v8.0+ installed for the automation below — this page uses the v8 client surface (client.slm.*), not the legacy transport helpers.
• The account applying the policy needs manage_slm at cluster scope, plus create_snapshot; read-only observers need only read_slm. Scope these the way ILM policies are scoped in securing ILM policies with RBAC.
• A rough idea of your snapshot cadence and required retention window (for example: daily snapshots, keep 30 days, never fewer than 7 copies) so schedule, expire_after, min_count, and max_count come from a real requirement rather than a guess.

Architecture: One Policy, Two Schedules #

An SLM policy is a small object with four required parts and one optional-but-essential part. The schedule is a cron expression telling the master when to fire the policy. The name is a date-math template that produces a unique, sortable snapshot name on every fire. The repository names where the snapshot is written. The config block scopes what is captured — which indices, and whether to fold in the global cluster state. The fifth part, retention, is what elevates SLM from a scheduler to a lifecycle manager: it declares how long copies live and how many to keep, and it is enforced not by the policy’s own schedule but by the separate cluster-wide retention run.

Because the create-snapshot event and the retention run are decoupled, they can be tuned independently — and they should be. A policy that snapshots hourly does not need to prune hourly; pruning once a day is almost always fine, and pruning too often just wastes repository list-and-delete API calls against object storage. The consequence to plan for is a lag: between the moment a snapshot ages past expire_after and the next retention run, it still sits in the repository. Size max_count to absorb that lag rather than to your exact snapshot cadence, and repository growth stays predictable.

Configuration Reference #

The policy below is the canonical shape. Every field is annotated with what it controls and the failure it prevents. Apply it with a single PUT to the _slm/policy endpoint under the policy id daily-logs-backup.

PUT _slm/policy/daily-logs-backup
{
  "schedule": "0 30 1 * * ?",
  "name": "<daily-logs-{now/d}>",
  "repository": "prod-s3-repo",
  "config": {
    "indices": ["logs-*"],
    "include_global_state": false,
    "ignore_unavailable": true
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 7,
    "max_count": 60
  }
}

Reading it field by field:

• schedule: "0 30 1 * * ?" — Elasticsearch cron in its six-field form: seconds minutes hours day-of-month month day-of-week. This one reads “at second 0 of minute 30 of hour 1, every day,” i.e. 01:30 daily. The leading seconds field is the part people forget — a five-field Unix crontab pasted here is rejected or misinterpreted, and the ? in the day-of-week slot means “no specific value” because day-of-month is already pinned. The scheduling details are the whole subject of scheduling automated snapshots with SLM.
• name: "<daily-logs-{now/d}>" — a date-math template wrapped in angle brackets. {now/d} rounds to the current day, so a fire on 2026-07-17 produces daily-logs-2026.07.17. Elasticsearch also appends a random suffix under the hood so two snapshots can never collide even within the same rounding bucket. Without a date-math name every snapshot would need a distinct literal name, which is impossible to schedule.
• repository: "prod-s3-repo" — the registered repository that receives the snapshot. SLM does not check reachability at apply time; a typo here produces a policy that applies cleanly and fails on every fire.
• config.indices: ["logs-*"] — the index pattern captured. A wildcard is evaluated at snapshot time, so newly-rolled logs-* indices are included automatically without editing the policy.
• config.include_global_state: false — keeps cluster settings, index templates, and ILM/SLM definitions out of the data snapshot. This is the safe default: it means a restore of daily-logs-backup can never silently overwrite live configuration. Back the global state up in a separate policy that sets this true.
• config.ignore_unavailable: true — a single deleted or closed index in the pattern will not fail the whole snapshot; the missing index is skipped instead of aborting the run.
• retention.expire_after: "30d" — snapshots older than 30 days are candidates for deletion by the retention run.
• retention.min_count: 7 — never keep fewer than 7 successful snapshots, even if all 7 are older than expire_after. This is your floor against a run of failed backups leaving you empty-handed.
• retention.max_count: 60 — never keep more than 60; the oldest beyond that are deleted regardless of age. This is your ceiling against unbounded repository growth.

The precedence between those three retention fields is exact and worth memorizing: min_count wins over expire_after (a snapshot inside the min-count floor is kept even when expired), and max_count caps the total. The full behaviour, including the manual _execute_retention trigger, is covered in setting snapshot retention with SLM.

Step-by-Step Implementation #

Deploying SLM is a three-step sequence: create the policy, prove it works with an on-demand run, then confirm the schedule is armed.

1. Create the policy #

Apply the policy document verbatim over HTTP. PUT is idempotent, so re-applying an edited policy updates it in place without a delete-and-recreate:

PUT _slm/policy/daily-logs-backup
{
  "schedule": "0 30 1 * * ?",
  "name": "<daily-logs-{now/d}>",
  "repository": "prod-s3-repo",
  "config": { "indices": ["logs-*"], "include_global_state": false },
  "retention": { "expire_after": "30d", "min_count": 7, "max_count": 60 }
}

2. Validate with an on-demand execution #

Never wait for the first scheduled fire to find out the repository name was wrong. Trigger the policy immediately; this runs the exact same path a scheduled fire would, and returns the generated snapshot name:

POST _slm/policy/daily-logs-backup/_execute

{ "snapshot_name": "daily-logs-2026.07.17-ab12cd34ef" }

If this returns an error instead of a snapshot name — repository_missing_exception, a permission failure, or a repository verification error — you have caught the misconfiguration in seconds rather than discovering it during a 3 a.m. restore.

3. Deploy and validate from Python (v8+) #

For a repeatable, version-controlled deploy, drive the whole sequence through the elasticsearch-py v8 client. The manager below applies the policy, fires a validating execution, and reads the policy back so you can log the generated snapshot name and confirm the next scheduled time. API and HTTP failures surface as ApiError — a sibling of TransportError, not a subclass — so catch ApiError explicitly; a bare except TransportError will miss a 403 or a 404.

import logging
from elasticsearch import Elasticsearch, ApiError, NotFoundError

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger("slm_config")


class SLMPolicyManager:
    """Apply and validate a single SLM policy idempotently."""

    def __init__(self, es: Elasticsearch):
        self.es = es

    def apply(self, policy_id: str, policy: dict) -> bool:
        try:
            # PUT semantics: creates the policy or overwrites it in place.
            self.es.slm.put_lifecycle(policy_id=policy_id, body=policy)
            logger.info("Policy '%s' applied.", policy_id)
            return True
        except ApiError as exc:
            logger.error("Apply failed (%s): %s", exc.status_code, exc.info)
            return False

    def validate_now(self, policy_id: str) -> str | None:
        """Fire one on-demand snapshot to exercise the whole path immediately."""
        try:
            result = self.es.slm.execute_lifecycle(policy_id=policy_id)
            name = result.get("snapshot_name")
            logger.info("On-demand snapshot started: %s", name)
            return name
        except NotFoundError:
            logger.error("Policy '%s' does not exist.", policy_id)
        except ApiError as exc:
            # A bad repository surfaces here, not at apply time.
            logger.error("Execution failed (%s): %s", exc.status_code, exc.info)
        return None

    def describe(self, policy_id: str) -> None:
        record = self.es.slm.get_lifecycle(policy_id=policy_id)[policy_id]
        logger.info(
            "Policy '%s': next_execution=%s last_success=%s",
            policy_id,
            record.get("next_execution"),
            (record.get("last_success") or {}).get("snapshot_name"),
        )


if __name__ == "__main__":
    es = Elasticsearch(
        "https://es-cluster-01:9200",
        api_key="YOUR_BASE64_ENCODED_API_KEY",
        verify_certs=True,
        request_timeout=30,
    )

    policy = {
        "schedule": "0 30 1 * * ?",
        "name": "<daily-logs-{now/d}>",
        "repository": "prod-s3-repo",
        "config": {"indices": ["logs-*"], "include_global_state": False},
        "retention": {"expire_after": "30d", "min_count": 7, "max_count": 60},
    }

    mgr = SLMPolicyManager(es)
    if mgr.apply("daily-logs-backup", policy):
        mgr.validate_now("daily-logs-backup")
        mgr.describe("daily-logs-backup")

Verification #

A policy is only trustworthy once you have seen it produce a real snapshot and reported a healthy next-execution time. Read the policy back and inspect its embedded status:

GET _slm/policy/daily-logs-backup

{
  "daily-logs-backup": {
    "version": 1,
    "policy": {
      "name": "<daily-logs-{now/d}>",
      "schedule": "0 30 1 * * ?",
      "repository": "prod-s3-repo"
    },
    "last_success": {
      "snapshot_name": "daily-logs-2026.07.17-ab12cd34ef",
      "time": 1752717001000
    },
    "next_execution_millis": 1752802200000,
    "stats": { "snapshots_taken": 1, "snapshots_failed": 0 }
  }
}

last_success.snapshot_name confirms the validating execution landed, and next_execution_millis proves the schedule is armed for the next fire. A last_failure newer than last_success is the signal to page on. Then confirm the snapshot actually exists in the repository, with its final state:

GET _cat/snapshots/prod-s3-repo?v&h=id,status,start_epoch,duration&s=start_epoch:desc

Every row should read SUCCESS. A PARTIAL means some shards were unavailable at snapshot time; a FAILED means the whole run aborted. Neither is silently retried, so treat both as an incident, not a warning.

Threshold Tuning & Performance Guidance #

The cost of a snapshot is not its logical size — it is the volume of Lucene segments that are new since the previous snapshot. Snapshots are incremental at the segment level: the first snapshot of logs-* copies everything, and each subsequent one copies only the segments written since. This makes three tuning decisions matter:

1. Do not force-merge immediately before a snapshot. A force_merge rewrites every segment in the index, which means the very next snapshot has no incremental base to build on and copies the entire index again. If an ILM warm phase force-merges, schedule the SLM snapshot to run before that phase reaches the index, or accept that the post-merge snapshot is a full copy.
2. Throttle when the object store is the bottleneck. snapshot.max_snapshot_bytes_per_sec caps per-node upload throughput to the repository (unlimited by default since 7.16). Reach for it when snapshots saturate the network path to S3 and starve indexing of bandwidth — a lower ceiling makes each snapshot slower but keeps ingest healthy.
3. Size max_count for the retention gap, not the cadence. Because pruning happens on the separate slm.retention_schedule and not per-snapshot, a policy that fires hourly can briefly hold up to a day’s extra snapshots before the daily retention run trims them. If expire_after implies roughly N kept snapshots, set max_count to N plus the number that can accumulate in one retention interval, so the ceiling never clips a snapshot that retention simply has not gotten to yet.

A healthy snapshot cadence also depends on the repository keeping up. When both SLM and an ILM searchable_snapshot action write into the same bucket, they contend for the same upload bandwidth; the relationship between those two producers is laid out in searchable snapshots and the frozen tier.

Troubleshooting #

Policy applied but no snapshots ever appear #

Symptom: GET _slm/policy/<id> shows the policy but stats.snapshots_taken stays at zero past the scheduled time, and _cat/snapshots is empty. Resolution:

1. Check whether the SLM service itself is running: GET _slm/status. If it reads STOPPED, no policy in the deployment fires. Restart it with POST _slm/start.
2. Verify the repository is reachable from every node: POST _snapshot/prod-s3-repo/_verify. An unreachable bucket produces a policy that applies fine and fails on each fire.
3. Read last_failure on the policy for the exact error message from the most recent attempt.

Snapshots complete in PARTIAL state #

Symptom: _cat/snapshots shows PARTIAL rather than SUCCESS. Resolution:

1. A PARTIAL snapshot means one or more shards were unavailable (a red index) when the snapshot ran — the successful shards are captured, but the snapshot is not a complete point-in-time copy.
2. Restore shard health, then re-run with POST _slm/policy/<id>/_execute to capture a clean snapshot.
3. If some indices are expected to be intermittently unavailable, config.ignore_unavailable: true skips missing indices, but it does not rescue a shard that exists but is unassigned.

Retention never runs #

Symptom: snapshots accumulate well past expire_after; the repository keeps growing. Resolution:

1. Confirm the retention schedule is configured cluster-wide: GET _cluster/settings?include_defaults=true&filter_path=**.slm.retention_schedule. If it was cleared, retention never fires.
2. Force a retention pass manually to see errors immediately: POST _slm/_execute_retention.
3. Check GET _slm/stats for retention_runs and retention_failed counters; a non-zero failure count usually points at a repository that denies delete operations.

security_exception on apply from automation #

Symptom: the CI/CD account cannot PUT the policy. Resolution:

1. Grant manage_slm at cluster level (authoring and running policies) plus create_snapshot; observers get read_slm only.
2. Confirm the token’s effective roles with GET _security/_authenticate.
3. Keep authoring privileges on a scoped service account rather than a human login, following the pattern in securing ILM policies with RBAC.

FAQ #

Related #

• Scheduling Automated Snapshots with SLM — the cron and date-math details behind the schedule and name fields.
• Setting Snapshot Retention with SLM — how expire_after, min_count, and max_count interact and when the retention run fires.
• Managing Snapshot Repositories — registering and verifying the repository a policy depends on.
• Searchable Snapshots & the Frozen Tier — the other producer writing into the same repository.
• Securing ILM Policies with RBAC — the governance model that also applies to manage_slm and read_slm.

← Back to SLM & Searchable Snapshots