ILM & Reindexing Automation Runbooks

Choosing Reindex or Shrink for Tier Migration

Given a hot index you want to consolidate onto the warm tier, four constraints decide whether _shrink will do the job cheaply or whether you are forced into a full _reindex — walk them in order and the answer falls out.

This is the operational shortcut to the fuller reindex vs shrink comparison, which lays out the mechanics of each operation; here the goal is narrower — hand you a checklist you can run against a real index in under a minute. It fits inside the broader practice of building automated reindexing pipelines and workflows, where choosing the wrong operation means either an unnecessary hours-long document copy or a shrink that Elasticsearch flatly refuses to start. Throughout, the worked index is a set of daily order indices, orders-2025-*, being consolidated into orders-2025-warm.

Prerequisites

  • Elasticsearch 8.x with data_warm nodes so the resulting index can be routed off the hot tier.
  • elasticsearch-py v8.0+ for the client snippets (client.indices.shrink, client.reindex).
  • Knowledge of the source’s current number_of_shards and its mapping, so you can evaluate the divisor and mapping-change questions below.

Implementation: the decision checklist

Run these four questions top to bottom. The first one that answers “yes to reindex” wins — you stop and reindex. If all four clear, shrink is the cheaper, ILM-native choice.

  1. Does the mapping, analyzer, or routing need to change? Shrink shares the source’s Lucene segments, which already encode the mapping — there is no way to alter field types or analysis in flight. Any schema change forces a reindex. → yes: reindex.
  2. Is the target shard count a factor of the source count? Shrink merges whole shards, so the target must divide the source evenly: 6 → 3, 2, or 1 is legal; 6 → 4 is not. A non-divisor target you cannot avoid forces a reindex. → not a divisor: reindex.
  3. Do documents need transforming — fields dropped, values rewritten, an ingest pipeline applied? Shrink copies documents verbatim. Any per-document transform forces a reindex. → yes: reindex.
  4. Is the resource budget tight, and is the index under a policy? If none of the above forced a reindex, shrink is far cheaper (it hard-links segments instead of re-indexing) — and if the index is managed, ILM’s warm-phase shrink action already does it for you. → all clear: shrink, ideally via ILM.
Four-question decision flow: any schema, divisor, or transform need routes to reindex; otherwise shrinkA left-to-right flow with three decision diamonds in sequence — mapping or analyzer change, target shard count a factor of the source, documents need transforming — each with a yes branch dropping to a single reindex box at the bottom. Passing all three no-branches reaches a shrink box on the right, annotated that a managed index uses the ILM warm-phase shrink action automatically.mapping/analyzer?shards afactor?transformdocs?noyesnoSHRINKILM does it if managedyesno (not a factor)yesREINDEXfull copy

The minimal command for each path

If the checklist lands on shrink, prepare the source and issue one call (the full runbook covers co-location and reset in detail):

from elasticsearch import Elasticsearch, ApiError

# Source already read-only and co-located on one node (see the runbook).
es.indices.shrink(
    index="orders-2025",
    target="orders-2025-warm",
    settings={"index.number_of_shards": 1, "index.routing.allocation.require._name": None},
)

If it lands on reindex, create the target with its new shape, then copy asynchronously:

resp = es.reindex(
    source={"index": "orders-2025-*"},
    dest={"index": "orders-2025-warm", "op_type": "create"},
    conflicts="proceed",
    wait_for_completion=False,
)
task_id = resp["task"]  # v8: async task id in the `task` key; poll es.tasks.get(task_id=...)

Verification

After a shrink, confirm the target carries the reduced primary count and nothing else changed — the mapping should be identical to the source:

GET _cat/shards/orders-2025-warm?v&h=index,shard,prirep,state

One primary row (plus its replica) confirms the 1-shard target. After a reindex, verify both the new shard count and that the mapping actually took the intended new shape:

GET orders-2025-warm/_mapping

For the reindex path, also confirm the document count matches by polling the task’s created against the source doc total before deleting the source.

Gotchas and edge cases

  • Shrink needs a divisor, a write block, and co-location — all three. Missing any one makes Elasticsearch reject the operation outright: the target count must divide the source, index.blocks.write must be true, and every primary must sit on one node. If you cannot satisfy the divisor rule, that alone forces a reindex.
  • Reindex transiently doubles storage. The new index is an independent copy, so until you delete the source you need up to 2× the source’s disk on the warm nodes. Shrink hard-links segments and adds almost nothing until the source is dropped. Budget capacity before you start the copy.
  • For pure shard-count reduction, ILM’s shrink action is the default — do not reinvent it. If the index is managed, adding "shrink": { "number_of_shards": 1 } to the warm phase makes the lifecycle engine co-locate, block, shrink, and swap automatically. Manual shrink is for unmanaged one-offs; reaching for a hand-run reindex here is strictly more expensive for no benefit.
  • Swap an alias so queries never chase the index name. Whichever operation you run, point read traffic at a stable alias and move it onto orders-2025-warm in a single _aliases action, so consumers never reference the concrete index name and the source can be retired without breaking queries.

FAQ

The mapping is unchanged but I want 4 shards from a 6-shard index — can I shrink?

No. Four does not divide six, so shrink is unavailable regardless of the mapping being unchanged. Your options are to pick a divisor target instead (6 → 3, 2, or 1) and shrink, or to reindex into a fresh 4-shard index. The divisor rule is a hard constraint on shrink, independent of every other consideration.

If shrink is almost always cheaper, when is reindex ever the right default?

Reindex is the right choice precisely when one of the forcing functions applies: the schema must change, the target shard count is not a divisor, or documents need transforming. In those cases shrink cannot do the job at all, so the cost comparison is moot. When none of those apply, shrink — and ideally ILM’s shrink action — wins on cost every time.

Do I still need to make the source read-only if ILM handles the shrink?

ILM does it for you. The warm-phase shrink action sets the write block, co-locates the primaries, performs the shrink, and swaps the alias without any manual step. You only manage the write block and co-location yourself when running a manual shrink on an unmanaged index, which the dedicated runbook walks through.

← Back to Reindex vs Shrink for Warm-Tier Migration · Automated Reindexing Pipelines & Workflows