Merge pull request #1332 from datanav/IS-17887

hglad · web-flow · commit b109600a96e0 · 2026-04-07T15:50:52.000+02:00
docs: $retract DTL flag for history pruning (IS-17887)
diff --git a/hub/changelog.rst b/hub/changelog.rst
@@ -1,6 +1,13 @@
 Changelog
 =========
 
+.. _changelog_2026-04-07:
+
+2026-04-07
+----------
+* Added :ref:`compaction.retract <compaction_retract_detail>`. When enabled on a pipe, writing an entity with ``$retract: true`` permanently removes all earlier versions of that entity from the sink dataset.
+
+
 .. _changelog_2026-03-30:
 
 2026-03-30
diff --git a/hub/documentation/data-management/entity-data-model.rst b/hub/documentation/data-management/entity-data-model.rst
@@ -173,6 +173,13 @@ Entity fields starting with ``$`` are semi-reserved. They have special meaning a
        the current one.
      -
 
+       .. _dollar_retract:
+   * - ``$retract``
+     - If set to ``true`` and :ref:`compaction.retract <compaction_feature>` is enabled on the pipe,
+       all previous versions of the entity are permanently removed from the dataset while the current
+       version is retained. See :ref:`Compaction: Retract <compaction_retract_detail>` for details.
+     -
+
 .. _entity_data_types:
 
 Standard types
diff --git a/hub/features/compaction-feature.rst b/hub/features/compaction-feature.rst
@@ -101,3 +101,37 @@ Properties
        times out, the compaction process will continue from where it last stopped the next time the pipe runs.
      - 60
      - No
+
+   * - ``compaction.retract``
+     - Boolean
+     - If ``true``, the pipe will honour the :ref:`$retract <dollar_retract>` field on output entities.
+       When an entity with ``$retract: true`` is written to the sink dataset, all earlier versions of that
+       entity are permanently removed while the current version is retained.
+     - ``false``
+     - No
+
+   * - ``compaction.retract_timeout_seconds``
+     - Number
+     - Determines the number of seconds that the pipe is allowed to spend on the retraction process. If the pipe
+       times out, the retraction process will continue from where it last stopped the next time the pipe runs.
+     - 60
+     - No
+
+.. _compaction_retract_detail:
+
+Retract
+^^^^^^^
+
+When ``compaction.retract`` is enabled and an output entity has ``$retract: true``, all earlier
+versions of that entity are permanently removed from the sink dataset while the current version
+is retained. Deletion state is unaffected. The operation is idempotent.
+
+.. WARNING::
+
+   Retract is irreversible. Pruned versions cannot be recovered. Enabling retract will override the
+   ``compaction.keep_versions`` setting on a per-entity basis.
+
+- Only the pipe's sink dataset is affected. Upstream datasets and external consumers are unchanged.
+- ``$retract`` propagates like any other field but may be dropped by :ref:`merge sources <merge_source>`
+  or :ref:`emit_children <emit_children_transform>`. Downstream pipes that must honour the retract
+  need ``compaction.retract`` enabled and ``$retract`` explicitly included in their output.
diff --git a/hub/quick-reference.rst b/hub/quick-reference.rst
@@ -337,4 +337,5 @@ Entity model
    * - Special fields
      - :ref:`$children <dollar_children>` ·
        :ref:`$ids <dollar_ids_field>` ·
-       :ref:`$replaced <dollar_replaced>`
+       :ref:`$replaced <dollar_replaced>` ·
+       :ref:`$retract <dollar_retract>`
