[BUG] Memory leak: `defer` callbacks accumulate in long-lived coroutines when using `pipeline(callback)` / `transaction(callback)`

# 内存泄漏：在长生命周期协程中使用 `pipeline(callback)` / `transaction(callback)` 会导致 `defer` 回调无限堆积

## 环境

- PHP: 8.2
- Swoole: 6.0.2

更细的环境展开查看⬇️

<details>

<summary>Command: `uname -a && php -v && composer info | grep hyperf && php --ri swoole`</summary>

```bash
Linux 99b9f28baf28 5.15.0-134-generic #145-Ubuntu SMP Wed Feb 12 20:08:39 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
PHP 8.2.29 (cli) (built: Jul  3 2025 13:07:49) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.2.29, Copyright (c) Zend Technologies
    with Zend OPcache v8.2.29, Copyright (c), by Zend Technologies
hyperf/async-queue                      3.1.64           A async queue component for hyperf.
hyperf/cache                            3.1.67           A cache component for hyperf.
hyperf/circuit-breaker                  3.1.63           A circuit breaker component for hyperf.
hyperf/code-parser                      3.1.63           A code parser component for Hyperf.
hyperf/codec                            3.1.63           A codec component for Hyperf.
hyperf/collection                       3.1.64           Hyperf Collection package which come from illuminate/collections
hyperf/command                          3.1.64           Command for hyperf
hyperf/conditionable                    3.1.63           Hyperf Macroable package which come from illuminate/conditionable
hyperf/config                           3.1.63           An independent component that provides configuration container.
hyperf/config-center                    3.1.63           The abstraction component of config center
hyperf/context                          3.1.63           A coroutine/application context library.
hyperf/contract                         3.1.63           The contracts of Hyperf.
hyperf/coordinator                      3.1.63           Hyperf Coordinator
hyperf/coroutine                        3.1.65           Hyperf Coroutine
hyperf/crontab                          3.1.67           A crontab component for Hyperf.
hyperf/database                         3.1.67           A flexible database library.
hyperf/db-connection                    3.1.66           A hyperf db connection handler for hyperf/database.
hyperf/di                               3.1.67           A DI for Hyperf.
hyperf/dispatcher                       3.1.63           A HTTP Server for Hyperf.
hyperf/engine                           2.15.0           Coroutine engine provided by swoole.
hyperf/engine-contract                  1.14.0           Contract for Coroutine Engine
hyperf/event                            3.1.63           an event manager that implements PSR-14.
hyperf/exception-handler                3.1.63           Exception handler for hyperf
hyperf/filesystem                       3.1.63           flysystem integration for hyperf
hyperf/flysystem-oss                    1.4.0           
hyperf/framework                        3.1.63           A coroutine framework that focuses on hyperspeed and flexible, specifically use for build microservices and middlewares.
hyperf/guzzle                           3.1.66           Swoole coroutine handler for guzzle
hyperf/http-message                     3.1.65           microservice framework base on swoole
hyperf/http-server                      3.1.65           A HTTP Server for Hyperf.
hyperf/json-rpc                         3.1.66           A JSON RPC component for Hyperf RPC Server or Client.
hyperf/laminas-mime                     3.0.0            Create and parse MIME messages and parts
hyperf/load-balancer                    3.1.63           A load balancer library for Hyperf.
hyperf/logger                           3.1.63           A logger component for hyperf.
hyperf/macroable                        3.1.63           Hyperf Macroable package which come from illuminate/macroable
hyperf/memory                           3.1.63           An independent component that use to operate and manage memory.
hyperf/model-listener                   3.1.63           A model listener for Hyperf.
hyperf/paginator                        3.1.65           A paginator component for hyperf.
hyperf/pipeline                         3.1.63           Hyperf Macroable package which come from illuminate/pipeline
hyperf/pool                             3.1.66           An independent universal connection pool component.
hyperf/process                          3.1.63           A process component for hyperf.
hyperf/rate-limit                       3.1.63           A rate limiter implemented for Hyperf or other coroutine framework
hyperf/redis                            3.1.66           A redis component for hyperf.
hyperf/retry                            3.1.63           A retry component for hyperf.
hyperf/rpc                              3.1.63           A rpc basic library for Hyperf.
hyperf/rpc-client                       3.1.63           An abstract rpc server component for Hyperf.
hyperf/rpc-server                       3.1.63           An abstract rpc server component for Hyperf.
hyperf/serializer                       3.1.63           A serializer component for Hyperf.
hyperf/server                           3.1.63           A base server library for Hyperf.
hyperf/signal                           3.1.63           A signal library for Hyperf.
hyperf/snowflake                        3.1.63           A snowflake library
hyperf/stdlib                           3.1.63           A stdlib component for Hyperf.
hyperf/stringable                       3.1.65           Hyperf Stringable package which come from illuminate/support
hyperf/support                          3.1.65           A support component for Hyperf.
hyperf/tappable                         3.1.63           Hyperf Macroable package which come from illuminate/tappable
hyperf/token-bucket                     2.0.0            Implementation of the Token Bucket algorithm.
hyperf/translation                      3.1.63           An independent translation component, forked by illuminate/translation.
hyperf/utils                            3.1.42           A tools package that could help developer solved the problem quickly.
hyperf/validation                       3.1.67           hyperf validation
trrtly/casbin                           3.1.1            casbin hyperf component
trrtly/ip2region                        3.0.0            hyperf ip2region

swoole

Swoole => enabled
Author => Swoole Team <team@swoole.com>
Version => 6.0.2
Built => Apr  7 2025 20:00:00
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 3.0.2 15 Mar 2022
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
curl-version => 7.81.0
c-ares => 1.18.1
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
coroutine_pgsql => enabled
coroutine_sqlite => enabled

Directive => Local Value => Master Value
swoole.enable_library => On => On
swoole.enable_fiber_mock => Off => Off
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608
```

</details>

## 组件

`hyperf/redis`（在 v3.1.66 上测试复现，但影响所有包含当前 `MultiExec` trait 实现的版本）

## 问题描述

在一个**长生命周期的协程**（例如 `while(true)` 无限循环）中反复调用 `pipeline(callback)` 或 `transaction(callback)`，每次调用都会注册一个新的 `defer` 闭包，但由于协程永远不会结束，这些闭包永远不会被执行，只会在协程的 defer 栈上无限堆积，导致进程内存缓慢而持续地增长。

相关文件：

- https://github.com/hyperf/hyperf/blob/0d55eb1949d6fcb62b3fc781b46033d916eb11b1/src/redis/src/Redis.php#L74
- https://github.com/hyperf/hyperf/blob/0d55eb1949d6fcb62b3fc781b46033d916eb11b1/src/redis/src/Traits/MultiExec.php#L66

## 根因分析

`pipeline(callback)` 由 `Traits\MultiExec::executeMultiExec()` 处理，内部调用 `Redis::__call()`。这两层各自有 `finally` 块，**两层 finally 的行为产生了冲突**：内层 `__call` 的 finally 注册了 `defer` 并将连接存入协程上下文，外层 `executeMultiExec` 的 finally 随即清空了上下文并释放了连接。下次循环时，上下文已被清空，于是 `__call` 再次走进 `defer` 注册分支，如此往复。

### 单次 `pipeline(callback)` 调用的完整执行流

```
executeMultiExec('pipeline', $callback)
│
├── $hasExistingConnection = Context::has(key)          // false（上次已被清空）
│
├── $this->__call('pipeline', [])
│   ├── $hasContextConnection = Context::has(key)       // false
│   ├── $connection = pool->get()                       // 从连接池借出连接
│   ├── 执行 pipeline 命令
│   └── finally（__call 内层）:
│       ├── !$hasContextConnection → true → 进入分支
│       ├── shouldUseSameConnection('pipeline') → true
│       ├── Context::set(key, $connection)              // ← 将连接存入协程上下文
│       └── defer(fn => releaseContextConnection())     // ← 注册 defer 回调
│
├── tap($instance, $callback)->exec()                   // 执行用户回调和 exec
│
└── finally（executeMultiExec 外层）:
    ├── !$hasExistingConnection → true → 进入分支
    └── releaseContextConnection()
        ├── Context::get(key) → 取出连接
        ├── Context::set(key, null)                     // ← 将上下文置为 null
        └── $connection->release()                      // ← 连接归还池
```

因此，当外层 `executeMultiExec` 的 finally 通过 `releaseContextConnection()` 将上下文值设为 `null` 后，下一次调用时 `Context::has()` 返回 `false`，`__call` 又会走进注册 `defer` 的分支。

**每次 `pipeline(callback)` 调用都会注册一个新的 `defer` 闭包，但外层 finally 已经释放了连接并清空了上下文。** 这些 `defer` 回调在真正执行时只是空操作（因为 `Context::get()` 返回 `null`），但在永远不结束的协程中，它们根本不会被执行——只会在协程的 defer 栈上无限堆积。

### 内存堆积速率

| 运行时长 | defer 闭包数量 | 预估内存占用 |
|----------|:--------------:|-----------------:|
| 1 小时（5秒间隔） | 720 | ~200 KB |
| 1 天 | 17,280 | ~5 MB |
| 1 周 | 120,960 | ~35 MB |
| 1 个月 | 518,400 | ~150 MB |

## 复现代码

```php
<?php

use Hyperf\Coroutine\Coroutine;

// 模拟一个长生命周期协程（类似 ConfigFetcherProcess）
$redis = make(\Hyperf\Redis\RedisProxy::class, ['pool' => 'default']);
Coroutine::create(function () use($redis) {
    $memStart = memory_get_usage();

    for ($i = 0; $i < 10000; $i++) {
        // 每次调用都会注册一个永远不会触发的 defer 闭包
        $redis->pipeline(function ($pipe) {
            $pipe->get('some_key');
        });
    }

    $memEnd = memory_get_usage();

    // 由于 defer 闭包堆积，内存随循环次数线性增长
    echo sprintf("内存增长: %d KB\n", ($memEnd - $memStart) / 1024);
});
```

作为对比，在同样的循环中使用 `get()`（非 pipeline）**不会泄漏**，因为普通命令在 `__call` 中走的是 `else` 分支，直接调用 `$connection->release()` 立即归还连接，不注册 `defer`。

## 修复建议

查看 https://github.com/hyperf/hyperf/pull/7734

### 修复效果

| | 修复前 | 修复后 |
|---|---|---|
| defer 注册次数 | **每次调用 1 个**（无限堆积） | **使用 callback 方式调用，不会注册 defer** |
| 连接行为 | 每次借出 → 归还 → 再借出 | 使用 callback 方式调用，如果协程内没有连接复用，执行后会立即释放连接 |
| 内存泄漏 | 有 | **无** |


## 影响范围

任何在长生命周期协程中调用 `pipeline(callback)`、`transaction(callback)` 或 `multi(callback)` 的代码都会触发此内存泄漏。典型受影响场景：

- `hyperf/config-center` 的自定义 ` RedisConfigFetcherProcess extends ConfigFetcherProcess`（伪代码，比如自定义在无限循环中使用 `pipeline` 定时拉取配置）
- 在 `BeforeWorkerStart::class`，`BeforeProcessHandle::class`，`MainCoroutineServerStart::class` 事件中，启动长生命周期协程收集内存和进程信息
- 任何包含无限循环且使用 Redis pipeline/transaction 的自定义进程
- 任何守护式长生命周期协程中使用这些方法的场景





Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[BUG] Memory leak: `defer` callbacks accumulate in long-lived coroutines when using `pipeline(callback)` / `transaction(callback)` #7733

内存泄漏：在长生命周期协程中使用 `pipeline(callback)` / `transaction(callback)` 会导致 `defer` 回调无限堆积

环境

组件

问题描述

根因分析

单次 `pipeline(callback)` 调用的完整执行流

内存堆积速率

复现代码

修复建议

修复效果

影响范围

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

运行时长	defer 闭包数量	预估内存占用
1 小时（5秒间隔）	720	~200 KB
1 天	17,280	~5 MB
1 周	120,960	~35 MB
1 个月	518,400	~150 MB

	修复前	修复后
defer 注册次数	每次调用 1 个（无限堆积）	使用 callback 方式调用，不会注册 defer
连接行为	每次借出 → 归还 → 再借出	使用 callback 方式调用，如果协程内没有连接复用，执行后会立即释放连接
内存泄漏	有	无

Uh oh!

[BUG] Memory leak: defer callbacks accumulate in long-lived coroutines when using pipeline(callback) / transaction(callback) #7733

Description

内存泄漏：在长生命周期协程中使用 pipeline(callback) / transaction(callback) 会导致 defer 回调无限堆积

环境

组件

问题描述

根因分析

单次 pipeline(callback) 调用的完整执行流

内存堆积速率

复现代码

修复建议

修复效果

影响范围

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[BUG] Memory leak: `defer` callbacks accumulate in long-lived coroutines when using `pipeline(callback)` / `transaction(callback)` #7733

内存泄漏：在长生命周期协程中使用 `pipeline(callback)` / `transaction(callback)` 会导致 `defer` 回调无限堆积

单次 `pipeline(callback)` 调用的完整执行流