Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions
Here is the official implenmentation of our WWW 2026 paper Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions. This repository provides a comprehensive benchmark and open-source toolkit for Modeling Cascaded Delay Feedback in online Net Conversion Rate (NetCVR) prediction.
In this work, we present systematic insights into the cascading nature of delayed feedback signals and propose effective modeling solutions. This codebase includes datasets, models, training pipelines, and evaluation tools to support future research in delay feedback modeling.
The experiments are based on a large-scale industrial dataset from Alibaba, capturing multi-stage user behaviors including click, add-to-cart, payment, and refund, with precise timestamps for modeling delay dynamics.
👉 Dataset Information:
CASCADE dataset on HuggingFace
📁 Data structure includes:
- User/item/Related Features
- Timestamps for each conversion stage (
click_time,pay_time,refund_time)
data source should be placed under data/CASCADE/.
Below are the baseline models included in this benchmark, along with their original paper references and corresponding implementation scripts.
| Model Name | Model Reference Script |
|---|---|
| ESDFM | ali_reesdfm_stream_pretrain.py |
| MISS | ali_remiss_stream_train.py |
| DFSN | ali_redfsn_stream_train.py |
| Oracle | ali_reoracle_stream_train.py |
| FNW | ali_refnw_stream_train.py |
| FNC | ali_refnc_stream_train.py |
| Defuse | ali_redefuse_stream_train.py |
| Defer | ali_redefer_stream_train.py |
| DDFM | ali_reddfm_stream_train.py |
| TESLA (Ours) | ali_TESLA_stream_train.py |
AirBench4OpenSource/
├── data/ # Raw and metadata files
├── dataloader/ # Custom data loading modules
├── datasets/ # Dataset classes and preprocessing scripts
├── log/ # Training logs and evaluation outputs
├── models/ # Model architectures (e.g., CascadeNet, ESDFM)
├── mx_utils/ # Utility functions: metrics, config, logging, etc.
├── trainers/ # Training and evaluation logic
├── examples/ # Example scripts for quick start
├── requirements.txt # Required Python packages
├── README.md # This file
└── LICENSE # MIT License
git clone [email protected]:CASCADE/AirBench4OpenSource.git
cd AirBench4OpenSource
python -m venv venv
source venv/bin/activate # Linux/Mac
# venv\Scripts\activate # Windows
# Install dependencies
pip install -r requirements.txtDownload the CASCADE dataset on HuggingFace and process it by using scripts below and place it under data/CASCADE/.
# to process data
python process_CASCADE_with_MappingDict.pyTo run the main training script for our model, use:
# to direct run our model
python AirBench4OpenSource/ali_TESLA_stream_train.pySpecifically, you need to run the following pre-training scripts in advance:
# Step 1: Pre-train the base model
python AirBench4OpenSource/ali_esdfmRf_PLE_pretrain.py
# Step 2: Pre-train the inw-tn-pay delay feedback model
python AirBench4OpenSource/ali_esdfmRF_inw_tn_pay_pretrain.py
# Step 3: Pre-train the inw-tn-refund delay feedback model
python AirBench4OpenSource/ali_esdfmRF_inw_tn_refund_pretrain.py
These scripts will generate the necessary checkpoint files (model weights), which are then loaded by ali_TESLA_stream_train.py during training. More usage examples can be found in the scripts under the examples/ directory.