Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Dec 16, 2025

• 支持csv pipelines, item 支持指定使用的pipelines

  Boris-code

  committed
  52dc67d

  Copy full SHA for 52dc67d

Commits on Nov 24, 2025

• feat: Item 支持指定 Pipeline 路由

  Show description for 1b9abd3
  

  ShellMonster

  committed
  1b9abd3

  Copy full SHA for 1b9abd3

Commits on Nov 7, 2023

• BloomFilter去重默认使用同一个key，其它类型去重默认使用redis_key

  Boris-code

  committed
  7946309

  Copy full SHA for 7946309

Commits on Nov 29, 2022

• fix airspider task queue max size bug

  Boris-code

  committed
  7ff9c79

  Copy full SHA for 7ff9c79

Commits on Nov 18, 2022

• 支持重新导入失败的item

  Boris-code

  committed
  13b751f

  Copy full SHA for 13b751f

Commits on Oct 25, 2022

• air spider 支持去重

  Boris-code

  committed
  051c2ae

  Copy full SHA for 051c2ae

Commits on Oct 20, 2022

• 优化监控

  Boris-code

  committed
  c8497a3

  Copy full SHA for c8497a3

Commits on Sep 29, 2022

• 下载器使用自定义配置生效

  Boris-code

  committed
  49d2eb7

  Copy full SHA for 49d2eb7

Commits on Aug 18, 2022

• 入库时修改打点的位置

  Boris-code

  committed
  11d7353

  Copy full SHA for 11d7353

Commits on May 18, 2022

• 支持飞书报警

  Boris-code

  committed
  da85aeb

  Copy full SHA for da85aeb

Commits on May 17, 2022

• 优化collector

  Boris-code

  committed
  f2afac2

  Copy full SHA for f2afac2

Commits on Sep 22, 2021

• pipeline 支持close方法

  Boris-code

  committed
  493ff7b

  Copy full SHA for 493ff7b

Commits on Sep 14, 2021

• 修改监控统计图

  Boris-code

  committed
  4b9306c

  Copy full SHA for 4b9306c

Commits on Aug 29, 2021

• 修复 item 获取 table_name的bug, request 支持获取代理及ua

  Boris-code

  committed
  a5de93d

  Copy full SHA for a5de93d

• 去重配置暴露到setting

  Boris-code

  committed
  469c1a1

  Copy full SHA for 469c1a1

Commits on Aug 23, 2021

• 调整入库超过最大次数 报警位置

  Boris-code

  committed
  6ce2bf6

  Copy full SHA for 6ce2bf6

• air_spider 入库失败不提示重试，支持报警

  Boris-code

  committed
  09e2888

  Copy full SHA for 09e2888

Commits on Aug 18, 2021

• fix bug

  Boris-code

  committed
  ec38a3c

  Copy full SHA for ec38a3c

Commits on Aug 17, 2021

• 多批item入库时，一批入库失败不影响其他批

  Boris-code

  committed
  6e7f487

  Copy full SHA for 6e7f487

• 数据入库失败 记录到redis

  Boris-code

  committed
  0429cd5

  Copy full SHA for 0429cd5

• 数据入库失败 自动重试

  Boris-code

  committed
  1b00eb3

  Copy full SHA for 1b00eb3

Commits on Aug 4, 2021

• 解决jion报错问题

  Boris-code

  committed
  32721c6

  Copy full SHA for 32721c6

• 集成打点监控

  Boris-code

  committed
  f5f17f9

  Copy full SHA for f5f17f9

Commits on May 28, 2021

• 修改邮箱

  Boris-code

  committed
  1ef34e2

  Copy full SHA for 1ef34e2

Commits on May 23, 2021

• 爬虫支持重复启动

  Boris-code

  committed
  3108880

  Copy full SHA for 3108880

Commits on May 13, 2021

• 线程可重复启动

  Boris-code

  committed
  b423722

  Copy full SHA for b423722

Commits on Mar 30, 2021

• AirSpider 支持itemz自动入库

  

  Boris

  committed
  a410275

  Copy full SHA for a410275

Commits on Mar 23, 2021

• 修复去重问题

  

  Boris

  committed
  dbb1c63

  Copy full SHA for dbb1c63

• pre_to_db在parser线程中调用

  

  Boris

  committed
  c7efaf9

  Copy full SHA for c7efaf9

• 修正 Item 中的per_to_db函数为pre_to_db

  

  Boris

  committed
  1a9b629

  Copy full SHA for 1a9b629

Commits on Mar 18, 2021

• 完善文档, update_items改为非必须实现接口

  

  Boris

  committed
  e017bdd

  Copy full SHA for e017bdd

Commits on Mar 17, 2021

• 删除无用的配置

  

  Boris

  committed
  601f3fc

  Copy full SHA for 601f3fc

• 支持pipline

  

  Boris

  committed
  8a01432

  Copy full SHA for 8a01432

• 简化代码

  

  Boris

  committed
  52191e9

  Copy full SHA for 52191e9

Commits on Mar 10, 2021

• 修复表名提取错误的bug

  

  Boris

  committed
  89dffb7

  Copy full SHA for 89dffb7