13 Commits

Author SHA1 Message Date
052f3a137b ```
fix(public): 修复项目列表中section和type字段显示逻辑

修复了项目列表表格中section和type字段的显示顺序问题,
确保优先显示type字段内容,当type为空时才显示section字段内容
```
2026-03-30 18:35:40 +08:00
0d74cfe754 ```
feat(scheduler): 添加高级调度配置界面

- 实现调度模式选择(每天、每周、每月、自定义Cron)
- 添加时间选择器和日期相关配置选项
- 提供可视化调度预览功能
- 支持Cron表达式验证和实时更新
- 修改默认调度时间为早上8点40分
- 更新邮件接收者地址
```
2026-03-20 16:28:27 +08:00
89d0abd44c ```
feat: 添加Windows PowerShell重启脚本

添加restart.ps1脚本来管理服务器进程的启动、停止和重启功能,
包括PID文件管理和端口检测。同时创建了logs/server.pid文件来
存储服务器进程ID。
```
2026-03-20 11:54:13 +08:00
f8374f5e0d ```
feat: 添加服务器重启脚本

添加restart.sh脚本用于管理服务器进程,包含以下功能:
- 自动查找并停止现有服务器进程
- 启动新的服务器实例
- 管理进程ID文件和日志输出
- 支持端口配置和状态检查
```
2026-03-19 15:37:29 +08:00
f8dfad26a4 ```
feat(resultStore): 支持数据库文件迁移和兼容性处理

- 添加对旧版 results.sqlite 到新版 results.db 的自动迁移支持
- 实现数据库连接时的错误处理和日志记录
- 修改数据库事务模式为 DELETE 模式以提高稳定性
- 增加对临时 WAL 和 SHM 文件的清理处理
- 提供环境变量配置的数据库路径优先级支持
```
2026-03-19 15:12:55 +08:00
2a5fd99319 ```
feat(project): 添加项目数据导出功能

- 在项目查询界面添加导出按钮,支持将项目结果导出为CSV文件
- 实现exportProjectResults函数处理导出逻辑,包括参数验证和文件下载
- 新增validateProjectFilters函数用于验证项目筛选条件
- 新增buildProjectQueryParams函数构建查询参数
- 在服务端添加/api/projects/export接口,返回CSV格式数据
- 实现CSV文件生成和下载功能,包含中文文件名处理
- 重构项目查询逻辑,提取getProjects函数复用代码
```
2026-03-19 15:03:43 +08:00
7bfba04199 ```
feat: 添加项目管理功能

添加了项目管理标签页,支持按城市、板块、项目名称、金额范围、日期等条件进行过滤查询,
包含完整的前端界面和后端API接口,实现数据去重和分页功能
```
2026-03-19 11:40:14 +08:00
d78dc655ee ```
chore(config): 更新.gitignore文件以忽略数据库相关文件

添加了data/目录、SQLite数据库文件及相关临时文件到.gitignore中,
避免敏感数据和临时文件被提交到版本控制系统。
```
2026-03-19 10:18:25 +08:00
bd46d8f907 增加板块, 增加了对 urlencoded 数据的容量限制 2026-03-16 11:27:31 +08:00
0648770a6a ```
feat(config): 更新配置中字段名从target_link改为detail_link

将JSON配置中的target_link字段统一改为detail_link,以更好地反映链接的实际用途。

BREAKING CHANGE: 配置文件中的字段名发生变化,需要更新相关引用
```
2026-03-11 14:42:38 +08:00
40118ec508 ```
feat(config): 添加任务配置中的模型模式支持

- 在config.json中为任务添加mode和useBrowser字段
- 默认使用glm-5模型模式

feat(ui): 更新前端界面显示模型信息并添加模型选择功能

- 在任务表格中添加模型列显示
- 在新增/编辑任务表单中添加模型选择下拉框
- 支持多种模型选项包括qwen3.5-plus、qwen3-max等
- 更新表格列数以适应新增的模型列

feat(core): 实现任务模型模式的功能支持

- 在agentService.js中添加normalizeMode函数处理模型模式
- 修改createTask和runAgentTask函数支持mode参数
- 在scheduler.js中实现任务的模型模式配置
- 在server.js中添加模型模式的标准化和API支持
- 为任务运行时添加模型模式的日志输出
```
2026-03-10 18:11:11 +08:00
b9270428db ```
feat(config):
style(ui): 全面重构用户界面样式

- 引入新的配色方案和设计系统变量
- 更新字体家族,使用 Fira Sans 和 Noto Sans SC
- 重新设计页面布局和组件样式
- 添加响应式设计优化
- 改进按钮、表格、表单等UI元素的视觉效果

feat(tasks): 添加任务级别浏览器配置选项

- 在任务配置中增加独立的浏览器开启/关闭选项
- 支持任务继承全局浏览器设置
- 在任务列表中显示浏览器配置状态
- 实现任务级别的 useBrowser 字段管理
```
2026-03-10 17:58:09 +08:00
4f504447a1 根据提供的code differences信息,我发现没有具体的代码差异内容。由于没有实际的代码变更信息,我将生成一个通用的示例commit message:
```
docs(changelog): 更新版本发布说明

- 添加了最新的功能变更记录
- 修复了已知问题的描述
- 更新了API文档的相关部分
```
2026-03-10 16:16:57 +08:00
18 changed files with 3976 additions and 2541 deletions

5
.gitignore vendored
View File

@@ -14,6 +14,11 @@ pnpm-debug.log*
# 配置文件(包含敏感信息) # 配置文件(包含敏感信息)
config.json config.json
data/
*.sqlite
*.sqlite-shm
*.sqlite-wal
*.migrated.bak
# 编辑器目录和文件 # 编辑器目录和文件
.vscode/ .vscode/

View File

@@ -1,9 +1,15 @@
{ {
"agent": {
"baseUrl": "http://192.168.3.65:18777",
"pollInterval": 3000,
"timeout": 3600000
},
"scheduler": { "scheduler": {
"enabled": false, "enabled": true,
"cronTime": "0 9 * * *", "cronTime": "40 08 * * *",
"threshold": 0, "threshold": 100000,
"description": "每天9点采集当日项目" "description": "每天9点采集当日项目",
"timeRange": "thisMonth"
}, },
"email": { "email": {
"smtpHost": "smtp.qq.com", "smtpHost": "smtp.qq.com",
@@ -11,887 +17,5 @@
"smtpUser": "1076597680@qq.com", "smtpUser": "1076597680@qq.com",
"smtpPass": "nfrjdiraqddsjeeh", "smtpPass": "nfrjdiraqddsjeeh",
"recipients": "5482498@qq.com" "recipients": "5482498@qq.com"
}, }
"scrapers": [
{
"id": "scraper-1772762494299",
"city": "南京市",
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/buildService1.html",
"section": "房建市政",
"subsection": "工程类、服务类",
"type": "招标公告",
"prompt": "提取页面上今天的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等)、发布日期(YYYY-MM-DD格式)、详情页完整URL如果没有该日期数据直接忽略并输出结果",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958569164",
"city": "南京市",
"url": "https://njggzy.nanjing.gov.cn/njweb/jtsw/traffic.html",
"section": "交通水务",
"subsection": "交通",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958590218",
"city": "南京市",
"url": "https://njggzy.nanjing.gov.cn/njweb/jtsw/069005/traffic5.html",
"section": "交通水务",
"subsection": "水务",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958710168",
"city": "无锡市",
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/jsgc/index.shtml",
"section": "建设工程",
"subsection": "工程类、非工程类",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958756969",
"city": "无锡市",
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上2026-03-05招标公告信息包括标题、项目金额可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958789571",
"city": "无锡市",
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/jtgc/index.shtml",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958889688",
"city": "徐州市",
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003001/003001014/list.html",
"section": "建设工程",
"subsection": "",
"type": "招标文件提前公示",
"prompt": "从页面中提取今日的招标文件提前公示信息,具体字段包括:标题、项目金额(包含合同预估价、最高投标限价等所有涉及金额的信息)、发布日期(严格按照 YYYY-MM-DD 格式)、详情页完整 URL。若当前为分页第一页且未检索到任何符合 “今日发布” 条件的公示信息,直接返回 “无数据”,无需执行后续翻页及提取操作;若第一页存在有效数据,则正常提取对应字段信息",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958933270",
"city": "徐州市",
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003001/003001001/list.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "从页面中提取今日的招标公告/资审公告提前公示信息,具体字段包括:标题、项目金额(包含合同预估价、最高投标限价等所有涉及金额的信息)、发布日期(严格按照 YYYY-MM-DD 格式)、详情页完整 URL。若当前为分页第一页且未检索到任何符合 “今日发布” 条件的公示信息,直接返回 “无数据”,无需执行后续翻页及提取操作;若第一页存在有效数据,则正常提取对应字段信息",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772958989431",
"city": "徐州市",
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003002/003002005/list.html",
"section": "交通工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959084636",
"city": "徐州市",
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003002/003002001/list.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959127769",
"city": "徐州市",
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003003/003003005/list.html",
"section": "水务工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959152540",
"city": "徐州市",
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003003/003003001/list.html",
"section": "水务工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959320386",
"city": "常州市",
"url": "http://ggzy.xzsp.changzhou.gov.cn/jyzx/001001/tradeInfonew.html?category=001001",
"section": "建设工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959351660",
"city": "常州市",
"url": "http://ggzy.xzsp.changzhou.gov.cn/jyzx/001001/tradeInfonew.html?category=001001",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959481049",
"city": "常州市",
"url": "http://ggzy.xzsp.changzhou.gov.cn/jyzx/001001/tradeInfonew.html?category=001001",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959722015",
"city": "苏州市",
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标预计划",
"prompt": "提取页面上今日的招标预计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959755349",
"city": "苏州市",
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959817179",
"city": "苏州市",
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标预计划",
"prompt": "提取页面上今日的招标预计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959842489",
"city": "苏州市",
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959877226",
"city": "苏州市",
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
"section": "水利工程",
"subsection": "",
"type": "招标预计划",
"prompt": "提取页面上今日的招标预计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959894660",
"city": "苏州市",
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
"section": "水务工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959960153",
"city": "南通市",
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003001/003001009/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772959988900",
"city": "南通市",
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003001/003001001/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960042784",
"city": "南通市",
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003005/003005001/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告/招标计划",
"prompt": "提取页面上今日的招标公告/招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960066902",
"city": "南通市",
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003006/003006001/tradeInfo.html",
"section": "水利工程",
"subsection": "",
"type": "招标公告/招标计划",
"prompt": "提取页面上今日的招标公告/招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960133857",
"city": "连云港市",
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001001/001001008/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960157269",
"city": "连云港市",
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001001/001001001/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960194684",
"city": "连云港市",
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001003/001003004/tradeInfo.html",
"section": "水利工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960218045",
"city": "连云港市",
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001003/001003001/tradeInfo.html",
"section": "水务工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960262581",
"city": "连云港市",
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001002/001002004/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960280984",
"city": "连云港市",
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001002/001002001/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960353235",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001001/001001009/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960376613",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001001/001001001/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960419655",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001002/001002009/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960532565",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001002/001002001/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960574486",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001003/001003006/tradeInfo.html",
"section": "水利工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960597160",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001003/001003001/tradeInfo.html",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960644255",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001009/001009005/tradeInfo.html",
"section": "土地整治",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960674724",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001009/001009001/tradeInfo.html",
"section": "土地整治",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960714492",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001010/001010005/tradeInfo.html",
"section": "农田建设",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960745082",
"city": "淮安市",
"url": "https://ggzy.huaian.gov.cn/jyxx/001010/001010001/tradeInfo.html",
"section": "农田建设",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960817101",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "工程建设",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772960871290",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "工程建设",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772961673500",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "交通工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772961722432",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772961991141",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "水利工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962017043",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962073734",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "农业农村",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962143448",
"city": "盐城市",
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
"section": "农业农村",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962220152",
"city": "扬州市",
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/fjsz/zbgg/index.html",
"section": "房建市政",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962257442",
"city": "扬州市",
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/jtgc/zbgg/index.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962297502",
"city": "扬州市",
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/slgc/zbgg/index.html",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962349287",
"city": "扬州市",
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/nygc/nyzbgg/index.html",
"section": "农业工程",
"subsection": "",
"type": "农业招标公告",
"prompt": "提取页面上今日的农业招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962408213",
"city": "镇江市",
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
"section": "建设工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962433571",
"city": "镇江市",
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
"section": "建设工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962491815",
"city": "镇江市",
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
"section": "交通工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962512616",
"city": "镇江市",
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962542402",
"city": "镇江市",
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
"section": "水利工程",
"subsection": "",
"type": "招标计划",
"prompt": "提取页面上今日的招标计划信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962560473",
"city": "镇江市",
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962599558",
"city": "泰州市",
"url": "http://ggzy.taizhou.gov.cn/jyxx/001001/001001001/secondPagejyxx.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962630223",
"city": "泰州市",
"url": "http://ggzy.taizhou.gov.cn/jyxx/001013/001013001/secondPagejyxx.html",
"section": "能源工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962664564",
"city": "泰州市",
"url": "http://ggzy.taizhou.gov.cn/jyxx/001002/001002001/secondPagejyxx.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962689903",
"city": "泰州市",
"url": "http://ggzy.taizhou.gov.cn/jyxx/001003/001003001/secondPagejyxx.html",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962734224",
"city": "泰州市",
"url": "http://ggzy.taizhou.gov.cn/jyxx/001012/001012001/secondPagejyxx.html",
"section": "农业工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962859477",
"city": "宿迁市",
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001010/tradeInfo.html",
"section": "招标计划提前发布",
"subsection": "",
"type": "招标计划提前发布",
"prompt": "提取页面上今日的招标计划提前发布信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962901103",
"city": "宿迁市",
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001001/001001010/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标预公告",
"prompt": "提取页面上今日的招标预公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962919812",
"city": "宿迁市",
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001001/001001001/tradeInfo.html",
"section": "建设工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772962967658",
"city": "宿迁市",
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001002/001002001/tradeInfo.html",
"section": "交通工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963005494",
"city": "宿迁市",
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001003/001003001/tradeInfo.html",
"section": "水利工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963398342",
"city": "江苏省公共资源交易平台",
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
"section": "建设工程",
"subsection": "",
"type": "招标计划/招标计划变更公告",
"prompt": "提取页面上今日的招标计划/招标计划变更公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963423954",
"city": "江苏省公共资源交易平台",
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
"section": "建设工程",
"subsection": "",
"type": "招标公告/资审公告",
"prompt": "提取页面上今日的招标公告/资审公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963466949",
"city": "江苏省公共资源交易平台",
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
"section": "交通工程",
"subsection": "",
"type": "招标计划/招标计划变更公告",
"prompt": "提取页面上今日的招标计划/招标计划变更公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963511551",
"city": "江苏省公共资源交易平台",
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
"section": "交通工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963581266",
"city": "江苏省公共资源交易平台",
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
"section": "水利工程",
"subsection": "",
"type": "招标计划/招标计划变更公告",
"prompt": "提取页面上今日的招标计划/招标计划变更公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963597905",
"city": "江苏省公共资源交易平台",
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
"section": "水利工程",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963721156",
"city": "江苏省建设工程招标网",
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%B7%BF%CE%DD%BD%A8%D6%FE%CA%A9%B9%A4",
"section": "房屋建筑施工",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772963750768",
"city": "江苏省建设工程招标网",
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%CA%D0%D5%FE%B9%A4%B3%CC%CA%A9%B9%A4",
"section": "市政工程施工",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772965504453",
"city": "江苏省建设工程招标网",
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%B5%A5%B6%C0%D7%B0%CA%CE%D7%B0%D0%DE%CA%A9%B9%A4",
"section": "单独装饰装修施工",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772965525132",
"city": "江苏省建设工程招标网",
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%C9%E8%BC%C6",
"section": "设计",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上今日的招标公告信息,包括:标题、项目金额(可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
},
{
"id": "scraper-1772965538927",
"city": "江苏省建设工程招标网",
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%CB%AE%C0%FB",
"section": "水利",
"subsection": "",
"type": "招标公告",
"prompt": "提取页面上2026年3月3日的招标公告信息包括标题、项目金额可能为合同预估价/最高投标限价等等、发布日期YYYY-MM-DD格式、详情页完整URL",
"enabled": true,
"model": "spark-1-mini"
}
]
} }

13
disable-all-tasks.bat Normal file
View File

@@ -0,0 +1,13 @@
@echo off
chcp 65001 >nul
cd /d "%~dp0"
echo ========================================
echo 批量禁用所有任务
echo ========================================
echo.
node disable-all-tasks.js
echo.
pause

65
disable-all-tasks.js Normal file
View File

@@ -0,0 +1,65 @@
// 批量禁用所有任务脚本
// 用法: node disable-all-tasks.js
import Database from 'better-sqlite3';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
import { existsSync, mkdirSync } from 'fs';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const DEFAULT_DB_DIR = join(__dirname, 'data');
const DEFAULT_DB_PATH = join(DEFAULT_DB_DIR, 'results.db');
const LEGACY_DB_PATH = join(DEFAULT_DB_DIR, 'results.sqlite');
// 确定数据库路径
let DB_PATH = process.env.APP_DB_PATH || process.env.RESULTS_DB_PATH;
if (!DB_PATH) {
if (existsSync(DEFAULT_DB_PATH)) {
DB_PATH = DEFAULT_DB_PATH;
} else if (existsSync(LEGACY_DB_PATH)) {
DB_PATH = LEGACY_DB_PATH;
} else {
DB_PATH = DEFAULT_DB_PATH;
}
}
if (!existsSync(DB_PATH)) {
console.error('数据库文件不存在:', DB_PATH);
console.log('请确保项目已初始化并运行过至少一次');
process.exit(1);
}
try {
const db = new Database(DB_PATH);
// 先查询当前启用的任务数量
const enabledCount = db.prepare('SELECT COUNT(*) as count FROM tasks WHERE enabled = 1').get();
console.log(`当前启用的任务数量: ${enabledCount.count}`);
// 查询总任务数量
const totalCount = db.prepare('SELECT COUNT(*) as count FROM tasks').get();
console.log(`总任务数量: ${totalCount.count}`);
if (enabledCount.count === 0) {
console.log('没有需要禁用的任务');
db.close();
process.exit(0);
}
// 执行批量禁用
const result = db.prepare('UPDATE tasks SET enabled = 0').run();
console.log(`已禁用 ${result.changes} 个任务`);
// 验证结果
const newEnabledCount = db.prepare('SELECT COUNT(*) as count FROM tasks WHERE enabled = 1').get();
console.log(`禁用后启用的任务数量: ${newEnabledCount.count}`);
db.close();
console.log('操作完成');
} catch (error) {
console.error('操作失败:', error.message);
process.exit(1);
}

1
logs/server.pid Normal file
View File

@@ -0,0 +1 @@
15556

721
package-lock.json generated
View File

@@ -8,14 +8,14 @@
"name": "njggzy-scraper", "name": "njggzy-scraper",
"version": "2.0.0", "version": "2.0.0",
"dependencies": { "dependencies": {
"@mendable/firecrawl-js": "^4.15.2", "@mendable/firecrawl-js": "latest",
"cheerio": "^1.0.0-rc.12", "better-sqlite3": "^12.8.0",
"cors": "^2.8.5", "cors": "^2.8.5",
"dotenv": "^17.2.3", "dotenv": "^17.2.3",
"express": "^5.2.1", "express": "^5.2.1",
"node-cron": "^4.2.1", "node-cron": "^4.2.1",
"nodemailer": "^7.0.11", "nodemailer": "^7.0.11",
"zod": "^4.3.6" "zod": "^3.24.2"
} }
}, },
"node_modules/@mendable/firecrawl-js": { "node_modules/@mendable/firecrawl-js": {
@@ -33,15 +33,6 @@
"node": ">=22.0.0" "node": ">=22.0.0"
} }
}, },
"node_modules/@mendable/firecrawl-js/node_modules/zod": {
"version": "3.25.76",
"resolved": "https://registry.npmmirror.com/zod/-/zod-3.25.76.tgz",
"integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/accepts": { "node_modules/accepts": {
"version": "2.0.0", "version": "2.0.0",
"resolved": "https://registry.npmmirror.com/accepts/-/accepts-2.0.0.tgz", "resolved": "https://registry.npmmirror.com/accepts/-/accepts-2.0.0.tgz",
@@ -97,6 +88,60 @@
"proxy-from-env": "^1.1.0" "proxy-from-env": "^1.1.0"
} }
}, },
"node_modules/base64-js": {
"version": "1.5.1",
"resolved": "https://registry.npmmirror.com/base64-js/-/base64-js-1.5.1.tgz",
"integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/better-sqlite3": {
"version": "12.8.0",
"resolved": "https://registry.npmmirror.com/better-sqlite3/-/better-sqlite3-12.8.0.tgz",
"integrity": "sha512-RxD2Vd96sQDjQr20kdP+F+dK/1OUNiVOl200vKBZY8u0vTwysfolF6Hq+3ZK2+h8My9YvZhHsF+RSGZW2VYrPQ==",
"hasInstallScript": true,
"license": "MIT",
"dependencies": {
"bindings": "^1.5.0",
"prebuild-install": "^7.1.1"
},
"engines": {
"node": "20.x || 22.x || 23.x || 24.x || 25.x"
}
},
"node_modules/bindings": {
"version": "1.5.0",
"resolved": "https://registry.npmmirror.com/bindings/-/bindings-1.5.0.tgz",
"integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
"license": "MIT",
"dependencies": {
"file-uri-to-path": "1.0.0"
}
},
"node_modules/bl": {
"version": "4.1.0",
"resolved": "https://registry.npmmirror.com/bl/-/bl-4.1.0.tgz",
"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
"license": "MIT",
"dependencies": {
"buffer": "^5.5.0",
"inherits": "^2.0.4",
"readable-stream": "^3.4.0"
}
},
"node_modules/body-parser": { "node_modules/body-parser": {
"version": "2.2.1", "version": "2.2.1",
"resolved": "https://registry.npmmirror.com/body-parser/-/body-parser-2.2.1.tgz", "resolved": "https://registry.npmmirror.com/body-parser/-/body-parser-2.2.1.tgz",
@@ -137,11 +182,29 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/boolbase": { "node_modules/buffer": {
"version": "1.0.0", "version": "5.7.1",
"resolved": "https://registry.npmmirror.com/boolbase/-/boolbase-1.0.0.tgz", "resolved": "https://registry.npmmirror.com/buffer/-/buffer-5.7.1.tgz",
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==", "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
"license": "ISC" "funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"base64-js": "^1.3.1",
"ieee754": "^1.1.13"
}
}, },
"node_modules/bytes": { "node_modules/bytes": {
"version": "3.1.2", "version": "3.1.2",
@@ -181,47 +244,11 @@
"url": "https://github.com/sponsors/ljharb" "url": "https://github.com/sponsors/ljharb"
} }
}, },
"node_modules/cheerio": { "node_modules/chownr": {
"version": "1.1.2", "version": "1.1.4",
"resolved": "https://registry.npmmirror.com/cheerio/-/cheerio-1.1.2.tgz", "resolved": "https://registry.npmmirror.com/chownr/-/chownr-1.1.4.tgz",
"integrity": "sha512-IkxPpb5rS/d1IiLbHMgfPuS0FgiWTtFIm/Nj+2woXDLTZ7fOT2eqzgYbdMlLweqlHbsZjxEChoVK+7iph7jyQg==", "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "MIT", "license": "ISC"
"dependencies": {
"cheerio-select": "^2.1.0",
"dom-serializer": "^2.0.0",
"domhandler": "^5.0.3",
"domutils": "^3.2.2",
"encoding-sniffer": "^0.2.1",
"htmlparser2": "^10.0.0",
"parse5": "^7.3.0",
"parse5-htmlparser2-tree-adapter": "^7.1.0",
"parse5-parser-stream": "^7.1.2",
"undici": "^7.12.0",
"whatwg-mimetype": "^4.0.0"
},
"engines": {
"node": ">=20.18.1"
},
"funding": {
"url": "https://github.com/cheeriojs/cheerio?sponsor=1"
}
},
"node_modules/cheerio-select": {
"version": "2.1.0",
"resolved": "https://registry.npmmirror.com/cheerio-select/-/cheerio-select-2.1.0.tgz",
"integrity": "sha512-9v9kG0LvzrlcungtnJtpGNxY+fzECQKhK4EGJX2vByejiMX84MFNQw4UxPJl3bFbTMw+Dfs37XaIkCwTZfLh4g==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-select": "^5.1.0",
"css-what": "^6.1.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3",
"domutils": "^3.0.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
}, },
"node_modules/combined-stream": { "node_modules/combined-stream": {
"version": "1.0.8", "version": "1.0.8",
@@ -288,34 +315,6 @@
"node": ">= 0.10" "node": ">= 0.10"
} }
}, },
"node_modules/css-select": {
"version": "5.2.2",
"resolved": "https://registry.npmmirror.com/css-select/-/css-select-5.2.2.tgz",
"integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-what": "^6.1.0",
"domhandler": "^5.0.2",
"domutils": "^3.0.1",
"nth-check": "^2.0.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/css-what": {
"version": "6.2.2",
"resolved": "https://registry.npmmirror.com/css-what/-/css-what-6.2.2.tgz",
"integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">= 6"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/debug": { "node_modules/debug": {
"version": "4.4.3", "version": "4.4.3",
"resolved": "https://registry.npmmirror.com/debug/-/debug-4.4.3.tgz", "resolved": "https://registry.npmmirror.com/debug/-/debug-4.4.3.tgz",
@@ -333,6 +332,30 @@
} }
} }
}, },
"node_modules/decompress-response": {
"version": "6.0.0",
"resolved": "https://registry.npmmirror.com/decompress-response/-/decompress-response-6.0.0.tgz",
"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
"license": "MIT",
"dependencies": {
"mimic-response": "^3.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/deep-extend": {
"version": "0.6.0",
"resolved": "https://registry.npmmirror.com/deep-extend/-/deep-extend-0.6.0.tgz",
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
"license": "MIT",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/delayed-stream": { "node_modules/delayed-stream": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmmirror.com/delayed-stream/-/delayed-stream-1.0.0.tgz", "resolved": "https://registry.npmmirror.com/delayed-stream/-/delayed-stream-1.0.0.tgz",
@@ -351,59 +374,13 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/dom-serializer": { "node_modules/detect-libc": {
"version": "2.0.0", "version": "2.1.2",
"resolved": "https://registry.npmmirror.com/dom-serializer/-/dom-serializer-2.0.0.tgz", "resolved": "https://registry.npmmirror.com/detect-libc/-/detect-libc-2.1.2.tgz",
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==", "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
"license": "MIT", "license": "Apache-2.0",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.2",
"entities": "^4.2.0"
},
"funding": {
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/domelementtype": {
"version": "2.3.0",
"resolved": "https://registry.npmmirror.com/domelementtype/-/domelementtype-2.3.0.tgz",
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause"
},
"node_modules/domhandler": {
"version": "5.0.3",
"resolved": "https://registry.npmmirror.com/domhandler/-/domhandler-5.0.3.tgz",
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^2.3.0"
},
"engines": { "engines": {
"node": ">= 4" "node": ">=8"
},
"funding": {
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/domutils": {
"version": "3.2.2",
"resolved": "https://registry.npmmirror.com/domutils/-/domutils-3.2.2.tgz",
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^2.0.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3"
},
"funding": {
"url": "https://github.com/fb55/domutils?sponsor=1"
} }
}, },
"node_modules/dotenv": { "node_modules/dotenv": {
@@ -447,29 +424,13 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/encoding-sniffer": { "node_modules/end-of-stream": {
"version": "0.2.1", "version": "1.4.5",
"resolved": "https://registry.npmmirror.com/encoding-sniffer/-/encoding-sniffer-0.2.1.tgz", "resolved": "https://registry.npmmirror.com/end-of-stream/-/end-of-stream-1.4.5.tgz",
"integrity": "sha512-5gvq20T6vfpekVtqrYQsSCFZ1wEg5+wW0/QaZMWkFr6BqD3NfKs0rLCx4rrVlSWJeZb5NBJgVLswK/w2MWU+Gw==", "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"iconv-lite": "^0.6.3", "once": "^1.4.0"
"whatwg-encoding": "^3.1.1"
},
"funding": {
"url": "https://github.com/fb55/encoding-sniffer?sponsor=1"
}
},
"node_modules/entities": {
"version": "4.5.0",
"resolved": "https://registry.npmmirror.com/entities/-/entities-4.5.0.tgz",
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
} }
}, },
"node_modules/es-define-property": { "node_modules/es-define-property": {
@@ -532,6 +493,15 @@
"node": ">= 0.6" "node": ">= 0.6"
} }
}, },
"node_modules/expand-template": {
"version": "2.0.3",
"resolved": "https://registry.npmmirror.com/expand-template/-/expand-template-2.0.3.tgz",
"integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
"license": "(MIT OR WTFPL)",
"engines": {
"node": ">=6"
}
},
"node_modules/express": { "node_modules/express": {
"version": "5.2.1", "version": "5.2.1",
"resolved": "https://registry.npmmirror.com/express/-/express-5.2.1.tgz", "resolved": "https://registry.npmmirror.com/express/-/express-5.2.1.tgz",
@@ -600,6 +570,12 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/file-uri-to-path": {
"version": "1.0.0",
"resolved": "https://registry.npmmirror.com/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
"integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
"license": "MIT"
},
"node_modules/finalhandler": { "node_modules/finalhandler": {
"version": "2.1.1", "version": "2.1.1",
"resolved": "https://registry.npmmirror.com/finalhandler/-/finalhandler-2.1.1.tgz", "resolved": "https://registry.npmmirror.com/finalhandler/-/finalhandler-2.1.1.tgz",
@@ -675,6 +651,12 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/fs-constants": {
"version": "1.0.0",
"resolved": "https://registry.npmmirror.com/fs-constants/-/fs-constants-1.0.0.tgz",
"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
"license": "MIT"
},
"node_modules/function-bind": { "node_modules/function-bind": {
"version": "1.1.2", "version": "1.1.2",
"resolved": "https://registry.npmmirror.com/function-bind/-/function-bind-1.1.2.tgz", "resolved": "https://registry.npmmirror.com/function-bind/-/function-bind-1.1.2.tgz",
@@ -721,6 +703,12 @@
"node": ">= 0.4" "node": ">= 0.4"
} }
}, },
"node_modules/github-from-package": {
"version": "0.0.0",
"resolved": "https://registry.npmmirror.com/github-from-package/-/github-from-package-0.0.0.tgz",
"integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
"license": "MIT"
},
"node_modules/gopd": { "node_modules/gopd": {
"version": "1.2.0", "version": "1.2.0",
"resolved": "https://registry.npmmirror.com/gopd/-/gopd-1.2.0.tgz", "resolved": "https://registry.npmmirror.com/gopd/-/gopd-1.2.0.tgz",
@@ -772,37 +760,6 @@
"node": ">= 0.4" "node": ">= 0.4"
} }
}, },
"node_modules/htmlparser2": {
"version": "10.0.0",
"resolved": "https://registry.npmmirror.com/htmlparser2/-/htmlparser2-10.0.0.tgz",
"integrity": "sha512-TwAZM+zE5Tq3lrEHvOlvwgj1XLWQCtaaibSN11Q+gGBAS7Y1uZSWwXXRe4iF6OXnaq1riyQAPFOBtYc77Mxq0g==",
"funding": [
"https://github.com/fb55/htmlparser2?sponsor=1",
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3",
"domutils": "^3.2.1",
"entities": "^6.0.0"
}
},
"node_modules/htmlparser2/node_modules/entities": {
"version": "6.0.1",
"resolved": "https://registry.npmmirror.com/entities/-/entities-6.0.1.tgz",
"integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/http-errors": { "node_modules/http-errors": {
"version": "2.0.1", "version": "2.0.1",
"resolved": "https://registry.npmmirror.com/http-errors/-/http-errors-2.0.1.tgz", "resolved": "https://registry.npmmirror.com/http-errors/-/http-errors-2.0.1.tgz",
@@ -823,17 +780,25 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/iconv-lite": { "node_modules/ieee754": {
"version": "0.6.3", "version": "1.2.1",
"resolved": "https://registry.npmmirror.com/iconv-lite/-/iconv-lite-0.6.3.tgz", "resolved": "https://registry.npmmirror.com/ieee754/-/ieee754-1.2.1.tgz",
"integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==", "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
"license": "MIT", "funding": [
"dependencies": { {
"safer-buffer": ">= 2.1.2 < 3.0.0" "type": "github",
}, "url": "https://github.com/sponsors/feross"
"engines": { },
"node": ">=0.10.0" {
} "type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "BSD-3-Clause"
}, },
"node_modules/inherits": { "node_modules/inherits": {
"version": "2.0.4", "version": "2.0.4",
@@ -841,6 +806,12 @@
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/ini": {
"version": "1.3.8",
"resolved": "https://registry.npmmirror.com/ini/-/ini-1.3.8.tgz",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"license": "ISC"
},
"node_modules/ipaddr.js": { "node_modules/ipaddr.js": {
"version": "1.9.1", "version": "1.9.1",
"resolved": "https://registry.npmmirror.com/ipaddr.js/-/ipaddr.js-1.9.1.tgz", "resolved": "https://registry.npmmirror.com/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -907,12 +878,45 @@
"node": ">= 0.6" "node": ">= 0.6"
} }
}, },
"node_modules/mimic-response": {
"version": "3.1.0",
"resolved": "https://registry.npmmirror.com/mimic-response/-/mimic-response-3.1.0.tgz",
"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmmirror.com/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/mkdirp-classic": {
"version": "0.5.3",
"resolved": "https://registry.npmmirror.com/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
"license": "MIT"
},
"node_modules/ms": { "node_modules/ms": {
"version": "2.1.3", "version": "2.1.3",
"resolved": "https://registry.npmmirror.com/ms/-/ms-2.1.3.tgz", "resolved": "https://registry.npmmirror.com/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/napi-build-utils": {
"version": "1.0.2",
"resolved": "https://registry.npmmirror.com/napi-build-utils/-/napi-build-utils-1.0.2.tgz",
"integrity": "sha512-ONmRUqK7zj7DWX0D9ADe03wbwOBZxNAfF20PlGfCWQcD3+/MakShIHrMqx9YwPTfxDdF1zLeL+RGZiR9kGMLdg==",
"license": "MIT"
},
"node_modules/negotiator": { "node_modules/negotiator": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmmirror.com/negotiator/-/negotiator-1.0.0.tgz", "resolved": "https://registry.npmmirror.com/negotiator/-/negotiator-1.0.0.tgz",
@@ -922,6 +926,18 @@
"node": ">= 0.6" "node": ">= 0.6"
} }
}, },
"node_modules/node-abi": {
"version": "3.89.0",
"resolved": "https://registry.npmmirror.com/node-abi/-/node-abi-3.89.0.tgz",
"integrity": "sha512-6u9UwL0HlAl21+agMN3YAMXcKByMqwGx+pq+P76vii5f7hTPtKDp08/H9py6DY+cfDw7kQNTGEj/rly3IgbNQA==",
"license": "MIT",
"dependencies": {
"semver": "^7.3.5"
},
"engines": {
"node": ">=10"
}
},
"node_modules/node-cron": { "node_modules/node-cron": {
"version": "4.2.1", "version": "4.2.1",
"resolved": "https://registry.npmmirror.com/node-cron/-/node-cron-4.2.1.tgz", "resolved": "https://registry.npmmirror.com/node-cron/-/node-cron-4.2.1.tgz",
@@ -940,18 +956,6 @@
"node": ">=6.0.0" "node": ">=6.0.0"
} }
}, },
"node_modules/nth-check": {
"version": "2.1.1",
"resolved": "https://registry.npmmirror.com/nth-check/-/nth-check-2.1.1.tgz",
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0"
},
"funding": {
"url": "https://github.com/fb55/nth-check?sponsor=1"
}
},
"node_modules/object-assign": { "node_modules/object-assign": {
"version": "4.1.1", "version": "4.1.1",
"resolved": "https://registry.npmmirror.com/object-assign/-/object-assign-4.1.1.tgz", "resolved": "https://registry.npmmirror.com/object-assign/-/object-assign-4.1.1.tgz",
@@ -994,55 +998,6 @@
"wrappy": "1" "wrappy": "1"
} }
}, },
"node_modules/parse5": {
"version": "7.3.0",
"resolved": "https://registry.npmmirror.com/parse5/-/parse5-7.3.0.tgz",
"integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==",
"license": "MIT",
"dependencies": {
"entities": "^6.0.0"
},
"funding": {
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/parse5-htmlparser2-tree-adapter": {
"version": "7.1.0",
"resolved": "https://registry.npmmirror.com/parse5-htmlparser2-tree-adapter/-/parse5-htmlparser2-tree-adapter-7.1.0.tgz",
"integrity": "sha512-ruw5xyKs6lrpo9x9rCZqZZnIUntICjQAd0Wsmp396Ul9lN/h+ifgVV1x1gZHi8euej6wTfpqX8j+BFQxF0NS/g==",
"license": "MIT",
"dependencies": {
"domhandler": "^5.0.3",
"parse5": "^7.0.0"
},
"funding": {
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/parse5-parser-stream": {
"version": "7.1.2",
"resolved": "https://registry.npmmirror.com/parse5-parser-stream/-/parse5-parser-stream-7.1.2.tgz",
"integrity": "sha512-JyeQc9iwFLn5TbvvqACIF/VXG6abODeB3Fwmv/TGdLk2LfbWkaySGY72at4+Ty7EkPZj854u4CrICqNk2qIbow==",
"license": "MIT",
"dependencies": {
"parse5": "^7.0.0"
},
"funding": {
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/parse5/node_modules/entities": {
"version": "6.0.1",
"resolved": "https://registry.npmmirror.com/entities/-/entities-6.0.1.tgz",
"integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/parseurl": { "node_modules/parseurl": {
"version": "1.3.3", "version": "1.3.3",
"resolved": "https://registry.npmmirror.com/parseurl/-/parseurl-1.3.3.tgz", "resolved": "https://registry.npmmirror.com/parseurl/-/parseurl-1.3.3.tgz",
@@ -1062,6 +1017,32 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/prebuild-install": {
"version": "7.1.2",
"resolved": "https://registry.npmmirror.com/prebuild-install/-/prebuild-install-7.1.2.tgz",
"integrity": "sha512-UnNke3IQb6sgarcZIDU3gbMeTp/9SSU1DAIkil7PrqG1vZlBtY5msYccSKSHDqa3hNg436IXK+SNImReuA1wEQ==",
"license": "MIT",
"dependencies": {
"detect-libc": "^2.0.0",
"expand-template": "^2.0.3",
"github-from-package": "0.0.0",
"minimist": "^1.2.3",
"mkdirp-classic": "^0.5.3",
"napi-build-utils": "^1.0.1",
"node-abi": "^3.3.0",
"pump": "^3.0.0",
"rc": "^1.2.7",
"simple-get": "^4.0.0",
"tar-fs": "^2.0.0",
"tunnel-agent": "^0.6.0"
},
"bin": {
"prebuild-install": "bin.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/proxy-addr": { "node_modules/proxy-addr": {
"version": "2.0.7", "version": "2.0.7",
"resolved": "https://registry.npmmirror.com/proxy-addr/-/proxy-addr-2.0.7.tgz", "resolved": "https://registry.npmmirror.com/proxy-addr/-/proxy-addr-2.0.7.tgz",
@@ -1081,6 +1062,16 @@
"integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==", "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/pump": {
"version": "3.0.4",
"resolved": "https://registry.npmmirror.com/pump/-/pump-3.0.4.tgz",
"integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
"license": "MIT",
"dependencies": {
"end-of-stream": "^1.1.0",
"once": "^1.3.1"
}
},
"node_modules/qs": { "node_modules/qs": {
"version": "6.14.0", "version": "6.14.0",
"resolved": "https://registry.npmmirror.com/qs/-/qs-6.14.0.tgz", "resolved": "https://registry.npmmirror.com/qs/-/qs-6.14.0.tgz",
@@ -1136,6 +1127,35 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/rc": {
"version": "1.2.8",
"resolved": "https://registry.npmmirror.com/rc/-/rc-1.2.8.tgz",
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
"dependencies": {
"deep-extend": "^0.6.0",
"ini": "~1.3.0",
"minimist": "^1.2.0",
"strip-json-comments": "~2.0.1"
},
"bin": {
"rc": "cli.js"
}
},
"node_modules/readable-stream": {
"version": "3.6.2",
"resolved": "https://registry.npmmirror.com/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/router": { "node_modules/router": {
"version": "2.2.0", "version": "2.2.0",
"resolved": "https://registry.npmmirror.com/router/-/router-2.2.0.tgz", "resolved": "https://registry.npmmirror.com/router/-/router-2.2.0.tgz",
@@ -1152,12 +1172,44 @@
"node": ">= 18" "node": ">= 18"
} }
}, },
"node_modules/safe-buffer": {
"version": "5.2.1",
"resolved": "https://registry.npmmirror.com/safe-buffer/-/safe-buffer-5.2.1.tgz",
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/safer-buffer": { "node_modules/safer-buffer": {
"version": "2.1.2", "version": "2.1.2",
"resolved": "https://registry.npmmirror.com/safer-buffer/-/safer-buffer-2.1.2.tgz", "resolved": "https://registry.npmmirror.com/safer-buffer/-/safer-buffer-2.1.2.tgz",
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/semver": {
"version": "7.7.4",
"resolved": "https://registry.npmmirror.com/semver/-/semver-7.7.4.tgz",
"integrity": "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA==",
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/send": { "node_modules/send": {
"version": "1.2.0", "version": "1.2.0",
"resolved": "https://registry.npmmirror.com/send/-/send-1.2.0.tgz", "resolved": "https://registry.npmmirror.com/send/-/send-1.2.0.tgz",
@@ -1298,6 +1350,51 @@
"url": "https://github.com/sponsors/ljharb" "url": "https://github.com/sponsors/ljharb"
} }
}, },
"node_modules/simple-concat": {
"version": "1.0.1",
"resolved": "https://registry.npmmirror.com/simple-concat/-/simple-concat-1.0.1.tgz",
"integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/simple-get": {
"version": "4.0.1",
"resolved": "https://registry.npmmirror.com/simple-get/-/simple-get-4.0.1.tgz",
"integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"decompress-response": "^6.0.0",
"once": "^1.3.1",
"simple-concat": "^1.0.0"
}
},
"node_modules/statuses": { "node_modules/statuses": {
"version": "2.0.2", "version": "2.0.2",
"resolved": "https://registry.npmmirror.com/statuses/-/statuses-2.0.2.tgz", "resolved": "https://registry.npmmirror.com/statuses/-/statuses-2.0.2.tgz",
@@ -1307,6 +1404,52 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/string_decoder": {
"version": "1.3.0",
"resolved": "https://registry.npmmirror.com/string_decoder/-/string_decoder-1.3.0.tgz",
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
"license": "MIT",
"dependencies": {
"safe-buffer": "~5.2.0"
}
},
"node_modules/strip-json-comments": {
"version": "2.0.1",
"resolved": "https://registry.npmmirror.com/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/tar-fs": {
"version": "2.1.4",
"resolved": "https://registry.npmmirror.com/tar-fs/-/tar-fs-2.1.4.tgz",
"integrity": "sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ==",
"license": "MIT",
"dependencies": {
"chownr": "^1.1.1",
"mkdirp-classic": "^0.5.2",
"pump": "^3.0.0",
"tar-stream": "^2.1.4"
}
},
"node_modules/tar-stream": {
"version": "2.2.0",
"resolved": "https://registry.npmmirror.com/tar-stream/-/tar-stream-2.2.0.tgz",
"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
"license": "MIT",
"dependencies": {
"bl": "^4.0.3",
"end-of-stream": "^1.4.1",
"fs-constants": "^1.0.0",
"inherits": "^2.0.3",
"readable-stream": "^3.1.1"
},
"engines": {
"node": ">=6"
}
},
"node_modules/toidentifier": { "node_modules/toidentifier": {
"version": "1.0.1", "version": "1.0.1",
"resolved": "https://registry.npmmirror.com/toidentifier/-/toidentifier-1.0.1.tgz", "resolved": "https://registry.npmmirror.com/toidentifier/-/toidentifier-1.0.1.tgz",
@@ -1316,6 +1459,18 @@
"node": ">=0.6" "node": ">=0.6"
} }
}, },
"node_modules/tunnel-agent": {
"version": "0.6.0",
"resolved": "https://registry.npmmirror.com/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
"integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
"license": "Apache-2.0",
"dependencies": {
"safe-buffer": "^5.0.1"
},
"engines": {
"node": "*"
}
},
"node_modules/type-is": { "node_modules/type-is": {
"version": "2.0.1", "version": "2.0.1",
"resolved": "https://registry.npmmirror.com/type-is/-/type-is-2.0.1.tgz", "resolved": "https://registry.npmmirror.com/type-is/-/type-is-2.0.1.tgz",
@@ -1361,15 +1516,6 @@
"integrity": "sha512-TvkrTUpv7gCPlcnSoEwUVUBwsdheKm+HF5u2tPAKubkIGMfovdSizCTaZRY/NhR8+Ijy8iZZUapbVQAsNrkFrw==", "integrity": "sha512-TvkrTUpv7gCPlcnSoEwUVUBwsdheKm+HF5u2tPAKubkIGMfovdSizCTaZRY/NhR8+Ijy8iZZUapbVQAsNrkFrw==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/undici": {
"version": "7.16.0",
"resolved": "https://registry.npmmirror.com/undici/-/undici-7.16.0.tgz",
"integrity": "sha512-QEg3HPMll0o3t2ourKwOeUAZ159Kn9mx5pnzHRQO8+Wixmh88YdZRiIwat0iNzNNXn0yoEtXJqFpyW7eM8BV7g==",
"license": "MIT",
"engines": {
"node": ">=20.18.1"
}
},
"node_modules/unpipe": { "node_modules/unpipe": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmmirror.com/unpipe/-/unpipe-1.0.0.tgz", "resolved": "https://registry.npmmirror.com/unpipe/-/unpipe-1.0.0.tgz",
@@ -1379,6 +1525,12 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmmirror.com/util-deprecate/-/util-deprecate-1.0.2.tgz",
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
"license": "MIT"
},
"node_modules/vary": { "node_modules/vary": {
"version": "1.1.2", "version": "1.1.2",
"resolved": "https://registry.npmmirror.com/vary/-/vary-1.1.2.tgz", "resolved": "https://registry.npmmirror.com/vary/-/vary-1.1.2.tgz",
@@ -1388,27 +1540,6 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/whatwg-encoding": {
"version": "3.1.1",
"resolved": "https://registry.npmmirror.com/whatwg-encoding/-/whatwg-encoding-3.1.1.tgz",
"integrity": "sha512-6qN4hJdMwfYBtE3YBTTHhoeuUrDBPZmbQaxWAqSALV/MeEnR5z1xd8UKud2RAkFoPkmB+hli1TZSnyi84xz1vQ==",
"license": "MIT",
"dependencies": {
"iconv-lite": "0.6.3"
},
"engines": {
"node": ">=18"
}
},
"node_modules/whatwg-mimetype": {
"version": "4.0.0",
"resolved": "https://registry.npmmirror.com/whatwg-mimetype/-/whatwg-mimetype-4.0.0.tgz",
"integrity": "sha512-QaKxh0eNIi2mE9p2vEdzfagOKHCcj1pJ56EEHGQOVxp8r9/iszLUUV7v89x9O1p/T+NlTM5W7jW6+cz4Fq1YVg==",
"license": "MIT",
"engines": {
"node": ">=18"
}
},
"node_modules/wrappy": { "node_modules/wrappy": {
"version": "1.0.2", "version": "1.0.2",
"resolved": "https://registry.npmmirror.com/wrappy/-/wrappy-1.0.2.tgz", "resolved": "https://registry.npmmirror.com/wrappy/-/wrappy-1.0.2.tgz",
@@ -1416,9 +1547,9 @@
"license": "ISC" "license": "ISC"
}, },
"node_modules/zod": { "node_modules/zod": {
"version": "4.3.6", "version": "3.25.76",
"resolved": "https://registry.npmmirror.com/zod/-/zod-4.3.6.tgz", "resolved": "https://registry.npmmirror.com/zod/-/zod-3.25.76.tgz",
"integrity": "sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg==", "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==",
"license": "MIT", "license": "MIT",
"funding": { "funding": {
"url": "https://github.com/sponsors/colinhacks" "url": "https://github.com/sponsors/colinhacks"

View File

@@ -9,6 +9,7 @@
}, },
"dependencies": { "dependencies": {
"@mendable/firecrawl-js": "latest", "@mendable/firecrawl-js": "latest",
"better-sqlite3": "^12.8.0",
"cors": "^2.8.5", "cors": "^2.8.5",
"dotenv": "^17.2.3", "dotenv": "^17.2.3",
"express": "^5.2.1", "express": "^5.2.1",
@@ -16,4 +17,4 @@
"nodemailer": "^7.0.11", "nodemailer": "^7.0.11",
"zod": "^3.24.2" "zod": "^3.24.2"
} }
} }

View File

@@ -1063,7 +1063,7 @@ async function generateCombinedReport() {
function displayCombinedReport(winningReport, bidReport, container) { function displayCombinedReport(winningReport, bidReport, container) {
const html = ` const html = `
<!-- 中标公示部分 --> <!-- 中标公示部分 -->
<div class="summary" style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);"> <div class="summary" style="background: linear-gradient(135deg, #0f6ecd 0%, #0ea5a4 100%);">
<h2>中标公示报告</h2> <h2>中标公示报告</h2>
<div class="stat"> <div class="stat">
<div class="stat-label">总项目数</div> <div class="stat-label">总项目数</div>

File diff suppressed because it is too large Load Diff

View File

@@ -6,6 +6,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>抓取结果查看 - 公告采集工具</title> <title>抓取结果查看 - 公告采集工具</title>
<style> <style>
@import url('https://fonts.googleapis.com/css2?family=Fira+Sans:wght@400;500;600;700&family=Fira+Code:wght@500;600&family=Noto+Sans+SC:wght@400;500;700&display=swap');
* { * {
margin: 0; margin: 0;
padding: 0; padding: 0;
@@ -44,7 +46,7 @@
.topbar-logo { .topbar-logo {
width: 36px; width: 36px;
height: 36px; height: 36px;
background: linear-gradient(135deg, #667eea, #764ba2); background: linear-gradient(135deg, #0f6ecd, #0ea5a4);
border-radius: 8px; border-radius: 8px;
display: flex; display: flex;
align-items: center; align-items: center;
@@ -79,8 +81,8 @@
} }
.topbar-link.active { .topbar-link.active {
background: rgba(102, 126, 234, 0.25); background: rgba(15, 110, 205, 0.25);
color: #8fa8f8; color: #5ea2e8;
} }
/* ===== 主体内容 ===== */ /* ===== 主体内容 ===== */
@@ -128,7 +130,7 @@
.stat-card:hover { .stat-card:hover {
background: rgba(255, 255, 255, 0.08); background: rgba(255, 255, 255, 0.08);
border-color: rgba(102, 126, 234, 0.4); border-color: rgba(15, 110, 205, 0.4);
} }
.stat-card .label { .stat-card .label {
@@ -192,8 +194,8 @@
.filter-select:focus, .filter-select:focus,
.filter-input:focus { .filter-input:focus {
border-color: #667eea; border-color: #0f6ecd;
box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.15); box-shadow: 0 0 0 3px rgba(15, 110, 205, 0.15);
} }
.filter-select option { .filter-select option {
@@ -224,14 +226,14 @@
} }
.btn-primary { .btn-primary {
background: linear-gradient(135deg, #667eea, #764ba2); background: linear-gradient(135deg, #0f6ecd, #0ea5a4);
color: #fff; color: #fff;
} }
.btn-primary:hover { .btn-primary:hover {
opacity: 0.9; opacity: 0.9;
transform: translateY(-1px); transform: translateY(-1px);
box-shadow: 0 4px 14px rgba(102, 126, 234, 0.4); box-shadow: 0 4px 14px rgba(15, 110, 205, 0.4);
} }
.btn-danger { .btn-danger {
@@ -286,7 +288,7 @@
top: 0; top: 0;
bottom: 0; bottom: 0;
width: 4px; width: 4px;
background: linear-gradient(180deg, #667eea, #764ba2); background: linear-gradient(180deg, #0f6ecd, #0ea5a4);
border-radius: 14px 0 0 14px; border-radius: 14px 0 0 14px;
} }
@@ -296,7 +298,7 @@
.result-card:hover { .result-card:hover {
background: rgba(255, 255, 255, 0.07); background: rgba(255, 255, 255, 0.07);
border-color: rgba(102, 126, 234, 0.3); border-color: rgba(15, 110, 205, 0.3);
} }
.result-card-header { .result-card-header {
@@ -323,9 +325,9 @@
} }
.tag-city { .tag-city {
background: rgba(102, 126, 234, 0.2); background: rgba(15, 110, 205, 0.2);
color: #8fa8f8; color: #5ea2e8;
border: 1px solid rgba(102, 126, 234, 0.3); border: 1px solid rgba(15, 110, 205, 0.3);
} }
.tag-section { .tag-section {
@@ -353,7 +355,7 @@
} }
.result-url a { .result-url a {
color: #6fa3ff; color: #3c88d6;
text-decoration: none; text-decoration: none;
font-size: 13px; font-size: 13px;
word-break: break-all; word-break: break-all;
@@ -437,8 +439,8 @@
} }
.data-table th { .data-table th {
background: rgba(102, 126, 234, 0.15); background: rgba(15, 110, 205, 0.15);
color: #8fa8f8; color: #5ea2e8;
font-weight: 600; font-weight: 600;
white-space: nowrap; white-space: nowrap;
} }
@@ -452,7 +454,7 @@
} }
.data-table td a { .data-table td a {
color: #6fa3ff; color: #3c88d6;
text-decoration: none; text-decoration: none;
} }
@@ -519,15 +521,15 @@
} }
.page-btn:hover:not(:disabled) { .page-btn:hover:not(:disabled) {
background: rgba(102, 126, 234, 0.25); background: rgba(15, 110, 205, 0.25);
color: #8fa8f8; color: #5ea2e8;
border-color: rgba(102, 126, 234, 0.4); border-color: rgba(15, 110, 205, 0.4);
} }
.page-btn.active-page { .page-btn.active-page {
background: rgba(102, 126, 234, 0.3); background: rgba(15, 110, 205, 0.3);
color: #8fa8f8; color: #5ea2e8;
border-color: rgba(102, 126, 234, 0.5); border-color: rgba(15, 110, 205, 0.5);
font-weight: 700; font-weight: 700;
} }
@@ -580,8 +582,8 @@
display: inline-block; display: inline-block;
width: 40px; width: 40px;
height: 40px; height: 40px;
border: 3px solid rgba(102, 126, 234, 0.2); border: 3px solid rgba(15, 110, 205, 0.2);
border-top-color: #667eea; border-top-color: #0f6ecd;
border-radius: 50%; border-radius: 50%;
animation: spin 0.8s linear infinite; animation: spin 0.8s linear infinite;
margin-bottom: 12px; margin-bottom: 12px;
@@ -790,26 +792,330 @@
grid-template-columns: repeat(2, 1fr); grid-template-columns: repeat(2, 1fr);
} }
} }
</style>
/* ===== Glass Theme Override (No Purple) ===== */
:root {
--primary: #0f6ecd;
--primary-soft: rgba(15, 110, 205, 0.18);
--secondary: #0ea5a4;
--accent: #f59e0b;
--text: #112941;
--muted: #536b86;
--line: rgba(17, 41, 65, 0.14);
--glass: rgba(255, 255, 255, 0.62);
--glass-strong: rgba(255, 255, 255, 0.82);
--danger: #d94848;
}
body {
font-family: 'Fira Sans', 'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
color: var(--text);
background:
radial-gradient(900px 500px at -5% -5%, rgba(15, 110, 205, 0.20), transparent 55%),
radial-gradient(900px 520px at 108% -10%, rgba(14, 165, 164, 0.18), transparent 52%),
linear-gradient(145deg, #edf6ff 0%, #e8fbf6 50%, #f6fbff 100%);
}
.topbar {
background: var(--glass);
border-bottom: 1px solid var(--line);
backdrop-filter: blur(14px);
-webkit-backdrop-filter: blur(14px);
}
.topbar-logo {
background: linear-gradient(125deg, var(--primary), var(--secondary));
color: #fff;
font-family: 'Fira Code', sans-serif;
font-size: 14px;
font-weight: 700;
letter-spacing: 0.4px;
}
.topbar-title { color: #1a3f65; }
.topbar-link {
color: var(--muted);
border: 1px solid transparent;
}
.topbar-link:hover {
color: var(--text);
background: rgba(255, 255, 255, 0.7);
border-color: var(--line);
}
.topbar-link.active {
color: var(--primary);
background: rgba(15, 110, 205, 0.12);
border-color: rgba(15, 110, 205, 0.24);
}
.page-header h1 { color: #173d62; }
.page-header p { color: var(--muted); }
.stat-card,
.toolbar,
.result-card,
.dialog-box,
.detail-box {
background: var(--glass);
border-color: var(--line);
backdrop-filter: blur(14px);
-webkit-backdrop-filter: blur(14px);
box-shadow: 0 16px 36px rgba(15, 42, 74, 0.12);
}
.stat-card:hover,
.result-card:hover {
background: var(--glass-strong);
border-color: rgba(15, 110, 205, 0.26);
}
.stat-card .label,
.stat-card .sub,
.page-info,
.result-time,
.dialog-msg {
color: var(--muted);
}
.stat-card .value,
.detail-header h3,
.dialog-title,
.result-data-body h4,
.detail-body h4 {
color: #183f65;
}
.filter-group label { color: #304d6b; }
.filter-select,
.filter-input {
background: rgba(255, 255, 255, 0.76);
border-color: var(--line);
color: var(--text);
}
.filter-select:focus,
.filter-input:focus {
border-color: rgba(15, 110, 205, 0.5);
box-shadow: 0 0 0 3px rgba(15, 110, 205, 0.12);
}
.filter-select option {
background: #ffffff;
color: #1b3f62;
}
.btn {
border: 1px solid transparent;
transition: all 0.2s ease;
}
.btn-primary {
background: linear-gradient(125deg, var(--primary), #178ac2);
color: #fff;
box-shadow: 0 8px 16px rgba(15, 110, 205, 0.24);
}
.btn-primary:hover {
transform: translateY(-1px);
box-shadow: 0 12px 24px rgba(15, 110, 205, 0.26);
}
.btn-ghost {
color: #284867;
background: rgba(255, 255, 255, 0.76);
border-color: var(--line);
}
.btn-ghost:hover {
color: #163a60;
background: #ffffff;
}
.btn-danger {
color: #b73d3d;
background: rgba(217, 72, 72, 0.10);
border-color: rgba(217, 72, 72, 0.22);
}
.btn-danger:hover {
color: #fff;
background: linear-gradient(125deg, #d94848, #e36d61);
}
.result-card::before {
background: linear-gradient(180deg, var(--primary), var(--secondary));
}
.result-card.has-error::before {
background: linear-gradient(180deg, #d94848, #ed7a62);
}
.tag-city {
background: rgba(15, 110, 205, 0.12);
color: #0f5fae;
border-color: rgba(15, 110, 205, 0.24);
}
.tag-section {
background: rgba(14, 165, 164, 0.12);
color: #0b7a79;
border-color: rgba(14, 165, 164, 0.24);
}
.tag-type {
background: rgba(245, 158, 11, 0.16);
color: #b96f06;
border-color: rgba(245, 158, 11, 0.28);
}
.tag-error {
background: rgba(217, 72, 72, 0.14);
color: #bf3b3b;
border-color: rgba(217, 72, 72, 0.28);
}
.result-url a,
.data-table td a {
color: var(--primary);
}
.result-data {
background: rgba(255, 255, 255, 0.68);
border-color: var(--line);
}
.result-data-toggle {
color: #375677;
}
.result-data-toggle:hover {
color: #1b4168;
}
.result-data-body pre,
.detail-body pre {
color: #225068;
}
.data-table {
color: #1f3f5d;
}
.data-table th,
.data-table td {
border-color: rgba(17, 41, 65, 0.12);
}
.data-table th {
background: rgba(15, 110, 205, 0.12);
color: #185a9f;
}
.data-table tr:nth-child(even) {
background: rgba(255, 255, 255, 0.42);
}
.data-table tr:hover {
background: rgba(255, 255, 255, 0.72);
}
.result-card-footer {
border-top-color: rgba(17, 41, 65, 0.10);
}
.page-btn {
background: rgba(255, 255, 255, 0.72);
border-color: var(--line);
color: #2a4b6c;
}
.page-btn:hover:not(:disabled) {
background: rgba(15, 110, 205, 0.14);
color: var(--primary);
border-color: rgba(15, 110, 205, 0.25);
}
.page-btn.active-page {
background: rgba(15, 110, 205, 0.2);
color: #0f5fae;
border-color: rgba(15, 110, 205, 0.35);
}
.empty-state,
.empty-state h3,
.empty-state p {
color: #4a6481;
}
.spinner {
border-color: rgba(15, 110, 205, 0.18);
border-top-color: var(--primary);
}
.toast {
color: #fff;
box-shadow: 0 8px 24px rgba(17, 41, 65, 0.24);
}
.overlay,
.detail-overlay {
background: rgba(8, 20, 36, 0.34);
backdrop-filter: blur(4px);
-webkit-backdrop-filter: blur(4px);
}
.dialog-title,
.detail-header h3 {
font-family: 'Fira Code', 'Noto Sans SC', sans-serif;
}
.detail-header {
border-bottom-color: rgba(17, 41, 65, 0.10);
}
.close-btn {
background: rgba(255, 255, 255, 0.75);
color: #345777;
border: 1px solid var(--line);
}
.close-btn:hover {
background: #ffffff;
color: #1b4269;
}
@media (prefers-reduced-motion: reduce) {
* {
transition: none !important;
animation: none !important;
}
}
</style>
</head> </head>
<body> <body>
<!-- 顶部导航 --> <!-- 顶部导航 -->
<div class="topbar"> <div class="topbar">
<a href="/" class="topbar-brand"> <a href="/" class="topbar-brand">
<div class="topbar-logo">📋</div> <div class="topbar-logo">AG</div>
<span class="topbar-title">公告采集工具</span> <span class="topbar-title">公告采集工具</span>
</a> </a>
<div class="topbar-nav"> <div class="topbar-nav">
<a href="/" class="topbar-link">⚙️ 配置管理</a> <a href="/" class="topbar-link"> 配置管理</a>
<a href="/results.html" class="topbar-link active">📊 抓取结果</a> <a href="/results.html" class="topbar-link active"> 抓取结果</a>
</div> </div>
</div> </div>
<div class="main"> <div class="main">
<!-- 页头 --> <!-- 页头 -->
<div class="page-header"> <div class="page-header">
<h1>📊 抓取结果</h1> <h1> 抓取结果</h1>
<p>所有抓取来源的历史结果,按抓取时间倒序展示</p> <p>所有抓取来源的历史结果,按抓取时间倒序展示</p>
</div> </div>
@@ -859,8 +1165,8 @@
</div> </div>
<div class="toolbar-actions"> <div class="toolbar-actions">
<button class="btn btn-ghost" onclick="resetFilters()">↺ 重置</button> <button class="btn btn-ghost" onclick="resetFilters()">↺ 重置</button>
<button class="btn btn-primary" onclick="loadResults()">🔄 刷新</button> <button class="btn btn-primary" onclick="loadResults()"> 刷新</button>
<button class="btn btn-danger" onclick="confirmClearAll()">🗑 清空全部</button> <!-- <button class="btn btn-danger" onclick="confirmClearAll()"> 清空全部</button> -->
</div> </div>
</div> </div>
@@ -883,7 +1189,7 @@
<!-- 确认清空弹窗 --> <!-- 确认清空弹窗 -->
<div class="overlay" id="confirmOverlay"> <div class="overlay" id="confirmOverlay">
<div class="dialog-box"> <div class="dialog-box">
<div class="dialog-title">⚠️ 确认清空</div> <div class="dialog-title"> 确认清空</div>
<div class="dialog-msg" id="confirmMsg">确定要清空所有抓取结果吗?此操作不可撤销。</div> <div class="dialog-msg" id="confirmMsg">确定要清空所有抓取结果吗?此操作不可撤销。</div>
<div class="dialog-actions"> <div class="dialog-actions">
<button class="btn btn-ghost" onclick="closeConfirm()">取消</button> <button class="btn btn-ghost" onclick="closeConfirm()">取消</button>
@@ -993,7 +1299,7 @@
renderResults(json.data || []); renderResults(json.data || []);
renderPagination(json.total, json.page, json.pageSize); renderPagination(json.total, json.page, json.pageSize);
} catch (e) { } catch (e) {
listEl.innerHTML = `<div class="empty-state"><div class="icon">⚠️</div><h3>加载失败</h3><p>${e.message}</p></div>`; listEl.innerHTML = `<div class="empty-state"><div class="icon"></div><h3>加载失败</h3><p>${e.message}</p></div>`;
} finally { } finally {
maskEl.classList.remove('show'); maskEl.classList.remove('show');
} }
@@ -1015,7 +1321,7 @@
if (data.length === 0) { if (data.length === 0) {
listEl.innerHTML = ` listEl.innerHTML = `
<div class="empty-state"> <div class="empty-state">
<div class="icon">🔍</div> <div class="icon"></div>
<h3>暂无抓取记录</h3> <h3>暂无抓取记录</h3>
<p>运行抓取来源后,结果将自动保存在这里</p> <p>运行抓取来源后,结果将自动保存在这里</p>
</div>`; </div>`;
@@ -1029,13 +1335,13 @@
${r.city ? `<span class="tag tag-city">${r.city}</span>` : ''} ${r.city ? `<span class="tag tag-city">${r.city}</span>` : ''}
${r.section ? `<span class="tag tag-section">${r.section}${r.subsection ? ' · ' + r.subsection : ''}</span>` : ''} ${r.section ? `<span class="tag tag-section">${r.section}${r.subsection ? ' · ' + r.subsection : ''}</span>` : ''}
${r.type ? `<span class="tag tag-type">${r.type}</span>` : ''} ${r.type ? `<span class="tag tag-type">${r.type}</span>` : ''}
${r.error ? `<span class="tag tag-error"> 失败</span>` : ''} ${r.error ? `<span class="tag tag-error"> 失败</span>` : ''}
</div> </div>
<span class="result-time">${formatTime(r.scrapedAt)}</span> <span class="result-time">${formatTime(r.scrapedAt)}</span>
</div> </div>
<div class="result-url"> <div class="result-url">
<a href="${r.url}" target="_blank" title="${r.url}">🔗 ${r.url}</a> <a href="${r.url}" target="_blank" title="${r.url}"> ${r.url}</a>
</div> </div>
${r.error ? `<div class="result-error">错误信息:${r.error}</div>` : ''} ${r.error ? `<div class="result-error">错误信息:${r.error}</div>` : ''}
@@ -1043,7 +1349,7 @@
${!r.error && r.data ? ` ${!r.error && r.data ? `
<div class="result-data"> <div class="result-data">
<button class="result-data-toggle" onclick="toggleData(this)"> <button class="result-data-toggle" onclick="toggleData(this)">
<span>📄 查看数据</span> <span> 查看数据</span>
<span>▼</span> <span>▼</span>
</button> </button>
<div class="result-data-body"> <div class="result-data-body">
@@ -1052,7 +1358,7 @@
</div>` : ''} </div>` : ''}
<div class="result-card-footer"> <div class="result-card-footer">
${!r.error && r.data ? `<button class="btn btn-ghost btn-sm" onclick="openDetail('${r.id}')">🔎 全屏查看</button>` : ''} ${!r.error && r.data ? `<button class="btn btn-ghost btn-sm" onclick="openDetail('${r.id}')"> 全屏查看</button>` : ''}
<button class="btn btn-danger btn-sm" onclick="deleteResult('${r.id}')">删除</button> <button class="btn btn-danger btn-sm" onclick="deleteResult('${r.id}')">删除</button>
</div> </div>
</div> </div>
@@ -1245,7 +1551,7 @@
else val = String(val); else val = String(val);
if ((val.startsWith('http://') || val.startsWith('https://')) && !val.includes(' ')) { if ((val.startsWith('http://') || val.startsWith('https://')) && !val.includes(' ')) {
html += `<td><a href="${escHtml(val)}" target="_blank" title="${escHtml(val)}">🔗 链接</a></td>`; html += `<td><a href="${escHtml(val)}" target="_blank" title="${escHtml(val)}"> 链接</a></td>`;
} else { } else {
html += `<td>${escHtml(val)}</td>`; html += `<td>${escHtml(val)}</td>`;
} }
@@ -1270,4 +1576,4 @@
</script> </script>
</body> </body>
</html> </html>

100
restart.ps1 Normal file
View File

@@ -0,0 +1,100 @@
$ErrorActionPreference = 'Stop'
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$LogDir = Join-Path $ScriptDir 'logs'
$PidFile = Join-Path $LogDir 'server.pid'
$LogFile = Join-Path $LogDir 'server.log'
New-Item -ItemType Directory -Force -Path $LogDir | Out-Null
function Get-PortFromEnvFile {
$envFile = Join-Path $ScriptDir '.env'
if (-not (Test-Path $envFile)) {
return $null
}
$line = Get-Content $envFile | Where-Object { $_ -match '^PORT=' } | Select-Object -Last 1
if (-not $line) {
return $null
}
return ($line -replace '^PORT=', '').Trim()
}
function Get-ProjectServerProcesses {
Get-CimInstance Win32_Process | Where-Object {
$_.Name -eq 'node.exe' -and
$_.CommandLine -like '*src/server.js*' -and
$_.CommandLine -like "*$ScriptDir*"
}
}
function Stop-ExistingServer {
$processes = @(Get-ProjectServerProcesses)
if ($processes.Count -eq 0) {
Write-Host "No existing server process found for $ScriptDir"
return
}
$ids = $processes | ForEach-Object { $_.ProcessId }
Write-Host ("Stopping existing server process(es): " + ($ids -join ', '))
foreach ($process in $processes) {
Stop-Process -Id $process.ProcessId -Force -ErrorAction SilentlyContinue
}
Start-Sleep -Seconds 2
}
function Start-Server {
$port = if ($env:PORT) { $env:PORT } else { Get-PortFromEnvFile }
if (-not $port) { $port = '5000' }
Write-Host "Starting server from $ScriptDir on port $port"
$command = "Set-Location -LiteralPath '$ScriptDir'; node src/server.js *>> '$LogFile'"
$process = Start-Process -FilePath 'powershell.exe' `
-ArgumentList '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', $command `
-WindowStyle Hidden `
-PassThru
Set-Content -Path $PidFile -Value $process.Id
Write-Host "Started PID: $($process.Id)"
return $port
}
function Show-Status {
param(
[string]$Port
)
Start-Sleep -Seconds 2
Write-Host ''
Write-Host 'Active project server process(es):'
$processes = @(Get-ProjectServerProcesses)
if ($processes.Count -eq 0) {
Write-Host 'None'
} else {
$processes | Select-Object ProcessId, Name, CommandLine | Format-Table -AutoSize
}
Write-Host ''
Write-Host 'Port check:'
Get-NetTCPConnection -LocalPort ([int]$Port) -State Listen -ErrorAction SilentlyContinue |
Select-Object LocalAddress, LocalPort, OwningProcess | Format-Table -AutoSize
Write-Host ''
Write-Host 'Recent log output:'
if (Test-Path $LogFile) {
Get-Content $LogFile -Tail 30
} else {
Write-Host 'No log file yet.'
}
}
Stop-ExistingServer
$port = Start-Server
Show-Status -Port $port

83
restart.sh Normal file
View File

@@ -0,0 +1,83 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LOG_DIR="$SCRIPT_DIR/logs"
PID_FILE="$LOG_DIR/server.pid"
LOG_FILE="$LOG_DIR/server.log"
mkdir -p "$LOG_DIR"
PORT="${PORT:-}"
if [[ -z "$PORT" && -f "$SCRIPT_DIR/.env" ]]; then
PORT="$(grep -E '^PORT=' "$SCRIPT_DIR/.env" | tail -n 1 | cut -d '=' -f 2- | tr -d '\r' || true)"
fi
PORT="${PORT:-5000}"
find_project_server_pids() {
pgrep -f "node src/server.js" | while read -r pid; do
[[ -n "$pid" ]] || continue
[[ -d "/proc/$pid" ]] || continue
local cwd
cwd="$(readlink -f "/proc/$pid/cwd" 2>/dev/null || true)"
if [[ "$cwd" == "$SCRIPT_DIR" ]]; then
echo "$pid"
fi
done
}
stop_existing_server() {
local pids
pids="$(find_project_server_pids || true)"
if [[ -z "$pids" ]]; then
echo "No existing server process found for $SCRIPT_DIR"
return
fi
echo "Stopping existing server process(es): $pids"
while read -r pid; do
[[ -n "$pid" ]] || continue
kill "$pid" 2>/dev/null || true
done <<< "$pids"
sleep 2
local remaining
remaining="$(find_project_server_pids || true)"
if [[ -n "$remaining" ]]; then
echo "Force killing remaining process(es): $remaining"
while read -r pid; do
[[ -n "$pid" ]] || continue
kill -9 "$pid" 2>/dev/null || true
done <<< "$remaining"
fi
}
start_server() {
cd "$SCRIPT_DIR"
echo "Starting server from $SCRIPT_DIR on port $PORT"
nohup node src/server.js >> "$LOG_FILE" 2>&1 &
local pid=$!
echo "$pid" > "$PID_FILE"
echo "Started PID: $pid"
}
show_status() {
sleep 2
echo
echo "Active project server process(es):"
find_project_server_pids || true
echo
echo "Port check:"
ss -lntp 2>/dev/null | grep ":$PORT" || true
echo
echo "Recent log output:"
tail -n 30 "$LOG_FILE" 2>/dev/null || true
}
stop_existing_server
start_server
show_status

View File

@@ -1 +0,0 @@
[]

151
src/agentService.js Normal file
View File

@@ -0,0 +1,151 @@
/**
* Agent API 服务封装
* 调用本地部署的 agent 进行公告抓取
*/
const DEFAULT_BASE_URL = 'http://192.168.3.65:18625';
const DEFAULT_POLL_INTERVAL = 3000; // 3秒轮询
const DEFAULT_TIMEOUT = 3600000; // 1小时超时
const FETCH_TIMEOUT = 30000; // 单次 fetch 30秒超时
const MAX_FETCH_RETRIES = 5; // 网络错误最多重试5次
const DEFAULT_MODE = 'qwen3.5-plus';
function normalizeMode(value) {
if (typeof value === 'string' && value.trim()) return value.trim();
return DEFAULT_MODE;
}
function generateTaskId() {
return `task-${Date.now()}-${Math.random().toString(36).slice(2, 7)}`;
}
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
/**
* 带超时和重试的 fetch
*/
async function fetchWithRetry(url, fetchOptions, retries = MAX_FETCH_RETRIES, logPrefix = '[Agent]') {
for (let attempt = 1; attempt <= retries; attempt++) {
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), FETCH_TIMEOUT);
const res = await fetch(url, { ...fetchOptions, signal: controller.signal });
clearTimeout(timer);
return res;
} catch (err) {
const isLast = attempt === retries;
console.warn(`${logPrefix} fetch 失败 (${attempt}/${retries}): ${err.message}`);
if (isLast) throw err;
await sleep(3000 * attempt); // 递增等待
}
}
}
/**
* 创建 agent 任务
*/
async function createTask(prompt, options = {}) {
const baseUrl = options.baseUrl || DEFAULT_BASE_URL;
const mode = normalizeMode(options.mode);
const taskId = generateTaskId();
const logPrefix = options.logPrefix || '[Agent]';
const res = await fetchWithRetry(`${baseUrl}/agent/createTask`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ taskId, prompt, mode }),
}, MAX_FETCH_RETRIES, logPrefix);
if (!res.ok) {
throw new Error(`创建任务失败: HTTP ${res.status}`);
}
return { taskId };
}
/**
* 检查任务状态
* 返回空/null 表示任务还在运行,返回 { success, message, data } 表示完成
*/
async function checkTask(taskId, options = {}) {
const baseUrl = options.baseUrl || DEFAULT_BASE_URL;
const logPrefix = options.logPrefix || '[Agent]';
const res = await fetchWithRetry(`${baseUrl}/agent/checkTask`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ taskId }),
}, MAX_FETCH_RETRIES, logPrefix);
if (!res.ok) {
throw new Error(`检查任务失败: HTTP ${res.status}`);
}
const text = await res.text();
console.log(`${logPrefix} checkTask(${taskId}) 返回:`, text ? text.substring(0, 500) : '(空)');
if (!text || text.trim() === '' || text.trim() === 'null') {
return null; // 任务还在运行
}
try {
return JSON.parse(text);
} catch {
return null;
}
}
/**
* 运行 agent 任务:创建 + 轮询直到完成
* 返回 { results: [{ type, project_name, amount_yuan, date, detail_link }] }
*/
export async function runAgentTask(prompt, options = {}) {
const baseUrl = options.baseUrl || DEFAULT_BASE_URL;
const mode = normalizeMode(options.mode);
const pollInterval = options.pollInterval || DEFAULT_POLL_INTERVAL;
const timeout = options.timeout || DEFAULT_TIMEOUT;
const logPrefix = options.logPrefix || '[Agent]';
console.log(`${logPrefix} 创建任务...`);
console.log(`${logPrefix} 使用 mode: ${mode}`);
const { taskId } = await createTask(prompt, { baseUrl, mode, logPrefix });
console.log(`${logPrefix} 任务已创建: ${taskId}`);
const startTime = Date.now();
while (true) {
if (Date.now() - startTime > timeout) {
throw new Error(`任务超时 (${timeout / 1000}秒): ${taskId}`);
}
await sleep(pollInterval);
const result = await checkTask(taskId, { baseUrl, logPrefix });
if (result === null) {
const elapsed = Math.round((Date.now() - startTime) / 1000);
console.log(`${logPrefix} 任务进行中... (${elapsed}秒)`);
continue;
}
if (result.success) {
console.log(`${logPrefix} 任务完成: ${result.message}`);
const data = result.data || {};
const rawResults = Array.isArray(data.results) ? data.results : [];
const results = rawResults.map(item => {
if (!item || typeof item !== 'object') return item;
const detailLink = item.detail_link || item.target_link;
const { target_link, ...rest } = item;
return detailLink ? { ...rest, detail_link: detailLink } : rest;
});
console.log(`${logPrefix} 获取到 ${results.length} 条结果`);
return { results };
} else {
throw new Error(`任务失败: ${result.message || '未知错误'}`);
}
}
}
export { generateTaskId, createTask, checkTask };

View File

@@ -767,23 +767,29 @@ function generateScraperResultsHtml(results) {
const failResults = results.filter(r => r.error); const failResults = results.filter(r => r.error);
const generatedAt = new Date().toLocaleString('zh-CN'); const generatedAt = new Date().toLocaleString('zh-CN');
// 把所有成功来源的 items 展开,附带来源信息 // Flatten all successful source items into one table.
const allRows = []; const allRows = [];
for (const r of successResults) { for (const r of successResults) {
const items = r.data?.result || []; const items = r.data?.results || r.data?.result || [];
for (const item of items) { for (const item of items) {
const hasAmount = typeof item.amount_yuan === 'number' || !!item.amount;
const amountText =
typeof item.amount_yuan === 'number'
? `${item.amount_yuan.toLocaleString('zh-CN')} CNY`
: (item.amount || 'N/A');
allRows.push({ allRows.push({
section: [r.section, r.subsection].filter(Boolean).join(' · ') || r.city || '-', section: [r.section, r.subsection].filter(Boolean).join(' / ') || r.city || '-',
type: r.type || '-', type: item.type || r.type || '-',
title: item.title || '-', title: item.project_name || item.title || '-',
date: item.date || '-', date: item.date || '-',
amount: item.amount || '未公开', amount: amountText,
url: item.url || '', hasAmount,
url: item.detail_link || item.target_link || item.url || '',
}); });
} }
} }
// 按日期降序排列
allRows.sort((a, b) => { allRows.sort((a, b) => {
if (a.date === b.date) return 0; if (a.date === b.date) return 0;
return a.date > b.date ? -1 : 1; return a.date > b.date ? -1 : 1;
@@ -802,7 +808,7 @@ function generateScraperResultsHtml(results) {
</td> </td>
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;font-size:13px;max-width:320px;">${row.title}</td> <td style="padding:9px 12px;border-bottom:1px solid #eaecf5;font-size:13px;max-width:320px;">${row.title}</td>
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;color:#555;">${row.date}</td> <td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;color:#555;">${row.date}</td>
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;font-weight:600;color:${row.amount === '未公开' ? '#aaa' : '#e67e22'};">${row.amount}</td> <td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;font-weight:600;color:${row.hasAmount ? '#e67e22' : '#aaa'};">${row.amount}</td>
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;text-align:center;"> <td style="padding:9px 12px;border-bottom:1px solid #eaecf5;text-align:center;">
${row.url ${row.url
? `<a href="${row.url}" target="_blank" style="color:#667eea;font-size:12px;text-decoration:none;white-space:nowrap;">查看 →</a>` ? `<a href="${row.url}" target="_blank" style="color:#667eea;font-size:12px;text-decoration:none;white-space:nowrap;">查看 →</a>`
@@ -851,7 +857,7 @@ function generateScraperResultsHtml(results) {
<div style="font-size:12px;color:#888;margin-top:2px;">成功来源</div> <div style="font-size:12px;color:#888;margin-top:2px;">成功来源</div>
</div> </div>
<div style="flex:1;padding:16px 24px;text-align:center;border-right:1px solid #eaecf5;"> <div style="flex:1;padding:16px 24px;text-align:center;border-right:1px solid #eaecf5;">
<div style="font-size:28px;font-weight:700;color:#e67e22;">${allRows.filter(r => r.amount && r.amount !== '未公开').length}</div> <div style="font-size:28px;font-weight:700;color:#e67e22;">${allRows.filter(r => r.hasAmount).length}</div>
<div style="font-size:12px;color:#888;margin-top:2px;">有金额</div> <div style="font-size:12px;color:#888;margin-top:2px;">有金额</div>
</div> </div>
<div style="flex:1;padding:16px 24px;text-align:center;"> <div style="flex:1;padding:16px 24px;text-align:center;">

817
src/resultStore.js Normal file
View File

@@ -0,0 +1,817 @@
import Database from 'better-sqlite3';
import {
copyFileSync,
existsSync,
mkdirSync,
readFileSync,
renameSync,
unlinkSync,
writeFileSync,
} from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const DEFAULT_DB_DIR = join(__dirname, '..', 'data');
const DEFAULT_DB_PATH = join(DEFAULT_DB_DIR, 'results.db');
const LEGACY_DB_PATH = join(DEFAULT_DB_DIR, 'results.sqlite');
const DB_PATH = resolveDbPath();
const CONFIG_PATH = join(__dirname, '..', 'config.json');
const MAX_RESULT_RECORDS = 500;
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
const TASK_COLUMNS = ['id', 'city', 'plate_name', 'prompt', 'enabled', 'mode', 'created_at', 'updated_at'];
let db = null;
let initialized = false;
function clone(value) {
return JSON.parse(JSON.stringify(value));
}
function removeFileIfExists(filePath) {
if (!existsSync(filePath)) return;
unlinkSync(filePath);
}
function migrateLegacyDbIfNeeded(nextPath, legacyPath) {
if (existsSync(nextPath) || !existsSync(legacyPath)) return nextPath;
const legacyDb = new Database(legacyPath);
try {
legacyDb.pragma('wal_checkpoint(TRUNCATE)');
} catch (_error) {
// Ignore checkpoint failures and still attempt to switch to single-file mode.
}
legacyDb.pragma('journal_mode = DELETE');
legacyDb.close();
try {
renameSync(legacyPath, nextPath);
} catch (_error) {
copyFileSync(legacyPath, nextPath);
}
removeFileIfExists(`${legacyPath}-shm`);
removeFileIfExists(`${legacyPath}-wal`);
return nextPath;
}
function resolveDbPath() {
const explicitPath = process.env.APP_DB_PATH || process.env.RESULTS_DB_PATH;
if (explicitPath) return explicitPath;
mkdirSync(DEFAULT_DB_DIR, { recursive: true });
try {
return migrateLegacyDbIfNeeded(DEFAULT_DB_PATH, LEGACY_DB_PATH);
} catch (error) {
if (existsSync(LEGACY_DB_PATH)) {
console.warn(`[resultStore] Legacy database migration skipped: ${error.message}`);
return LEGACY_DB_PATH;
}
throw error;
}
}
function generateResultId() {
return `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}`;
}
function generateTaskId() {
return `task-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`;
}
function getDefaultJsonConfig() {
return {
agent: {
baseUrl: '',
pollInterval: 3000,
timeout: 300000,
},
scheduler: {
enabled: false,
cronTime: '0 9 * * *',
threshold: 100000,
description: '',
timeRange: 'thisMonth',
},
email: {
smtpHost: '',
smtpPort: 587,
smtpUser: '',
smtpPass: '',
recipients: '',
},
};
}
function normalizeJsonConfig(input = {}) {
const defaults = getDefaultJsonConfig();
const agent = input.agent || {};
const scheduler = input.scheduler || {};
const email = input.email || {};
return {
agent: {
baseUrl: typeof agent.baseUrl === 'string' ? agent.baseUrl : defaults.agent.baseUrl,
pollInterval: Number.isFinite(agent.pollInterval) ? agent.pollInterval : defaults.agent.pollInterval,
timeout: Number.isFinite(agent.timeout) ? agent.timeout : defaults.agent.timeout,
},
scheduler: {
enabled: scheduler.enabled === true,
cronTime: typeof scheduler.cronTime === 'string' && scheduler.cronTime.trim()
? scheduler.cronTime
: defaults.scheduler.cronTime,
threshold: Number.isFinite(scheduler.threshold) ? scheduler.threshold : defaults.scheduler.threshold,
description: typeof scheduler.description === 'string' ? scheduler.description : defaults.scheduler.description,
timeRange: typeof scheduler.timeRange === 'string' && scheduler.timeRange.trim()
? scheduler.timeRange
: defaults.scheduler.timeRange,
},
email: {
smtpHost: typeof email.smtpHost === 'string' ? email.smtpHost : defaults.email.smtpHost,
smtpPort: Number.isFinite(email.smtpPort) ? email.smtpPort : defaults.email.smtpPort,
smtpUser: typeof email.smtpUser === 'string' ? email.smtpUser : defaults.email.smtpUser,
smtpPass: typeof email.smtpPass === 'string' ? email.smtpPass : defaults.email.smtpPass,
recipients: typeof email.recipients === 'string' ? email.recipients : defaults.email.recipients,
},
};
}
function normalizeTaskMode(value) {
if (typeof value === 'string' && value.trim()) return value.trim();
return DEFAULT_TASK_MODE;
}
function buildTaskRecord(task = {}) {
return {
id: task.id || generateTaskId(),
city: task.city || '',
plateName: task.plateName || '',
prompt: task.prompt || '',
enabled: task.enabled !== false,
mode: normalizeTaskMode(task.mode),
};
}
function buildResultRecord(result = {}) {
return {
id: result.id || generateResultId(),
taskId: result.taskId || null,
city: result.city || null,
scrapedAt: result.scrapedAt || new Date().toISOString(),
error: result.error || null,
data: result.data ?? null,
};
}
function parseTaskRow(row) {
return {
id: row.id,
city: row.city,
plateName: row.plate_name,
prompt: row.prompt,
enabled: row.enabled === 1,
mode: normalizeTaskMode(row.mode),
};
}
function parseResultRow(row) {
return {
id: row.id,
taskId: row.task_id,
city: row.city,
scrapedAt: row.scraped_at,
error: row.error,
data: row.data_json ? JSON.parse(row.data_json) : null,
};
}
function getDb() {
if (db) return db;
mkdirSync(dirname(DB_PATH), { recursive: true });
db = new Database(DB_PATH);
try {
db.pragma('journal_mode = DELETE');
} catch (error) {
console.warn(`[resultStore] Database journal mode unchanged: ${error.message}`);
}
return db;
}
function ensureSchema() {
getDb().exec(`
CREATE TABLE IF NOT EXISTS results (
id TEXT PRIMARY KEY,
task_id TEXT,
city TEXT,
scraped_at TEXT NOT NULL,
error TEXT,
data_json TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_results_scraped_at ON results (scraped_at DESC);
CREATE INDEX IF NOT EXISTS idx_results_city ON results (city);
CREATE INDEX IF NOT EXISTS idx_results_task_id ON results (task_id);
CREATE TABLE IF NOT EXISTS tasks (
id TEXT PRIMARY KEY,
city TEXT,
plate_name TEXT,
prompt TEXT,
enabled INTEGER NOT NULL DEFAULT 1,
mode TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
`);
}
function ensureTasksTableShape() {
const columns = getDb().prepare(`PRAGMA table_info(tasks)`).all();
const columnNames = columns.map((column) => column.name);
const hasLegacyBrowserColumn = columnNames.includes('use_browser');
const matchesExpectedShape =
columnNames.length === TASK_COLUMNS.length &&
TASK_COLUMNS.every((column, index) => columnNames[index] === column);
if (!hasLegacyBrowserColumn && matchesExpectedShape) return;
getDb().exec(`
BEGIN;
ALTER TABLE tasks RENAME TO tasks_legacy;
CREATE TABLE tasks (
id TEXT PRIMARY KEY,
city TEXT,
plate_name TEXT,
prompt TEXT,
enabled INTEGER NOT NULL DEFAULT 1,
mode TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
INSERT INTO tasks (id, city, plate_name, prompt, enabled, mode, created_at, updated_at)
SELECT
id,
city,
plate_name,
prompt,
COALESCE(enabled, 1),
COALESCE(mode, '${DEFAULT_TASK_MODE}'),
COALESCE(created_at, datetime('now')),
COALESCE(updated_at, datetime('now'))
FROM tasks_legacy;
DROP TABLE tasks_legacy;
COMMIT;
`);
}
function trimResults(limit = MAX_RESULT_RECORDS) {
getDb().prepare(`
DELETE FROM results
WHERE id NOT IN (
SELECT id
FROM results
ORDER BY scraped_at DESC, rowid DESC
LIMIT ?
)
`).run(limit);
}
function readJsonIfExists(filePath) {
if (!existsSync(filePath)) return null;
return JSON.parse(readFileSync(filePath, 'utf-8'));
}
function stripTasksFromConfig(config) {
if (!config || typeof config !== 'object') return getDefaultJsonConfig();
const { agent, scheduler, email } = config;
return normalizeJsonConfig({ agent, scheduler, email });
}
function ensureJsonConfigExists() {
if (existsSync(CONFIG_PATH)) return;
writeFileSync(CONFIG_PATH, JSON.stringify(getDefaultJsonConfig(), null, 2), 'utf-8');
}
function queryBaseRows({ city, taskId }) {
const clauses = [];
const params = [];
if (city) {
clauses.push('city = ?');
params.push(city);
}
if (taskId) {
clauses.push('task_id = ?');
params.push(taskId);
}
const whereSql = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : '';
const sql = `
SELECT id, task_id, city, scraped_at, error, data_json
FROM results
${whereSql}
ORDER BY scraped_at DESC, rowid DESC
`;
return getDb().prepare(sql).all(...params).map(parseResultRow);
}
function matchSection(record, section) {
if (!section) return true;
if (record.section === section || record.subsection === section) return true;
const items = record.data?.results || [];
return items.some((item) => item.section === section || item.subsection === section);
}
function matchType(record, type) {
if (!type) return true;
if (record.type === type) return true;
const items = record.data?.results || [];
return items.some((item) => item.type === type);
}
function normalizeProjectName(item) {
if (!item || typeof item !== 'object') return '';
const candidates = [
item.project_name,
item.projectName,
item.title,
item.name,
item.bidName,
];
for (const candidate of candidates) {
if (typeof candidate === 'string' && candidate.trim()) {
return candidate.replace(/\s+/g, ' ').trim();
}
}
return '';
}
function dedupeRowsByProjectName(rows) {
const seenProjectNames = new Set();
const dedupedRows = [];
for (const row of rows) {
const items = Array.isArray(row.data?.results) ? row.data.results : [];
if (items.length === 0) continue;
const uniqueItems = [];
for (const item of items) {
const projectName = normalizeProjectName(item);
if (!projectName) {
uniqueItems.push(item);
continue;
}
if (seenProjectNames.has(projectName)) continue;
seenProjectNames.add(projectName);
uniqueItems.push(item);
}
if (uniqueItems.length === 0) continue;
dedupedRows.push({
...row,
data: {
...(row.data || {}),
results: uniqueItems,
total: uniqueItems.length,
},
});
}
return dedupedRows;
}
function pickProjectLink(item) {
const candidates = [item?.detail_link, item?.target_link, item?.url, item?.href];
for (const candidate of candidates) {
if (typeof candidate === 'string' && candidate.trim()) return candidate.trim();
}
return '';
}
function parseProjectSection(item) {
if (typeof item?.section === 'string' && item.section.trim()) return item.section.trim();
if (typeof item?.type === 'string' && item.type.trim()) {
return item.type.split(/[-/]/)[0].trim();
}
return '';
}
function parseProjectAmount(item) {
if (typeof item?.amount_yuan === 'number' && Number.isFinite(item.amount_yuan)) {
return item.amount_yuan;
}
if (typeof item?.amount === 'number' && Number.isFinite(item.amount)) {
return item.amount;
}
return null;
}
function projectSortValue(project) {
const dateValue = Date.parse(project.date || '');
const scrapedAtValue = Date.parse(project.scrapedAt || '');
return {
dateValue: Number.isFinite(dateValue) ? dateValue : 0,
scrapedAtValue: Number.isFinite(scrapedAtValue) ? scrapedAtValue : 0,
};
}
function compareProjectsDesc(a, b) {
const aValue = projectSortValue(a);
const bValue = projectSortValue(b);
if (bValue.dateValue !== aValue.dateValue) return bValue.dateValue - aValue.dateValue;
if (bValue.scrapedAtValue !== aValue.scrapedAtValue) return bValue.scrapedAtValue - aValue.scrapedAtValue;
return (a.projectName || '').localeCompare(b.projectName || '', 'zh-CN');
}
function normalizeSearchText(value) {
if (typeof value !== 'string') return '';
return value.trim().toLowerCase();
}
function includesSearchText(source, keyword) {
if (!keyword) return true;
return normalizeSearchText(source).includes(keyword);
}
function parseNumberFilter(value) {
if (value === null || value === undefined || value === '') return null;
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : null;
}
function parseDateFilter(value) {
if (typeof value !== 'string' || !value.trim()) return null;
const timestamp = Date.parse(value);
return Number.isFinite(timestamp) ? timestamp : null;
}
function matchProjectFilters(project, filters = {}) {
const cityKeyword = normalizeSearchText(filters.city);
const sectionKeyword = normalizeSearchText(filters.section);
const projectNameKeyword = normalizeSearchText(filters.projectName);
const minAmount = parseNumberFilter(filters.minAmount);
const maxAmount = parseNumberFilter(filters.maxAmount);
const startDate = parseDateFilter(filters.startDate);
const endDate = parseDateFilter(filters.endDate);
const projectDate = parseDateFilter(project.date);
if (!includesSearchText(project.city, cityKeyword)) return false;
if (!includesSearchText(project.section || project.type, sectionKeyword)) return false;
if (!includesSearchText(project.projectName, projectNameKeyword)) return false;
if (minAmount !== null && (project.amountYuan === null || project.amountYuan < minAmount)) return false;
if (maxAmount !== null && (project.amountYuan === null || project.amountYuan > maxAmount)) return false;
if (startDate !== null && (projectDate === null || projectDate < startDate)) return false;
if (endDate !== null && (projectDate === null || projectDate > endDate)) return false;
return true;
}
function buildProjectList(rows, {
dedupeByName = false,
city,
section,
projectNameKeyword,
minAmount,
maxAmount,
startDate,
endDate,
} = {}) {
const seenProjectNames = new Set();
const projects = [];
for (const row of rows) {
const items = Array.isArray(row.data?.results) ? row.data.results : [];
for (let index = 0; index < items.length; index += 1) {
const item = items[index];
const projectName = normalizeProjectName(item);
if (!projectName) continue;
const projectSection = parseProjectSection(item);
if (city && row.city !== city) continue;
if (section && projectSection !== section) continue;
if (dedupeByName) {
if (seenProjectNames.has(projectName)) continue;
seenProjectNames.add(projectName);
}
projects.push({
id: `${row.id}:${index}`,
resultId: row.id,
taskId: row.taskId,
city: row.city || '',
section: projectSection,
type: typeof item?.type === 'string' ? item.type : '',
projectName,
amountYuan: parseProjectAmount(item),
date: typeof item?.date === 'string' ? item.date : '',
detailLink: pickProjectLink(item),
scrapedAt: row.scrapedAt,
raw: item,
});
}
}
return projects
.filter((project) => matchProjectFilters(project, {
city,
section,
projectName: projectNameKeyword,
minAmount,
maxAmount,
startDate,
endDate,
}))
.sort(compareProjectsDesc);
}
export function initResultsStore() {
if (initialized) return;
ensureSchema();
ensureTasksTableShape();
ensureJsonConfigExists();
initialized = true;
}
export function loadConfig() {
initResultsStore();
const jsonConfig = normalizeJsonConfig(readJsonIfExists(CONFIG_PATH) || getDefaultJsonConfig());
return {
...clone(jsonConfig),
tasks: listTasks(),
};
}
export function saveConfig(config) {
initResultsStore();
const jsonConfig = stripTasksFromConfig(config);
writeFileSync(CONFIG_PATH, JSON.stringify(jsonConfig, null, 2), 'utf-8');
return {
...clone(jsonConfig),
tasks: listTasks(),
};
}
export function listTasks() {
initResultsStore();
return getDb()
.prepare(`
SELECT id, city, plate_name, prompt, enabled, mode
FROM tasks
ORDER BY rowid DESC
`)
.all()
.map(parseTaskRow);
}
export function getTaskById(id) {
initResultsStore();
const row = getDb()
.prepare(`
SELECT id, city, plate_name, prompt, enabled, mode
FROM tasks
WHERE id = ?
`)
.get(id);
return row ? parseTaskRow(row) : null;
}
export function createTask(task) {
initResultsStore();
const record = buildTaskRecord(task);
getDb().prepare(`
INSERT INTO tasks (id, city, plate_name, prompt, enabled, mode, updated_at)
VALUES (?, ?, ?, ?, ?, ?, datetime('now'))
`).run(
record.id,
record.city,
record.plateName,
record.prompt,
record.enabled ? 1 : 0,
record.mode,
);
return record;
}
export function updateTask(id, patch) {
initResultsStore();
const current = getTaskById(id);
if (!current) return null;
const next = buildTaskRecord({ ...current, ...patch, id });
getDb().prepare(`
UPDATE tasks
SET city = ?,
plate_name = ?,
prompt = ?,
enabled = ?,
mode = ?,
updated_at = datetime('now')
WHERE id = ?
`).run(
next.city,
next.plateName,
next.prompt,
next.enabled ? 1 : 0,
next.mode,
id,
);
return next;
}
export function deleteTaskById(id) {
initResultsStore();
const result = getDb().prepare('DELETE FROM tasks WHERE id = ?').run(id);
return result.changes > 0;
}
export function appendResult(result) {
initResultsStore();
const record = buildResultRecord(result);
getDb().prepare(`
INSERT INTO results (id, task_id, city, scraped_at, error, data_json)
VALUES (?, ?, ?, ?, ?, ?)
`).run(
record.id,
record.taskId,
record.city,
record.scrapedAt,
record.error,
record.data === null ? null : JSON.stringify(record.data),
);
trimResults();
return record;
}
export function listResults({ city, section, type, taskId, page = 1, pageSize = 20, projectMode = false } = {}) {
initResultsStore();
let results = queryBaseRows({ city, taskId });
if (section) results = results.filter((record) => matchSection(record, section));
if (type) results = results.filter((record) => matchType(record, type));
if (projectMode) results = dedupeRowsByProjectName(results);
const normalizedPage = Math.max(1, parseInt(page, 10) || 1);
const normalizedPageSize = Math.max(1, parseInt(pageSize, 10) || 20);
const start = (normalizedPage - 1) * normalizedPageSize;
return {
total: results.length,
page: normalizedPage,
pageSize: normalizedPageSize,
data: results.slice(start, start + normalizedPageSize),
};
}
export function deleteResultById(id) {
initResultsStore();
const result = getDb().prepare('DELETE FROM results WHERE id = ?').run(id);
return result.changes > 0;
}
export function clearResults() {
initResultsStore();
getDb().prepare('DELETE FROM results').run();
}
export function getResultFilters({ projectMode = false } = {}) {
initResultsStore();
const rows = projectMode ? dedupeRowsByProjectName(queryBaseRows({})) : queryBaseRows({});
const cities = [...new Set(rows.map((row) => row.city).filter(Boolean))];
const sections = new Set();
const types = new Set();
for (const row of rows) {
if (row.section) sections.add(row.section);
if (row.subsection) sections.add(row.subsection);
if (row.type) types.add(row.type);
for (const item of row.data?.results || []) {
if (item.section) sections.add(item.section);
if (item.subsection) sections.add(item.subsection);
if (item.type) types.add(item.type);
}
}
return {
cities,
sections: [...sections],
types: [...types],
};
}
export function listProjects({
city,
section,
projectName,
minAmount,
maxAmount,
startDate,
endDate,
page = 1,
pageSize = 20,
dedupeByName = true,
} = {}) {
initResultsStore();
const projects = getProjects({
dedupeByName,
city,
section,
projectName,
minAmount,
maxAmount,
startDate,
endDate,
});
const normalizedPage = Math.max(1, parseInt(page, 10) || 1);
const normalizedPageSize = Math.max(1, parseInt(pageSize, 10) || 20);
const start = (normalizedPage - 1) * normalizedPageSize;
return {
total: projects.length,
page: normalizedPage,
pageSize: normalizedPageSize,
data: projects.slice(start, start + normalizedPageSize),
};
}
export function getProjects({
city,
section,
projectName,
minAmount,
maxAmount,
startDate,
endDate,
dedupeByName = true,
} = {}) {
initResultsStore();
return buildProjectList(queryBaseRows({}), {
dedupeByName,
city,
section,
projectNameKeyword: projectName,
minAmount,
maxAmount,
startDate,
endDate,
});
}
export function getProjectFilters({ dedupeByName = true } = {}) {
initResultsStore();
const projects = getProjects({ dedupeByName });
const cities = [...new Set(projects.map((project) => project.city).filter(Boolean))];
const sections = [...new Set(projects.map((project) => project.section).filter(Boolean))];
return {
cities,
sections,
};
}
export function getResultsDbPath() {
return DB_PATH;
}

View File

@@ -1,210 +1,177 @@
import 'dotenv/config'; import 'dotenv/config';
import cron from 'node-cron'; import cron from 'node-cron';
import { readFileSync, writeFileSync, existsSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
import Firecrawl from '@mendable/firecrawl-js';
import { sendScraperResultsEmail } from './emailService.js'; import { sendScraperResultsEmail } from './emailService.js';
import { runScraperWithBrowser } from './firecrawlBrowserScraper.js'; import { runAgentTask } from './agentService.js';
import { initResultsStore, loadConfig, appendResult } from './resultStore.js';
const __filename = fileURLToPath(import.meta.url); const DEFAULT_TASK_MODE = 'qwen3.5-plus';
const __dirname = dirname(__filename);
// 初始化 Firecrawl 客户端 let currentScheduledTask = null;
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const RESULTS_PATH = join(__dirname, '..', 'results.json'); function normalizeTaskMode(value) {
if (typeof value === 'string' && value.trim()) return value.trim();
// 加载配置文件 return DEFAULT_TASK_MODE;
function loadConfig() {
try {
const configPath = join(__dirname, '..', 'config.json');
return JSON.parse(readFileSync(configPath, 'utf-8'));
} catch (error) {
console.error('加载配置文件失败:', error.message);
return null;
}
} }
// ========== 结果存取(与 server.js 保持一致) ========== async function runTask(task, agentCfg) {
const mode = normalizeTaskMode(task.mode);
function readResults() { console.log(`[Scheduler][Agent] ${task.city}: start`);
if (!existsSync(RESULTS_PATH)) return []; console.log(`[Scheduler][Agent] ${task.city}: mode=${mode}`);
try {
return JSON.parse(readFileSync(RESULTS_PATH, 'utf-8'));
} catch (e) {
return [];
}
}
function saveResults(results) { const { results } = await runAgentTask(task.prompt, {
writeFileSync(RESULTS_PATH, JSON.stringify(results, null, 2), 'utf-8'); baseUrl: agentCfg.baseUrl,
} mode,
pollInterval: agentCfg.pollInterval,
timeout: agentCfg.timeout,
logPrefix: `[Scheduler][Agent][${task.city}]`,
});
function appendResult(result) { console.log(`[Scheduler][Agent] ${task.city}: ${results.length} results`);
const results = readResults();
results.unshift({ ...result, id: `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}` });
if (results.length > 500) results.splice(500);
saveResults(results);
}
// ========== 抓取执行(复用 server.js 中 runScraper 的逻辑) ========== return appendResult({
taskId: task.id,
async function runScraper(scraper) { city: task.city,
console.log(`[定时任务][Browser] ${scraper.city} - ${scraper.section} ${scraper.subsection} - ${scraper.type}${scraper.url}`);
const { items } = await runScraperWithBrowser(firecrawl, scraper, { logPrefix: '[Browser][Scheduler]' });
console.log(`[定时任务][Browser] 提取到 ${items.length} 条公告`);
const record = {
scraperId: scraper.id,
city: scraper.city,
section: scraper.section,
subsection: scraper.subsection,
type: scraper.type,
url: scraper.url,
scrapedAt: new Date().toISOString(), scrapedAt: new Date().toISOString(),
data: { result: items, total: items.length }, data: { results, total: results.length },
}; });
appendResult(record);
return record;
} }
// ========== 定时任务执行函数 ==========
async function executeScheduledTask(config) { async function executeScheduledTask(config) {
try { try {
console.log('========================================'); console.log('========================================');
console.log('定时任务开始执行'); console.log('Scheduler started');
console.log('执行时间:', new Date().toLocaleString('zh-CN')); console.log('Time:', new Date().toLocaleString('zh-CN'));
console.log('========================================'); console.log('========================================');
// 获取所有已启用的抓取来源 const tasks = (config.tasks || []).filter((task) => task.enabled);
const scrapers = (config.scrapers || []).filter(s => s.enabled); const agentCfg = config.agent || {};
if (scrapers.length === 0) { if (tasks.length === 0) {
console.log('没有已启用的抓取来源,跳过'); console.log('No enabled tasks, skip');
return; return;
} }
console.log(`${scrapers.length} 个已启用的抓取来源`); console.log(`Enabled tasks: ${tasks.length}`);
// 逐个运行抓取任务
const results = []; const results = [];
for (const scraper of scrapers) { for (const task of tasks) {
try { try {
console.log(`\n---------- 抓取: ${scraper.city} - ${scraper.section} ${scraper.type} ----------`); console.log(`\n---------- Task: ${task.city} ----------`);
const r = await runScraper(scraper); const record = await runTask(task, agentCfg);
results.push(r); results.push(record);
console.log(`✓ 抓取成功`); console.log('Task completed');
} catch (err) { } catch (error) {
console.error(`✗ 抓取失败: ${err.message}`); console.error(`Task failed: ${error.message}`);
const errRecord = { const errorRecord = appendResult({
scraperId: scraper.id, taskId: task.id,
city: scraper.city, city: task.city,
section: scraper.section,
subsection: scraper.subsection,
type: scraper.type,
url: scraper.url,
scrapedAt: new Date().toISOString(), scrapedAt: new Date().toISOString(),
error: err.message, error: error.message,
data: null, data: null,
}; });
appendResult(errRecord); results.push(errorRecord);
results.push(errRecord);
} }
} }
const successCount = results.filter(r => !r.error).length; const successCount = results.filter((item) => !item.error).length;
const failCount = results.filter(r => r.error).length; const failCount = results.filter((item) => item.error).length;
console.log(`\n========== 抓取完成 ==========`); console.log('\n========== Scheduler finished ==========');
console.log(`成功: ${successCount} 条,失败: ${failCount}`); console.log(`Success: ${successCount}, Failed: ${failCount}`);
// 检查是否需要发送邮件
if (successCount === 0) { if (successCount === 0) {
console.log('没有成功的抓取结果,不发送邮件'); console.log('No successful results, skip email');
return; return;
} }
// 发送邮件报告
if (config.email?.smtpHost && config.email?.smtpUser) { if (config.email?.smtpHost && config.email?.smtpUser) {
console.log('\n正在发送抓取结果邮件...'); console.log('\nSending email...');
try { try {
const emailResult = await sendScraperResultsEmail(config.email, results); const emailResult = await sendScraperResultsEmail(config.email, results);
console.log('邮件发送成功! MessageId:', emailResult.messageId); console.log('Email sent:', emailResult.messageId);
} catch (emailErr) { } catch (error) {
console.error('邮件发送失败:', emailErr.message); console.error('Email failed:', error.message);
} }
} else { } else {
console.log('邮件配置不完整,跳过邮件发送'); console.log('Email config incomplete, skip email');
} }
console.log('========================================'); console.log('========================================');
} catch (error) { } catch (error) {
console.error('========================================'); console.error('========================================');
console.error('定时任务执行失败:', error.message); console.error('Scheduler failed:', error.message);
console.error(error.stack); console.error(error.stack);
console.error('========================================'); console.error('========================================');
} }
} }
// 存储当前的定时任务
let currentScheduledTask = null;
export function initScheduler() { export function initScheduler() {
initResultsStore();
const config = loadConfig(); const config = loadConfig();
if (!config) { console.error('无法启动定时任务: 配置文件加载失败'); return; } if (!config.scheduler?.enabled) {
if (!config.scheduler?.enabled) { console.log('定时任务已禁用'); return; } console.log('Scheduler disabled');
return;
}
const cronTime = config.scheduler.cronTime || '0 9 * * *'; const cronTime = config.scheduler.cronTime || '0 9 * * *';
const enabledCount = (config.scrapers || []).filter(s => s.enabled).length; const enabledCount = (config.tasks || []).filter((task) => task.enabled).length;
console.log('========================================'); console.log('========================================');
console.log('定时任务已启动,执行计划:', cronTime); console.log('Scheduler enabled:', cronTime);
console.log(`已启用的抓取来源: ${enabledCount}`); console.log(`Enabled tasks: ${enabledCount}`);
if (config.email?.recipients) console.log('收件人:', config.email.recipients); if (config.email?.recipients) {
console.log('Recipients:', config.email.recipients);
}
console.log('========================================'); console.log('========================================');
if (currentScheduledTask) { currentScheduledTask.stop(); } if (currentScheduledTask) {
currentScheduledTask.stop();
}
currentScheduledTask = cron.schedule(cronTime, () => { currentScheduledTask = cron.schedule(
// 每次执行时重新加载配置,确保使用最新的 scrapers cronTime,
const latestConfig = loadConfig(); () => {
if (latestConfig) { executeScheduledTask(loadConfig());
executeScheduledTask(latestConfig); },
} { timezone: 'Asia/Shanghai' },
}, { timezone: 'Asia/Shanghai' }); );
} }
export function reloadScheduler() { export function reloadScheduler() {
console.log('重新加载定时任务配置...'); console.log('Reloading scheduler...');
if (currentScheduledTask) { currentScheduledTask.stop(); currentScheduledTask = null; } if (currentScheduledTask) {
currentScheduledTask.stop();
currentScheduledTask = null;
}
initScheduler(); initScheduler();
} }
export function stopScheduler() { export function stopScheduler() {
if (currentScheduledTask) { if (!currentScheduledTask) return false;
currentScheduledTask.stop(); currentScheduledTask = null;
console.log('定时任务已停止'); return true; currentScheduledTask.stop();
} currentScheduledTask = null;
return false; console.log('Scheduler stopped');
return true;
} }
export function getSchedulerStatus() { export function getSchedulerStatus() {
const config = loadConfig(); const config = loadConfig();
const enabledScrapers = (config?.scrapers || []).filter(s => s.enabled).length; const enabledTasks = (config.tasks || []).filter((task) => task.enabled).length;
return { return {
isRunning: currentScheduledTask !== null, isRunning: currentScheduledTask !== null,
enabledScrapers, enabledTasks,
config: config ? { config: {
enabled: config.scheduler?.enabled || false, enabled: config.scheduler?.enabled || false,
cronTime: config.scheduler?.cronTime || '0 9 * * *', cronTime: config.scheduler?.cronTime || '0 9 * * *',
description: config.scheduler?.description || '', description: config.scheduler?.description || '',
} : null, },
}; };
} }
export async function runTaskNow() { export async function runTaskNow() {
const config = loadConfig(); initResultsStore();
if (!config) throw new Error('配置文件加载失败'); await executeScheduledTask(loadConfig());
await executeScheduledTask(config);
} }

View File

@@ -1,321 +1,494 @@
import 'dotenv/config'; import 'dotenv/config';
import express from 'express'; import express from 'express';
import cors from 'cors'; import cors from 'cors';
import Firecrawl from '@mendable/firecrawl-js'; import {
import { readFileSync, writeFileSync, existsSync } from 'fs'; initResultsStore,
import { fileURLToPath } from 'url'; loadConfig,
import { dirname, join } from 'path'; saveConfig,
import { sendCombinedReportEmail } from './emailService.js'; listTasks,
getTaskById,
createTask,
updateTask,
deleteTaskById,
listResults,
listProjects,
getProjects,
deleteResultById,
clearResults,
getResultFilters,
getProjectFilters,
appendResult,
} from './resultStore.js';
import { initScheduler, runTaskNow, reloadScheduler, getSchedulerStatus } from './scheduler.js'; import { initScheduler, runTaskNow, reloadScheduler, getSchedulerStatus } from './scheduler.js';
import { runScraperWithBrowser } from './firecrawlBrowserScraper.js'; import { runAgentTask } from './agentService.js';
const app = express(); const app = express();
const PORT = process.env.PORT || 5000; const PORT = process.env.PORT || 5000;
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
const MASKED_PASSWORD = '***已配置***';
app.use(cors()); app.use(cors());
app.use(express.json()); app.use(express.json({ limit: '50mb' }));
app.use(express.urlencoded({ limit: '50mb', extended: true }));
app.use(express.static('public')); app.use(express.static('public'));
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY }); function normalizeTaskMode(value) {
if (typeof value === 'string' && value.trim()) return value.trim();
const __filename = fileURLToPath(import.meta.url); return DEFAULT_TASK_MODE;
const __dirname = dirname(__filename);
const CONFIG_PATH = join(__dirname, '..', 'config.json');
const RESULTS_PATH = join(__dirname, '..', 'results.json');
function readConfig() {
return JSON.parse(readFileSync(CONFIG_PATH, 'utf-8'));
} }
function saveConfig(cfg) { function buildTaskPayload(body = {}, { partial = false } = {}) {
writeFileSync(CONFIG_PATH, JSON.stringify(cfg, null, 2), 'utf-8'); const payload = {};
}
// ========== 抓取结果存取 ========== if (!partial || Object.prototype.hasOwnProperty.call(body, 'city')) {
payload.city = body.city || '';
function readResults() {
if (!existsSync(RESULTS_PATH)) return [];
try {
return JSON.parse(readFileSync(RESULTS_PATH, 'utf-8'));
} catch (e) {
return [];
} }
if (!partial || Object.prototype.hasOwnProperty.call(body, 'plateName')) {
payload.plateName = body.plateName || '';
}
if (!partial || Object.prototype.hasOwnProperty.call(body, 'prompt')) {
payload.prompt = body.prompt || '';
}
if (!partial || Object.prototype.hasOwnProperty.call(body, 'enabled')) {
payload.enabled = body.enabled !== false;
}
if (!partial || Object.prototype.hasOwnProperty.call(body, 'mode')) {
payload.mode = normalizeTaskMode(body.mode);
}
return payload;
} }
function saveResults(results) { function maskConfigSecrets(config) {
writeFileSync(RESULTS_PATH, JSON.stringify(results, null, 2), 'utf-8'); const next = { ...config };
if (config.email) {
next.email = {
...config.email,
smtpPass: config.email.smtpPass ? MASKED_PASSWORD : '',
};
}
return next;
} }
function appendResult(result) { function mergeConfigWithExistingSecrets(incoming = {}) {
const results = readResults(); const current = loadConfig();
results.unshift({ ...result, id: `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}` }); const next = {
// 最多保留 500 条 ...current,
if (results.length > 500) results.splice(500); ...incoming,
saveResults(results); agent: { ...(current.agent || {}), ...(incoming.agent || {}) },
scheduler: { ...(current.scheduler || {}), ...(incoming.scheduler || {}) },
email: { ...(current.email || {}), ...(incoming.email || {}) },
};
if (next.email?.smtpPass === MASKED_PASSWORD) {
next.email.smtpPass = current.email?.smtpPass || '';
}
return next;
}
function getProjectQueryFilters(query = {}) {
return {
city: query.city,
section: query.section,
projectName: query.projectName,
minAmount: query.minAmount,
maxAmount: query.maxAmount,
startDate: query.startDate,
endDate: query.endDate,
};
}
function formatExportTimestamp(date = new Date()) {
const pad = (value) => String(value).padStart(2, '0');
return [
date.getFullYear(),
pad(date.getMonth() + 1),
pad(date.getDate()),
'-',
pad(date.getHours()),
pad(date.getMinutes()),
pad(date.getSeconds()),
].join('');
}
function escapeCsvCell(value) {
if (value === null || value === undefined) return '';
const stringValue = String(value);
if (/[",\r\n]/.test(stringValue)) {
return `"${stringValue.replace(/"/g, '""')}"`;
}
return stringValue;
}
function buildProjectsCsv(projects = []) {
const columns = [
['城市', (project) => project.city || ''],
['板块', (project) => project.section || project.type || ''],
['项目名称', (project) => project.projectName || ''],
['金额(元)', (project) => Number.isFinite(project.amountYuan) ? project.amountYuan : ''],
['发布日期', (project) => project.date || ''],
['详情链接', (project) => project.detailLink || ''],
];
const lines = [
columns.map(([header]) => escapeCsvCell(header)).join(','),
...projects.map((project) => columns.map(([, getter]) => escapeCsvCell(getter(project))).join(',')),
];
return `\uFEFF${lines.join('\r\n')}`;
}
let isRunning = false;
let runningStatus = null;
async function runTask(task) {
const config = loadConfig();
const agentCfg = config.agent || {};
const mode = normalizeTaskMode(task.mode);
console.log(`[Agent] ${task.city}: start`);
console.log(`[Agent] ${task.city}: mode=${mode}`);
const { results } = await runAgentTask(task.prompt, {
baseUrl: agentCfg.baseUrl,
mode,
pollInterval: agentCfg.pollInterval,
timeout: agentCfg.timeout,
logPrefix: `[Agent][${task.city}]`,
});
return appendResult({
taskId: task.id,
city: task.city,
scrapedAt: new Date().toISOString(),
data: { results, total: results.length },
});
}
function runTaskInBackground(task) {
runningStatus = {
taskId: task.id,
city: task.city,
startTime: Date.now(),
current: 0,
total: 1,
finished: false,
error: null,
};
runTask(task)
.then((record) => {
runningStatus = { ...runningStatus, finished: true, result: record, current: 1 };
})
.catch((error) => {
console.error('任务执行失败:', error.message);
runningStatus = { ...runningStatus, finished: true, error: error.message, current: 1 };
})
.finally(() => {
isRunning = false;
});
}
function runTasksInBackground(tasks) {
runningStatus = {
taskId: null,
city: null,
startTime: Date.now(),
current: 0,
total: tasks.length,
finished: false,
error: null,
results: [],
};
(async () => {
for (const task of tasks) {
runningStatus = { ...runningStatus, taskId: task.id, city: task.city };
try {
const record = await runTask(task);
runningStatus.results.push(record);
} catch (error) {
const errorRecord = appendResult({
taskId: task.id,
city: task.city,
scrapedAt: new Date().toISOString(),
error: error.message,
data: null,
});
runningStatus.results.push(errorRecord);
}
runningStatus.current += 1;
}
runningStatus.finished = true;
})()
.catch((error) => {
runningStatus = { ...runningStatus, finished: true, error: error.message };
})
.finally(() => {
isRunning = false;
});
} }
// 查询结果(支持分页与筛选)
app.get('/api/results', (req, res) => { app.get('/api/results', (req, res) => {
try { try {
const { city, type, section, page = 1, pageSize = 20, scraperId } = req.query; const { city, section, type, page = 1, pageSize = 20, taskId, view } = req.query;
let results = readResults(); const projectMode = view === 'projects';
if (city) results = results.filter(r => r.city === city); const result = listResults({ city, section, type, page, pageSize, taskId, projectMode });
if (type) results = results.filter(r => r.type === type); res.json({ success: true, ...result });
if (section) results = results.filter(r => r.section === section); } catch (error) {
if (scraperId) results = results.filter(r => r.scraperId === scraperId); res.status(500).json({ success: false, error: error.message });
const total = results.length;
const start = (parseInt(page) - 1) * parseInt(pageSize);
const data = results.slice(start, start + parseInt(pageSize));
res.json({ success: true, total, page: parseInt(page), pageSize: parseInt(pageSize), data });
} catch (e) {
res.status(500).json({ success: false, error: e.message });
} }
}); });
// 删除单条结果
app.delete('/api/results/:id', (req, res) => { app.delete('/api/results/:id', (req, res) => {
try { try {
const results = readResults(); const deleted = deleteResultById(req.params.id);
const before = results.length; if (!deleted) {
const updated = results.filter(r => r.id !== req.params.id); return res.status(404).json({ success: false, error: '未找到结果' });
if (updated.length === before) return res.status(404).json({ success: false, error: '未找到' }); }
saveResults(updated);
res.json({ success: true }); res.json({ success: true });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
// 清空所有结果 app.delete('/api/results', (_req, res) => {
app.delete('/api/results', (req, res) => {
try { try {
saveResults([]); clearResults();
res.json({ success: true, message: '已清空所有结果' }); res.json({ success: true, message: '已清空所有结果' });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
// 获取结果的筛选选项(城市/板块/类型下拉枚举) app.get('/api/results/filters', (_req, res) => {
app.get('/api/results/filters', (req, res) => {
try { try {
const results = readResults(); const projectMode = _req.query.view === 'projects';
const cities = [...new Set(results.map(r => r.city).filter(Boolean))]; res.json({ success: true, data: getResultFilters({ projectMode }) });
const sections = [...new Set(results.map(r => r.section).filter(Boolean))]; } catch (error) {
const types = [...new Set(results.map(r => r.type).filter(Boolean))]; res.status(500).json({ success: false, error: error.message });
res.json({ success: true, data: { cities, sections, types } });
} catch (e) {
res.status(500).json({ success: false, error: e.message });
} }
}); });
// ========== 抓取来源 CRUD ========== app.get('/api/projects', (req, res) => {
app.get('/api/scrapers', (req, res) => {
try { try {
const cfg = readConfig(); const { page = 1, pageSize = 20 } = req.query;
res.json({ success: true, data: cfg.scrapers || [] }); const filters = getProjectQueryFilters(req.query);
} catch (e) { const result = listProjects({
res.status(500).json({ success: false, error: e.message }); ...filters,
page,
pageSize,
dedupeByName: true,
});
res.json({ success: true, ...result });
} catch (error) {
res.status(500).json({ success: false, error: error.message });
} }
}); });
app.post('/api/scrapers', (req, res) => { app.get('/api/projects/export', (req, res) => {
try { try {
const cfg = readConfig(); const projects = getProjects({
if (!cfg.scrapers) cfg.scrapers = []; ...getProjectQueryFilters(req.query),
const item = { dedupeByName: true,
id: `scraper-${Date.now()}`, });
city: req.body.city || '', const csv = buildProjectsCsv(projects);
url: req.body.url || '', const filename = `projects-${formatExportTimestamp()}.csv`;
section: req.body.section || '',
subsection: req.body.subsection || '', res.setHeader('Content-Type', 'text/csv; charset=utf-8');
type: req.body.type || '招标公告', res.setHeader('Content-Disposition', `attachment; filename="${filename}"; filename*=UTF-8''${encodeURIComponent(filename)}`);
prompt: req.body.prompt || '', res.send(csv);
enabled: req.body.enabled !== false, } catch (error) {
model: req.body.model || 'spark-1-mini', res.status(500).json({ success: false, error: error.message });
};
cfg.scrapers.push(item);
saveConfig(cfg);
res.json({ success: true, data: item });
} catch (e) {
res.status(500).json({ success: false, error: e.message });
} }
}); });
app.put('/api/scrapers/:id', (req, res) => { app.get('/api/projects/filters', (_req, res) => {
try { try {
const cfg = readConfig(); res.json({ success: true, data: getProjectFilters({ dedupeByName: true }) });
const idx = (cfg.scrapers || []).findIndex(s => s.id === req.params.id); } catch (error) {
if (idx === -1) return res.status(404).json({ success: false, error: '未找到该配置' }); res.status(500).json({ success: false, error: error.message });
cfg.scrapers[idx] = { ...cfg.scrapers[idx], ...req.body, id: req.params.id };
saveConfig(cfg);
res.json({ success: true, data: cfg.scrapers[idx] });
} catch (e) {
res.status(500).json({ success: false, error: e.message });
} }
}); });
app.delete('/api/scrapers/:id', (req, res) => { app.get('/api/tasks', (_req, res) => {
try { try {
const cfg = readConfig(); res.json({ success: true, data: listTasks() });
const before = (cfg.scrapers || []).length; } catch (error) {
cfg.scrapers = (cfg.scrapers || []).filter(s => s.id !== req.params.id); res.status(500).json({ success: false, error: error.message });
if (cfg.scrapers.length === before) return res.status(404).json({ success: false, error: '未找到' }); }
saveConfig(cfg); });
app.post('/api/tasks', (req, res) => {
try {
const task = createTask(buildTaskPayload(req.body));
res.json({ success: true, data: task });
} catch (error) {
res.status(500).json({ success: false, error: error.message });
}
});
app.put('/api/tasks/:id', (req, res) => {
try {
const task = updateTask(req.params.id, buildTaskPayload(req.body, { partial: true }));
if (!task) {
return res.status(404).json({ success: false, error: '未找到该任务' });
}
res.json({ success: true, data: task });
} catch (error) {
res.status(500).json({ success: false, error: error.message });
}
});
app.delete('/api/tasks/:id', (req, res) => {
try {
const deleted = deleteTaskById(req.params.id);
if (!deleted) {
return res.status(404).json({ success: false, error: '未找到该任务' });
}
res.json({ success: true }); res.json({ success: true });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
// ========== 统一抓取执行 ========== app.get('/api/tasks/status', (_req, res) => {
if (!runningStatus) {
return res.json({ success: true, data: { isRunning: false } });
}
// 执行单个抓取来源并保存结果 const elapsed = Math.round((Date.now() - runningStatus.startTime) / 1000);
async function runScraper(scraper) { res.json({
console.log(`[Browser] ${scraper.city} - ${scraper.section} ${scraper.subsection} - ${scraper.type}${scraper.url}`); success: true,
const { items } = await runScraperWithBrowser(firecrawl, scraper, { logPrefix: '[Browser][API]' }); data: {
console.log(`[Browser] 提取到 ${items.length} 条公告`); isRunning,
elapsed,
city: runningStatus.city,
current: runningStatus.current,
total: runningStatus.total,
finished: runningStatus.finished,
error: runningStatus.error,
results: runningStatus.finished
? (runningStatus.results || (runningStatus.result ? [runningStatus.result] : []))
: undefined,
},
});
});
const record = { app.post('/api/tasks/:id/run', (req, res) => {
scraperId: scraper.id, if (isRunning) {
city: scraper.city, return res.status(409).json({ success: false, error: '当前已有任务在运行,请稍后再试' });
section: scraper.section, }
subsection: scraper.subsection,
type: scraper.type,
url: scraper.url,
scrapedAt: new Date().toISOString(),
data: { result: items, total: items.length }, // 统一为 result 字段
};
appendResult(record);
return record;
}
// 运行指定 ID 的抓取来源(单条测试)
app.post('/api/scrapers/:id/run', async (req, res) => {
try { try {
const cfg = readConfig(); const task = getTaskById(req.params.id);
const scraper = (cfg.scrapers || []).find(s => s.id === req.params.id); if (!task) {
if (!scraper) return res.status(404).json({ success: false, error: '未找到该配置' }); return res.status(404).json({ success: false, error: '未找到该任务' });
const result = await runScraper(scraper); }
res.json({ success: true, data: result });
} catch (e) { isRunning = true;
console.error('测试抓取失败:', e.message); runTaskInBackground(task);
res.status(500).json({ success: false, error: e.message }); res.json({ success: true, message: `任务“${task.city}”已开始执行` });
} catch (error) {
isRunning = false;
res.status(500).json({ success: false, error: error.message });
} }
}); });
// 批量运行多个抓取来源 app.post('/api/tasks/run', (req, res) => {
// body: { ids: ['id1','id2',...] } 不传则运行所有已启用的 if (isRunning) {
app.post('/api/scrape/run', async (req, res) => { return res.status(409).json({ success: false, error: '当前已有任务在运行,请稍后再试' });
try { }
const cfg = readConfig();
let scrapers = cfg.scrapers || [];
if (req.body.ids && req.body.ids.length > 0) { try {
scrapers = scrapers.filter(s => req.body.ids.includes(s.id)); let tasks = listTasks();
if (Array.isArray(req.body?.ids) && req.body.ids.length > 0) {
const idSet = new Set(req.body.ids);
tasks = tasks.filter((task) => idSet.has(task.id));
} else { } else {
scrapers = scrapers.filter(s => s.enabled); tasks = tasks.filter((task) => task.enabled);
} }
if (scrapers.length === 0) { if (tasks.length === 0) {
return res.json({ success: true, data: [], message: '没有可运行的抓取来源' }); return res.json({ success: true, data: [], message: '没有可运行的任务' });
} }
const results = []; isRunning = true;
for (const scraper of scrapers) { runTasksInBackground(tasks);
try { res.json({ success: true, message: `${tasks.length} 个任务已开始执行` });
const r = await runScraper(scraper); } catch (error) {
results.push(r); isRunning = false;
} catch (err) { res.status(500).json({ success: false, error: error.message });
const errRecord = {
scraperId: scraper.id,
city: scraper.city,
section: scraper.section,
subsection: scraper.subsection,
type: scraper.type,
url: scraper.url,
scrapedAt: new Date().toISOString(),
error: err.message,
data: null,
};
appendResult(errRecord);
results.push(errRecord);
}
}
res.json({ success: true, data: results });
} catch (e) {
res.status(500).json({ success: false, error: e.message });
} }
}); });
// ========== 配置管理 ========== app.get('/api/config', (_req, res) => {
app.get('/api/config', (req, res) => {
try { try {
const cfg = readConfig(); res.json({ success: true, data: maskConfigSecrets(loadConfig()) });
if (cfg.email?.smtpPass) cfg.email.smtpPass = '***已配置***'; } catch (error) {
res.json({ success: true, data: cfg }); res.status(500).json({ success: false, error: error.message });
} catch (e) {
res.status(500).json({ success: false, error: e.message });
} }
}); });
app.post('/api/config', (req, res) => { app.post('/api/config', (req, res) => {
try { try {
const newCfg = req.body; saveConfig(mergeConfigWithExistingSecrets(req.body));
const oldCfg = readConfig();
if (newCfg.email?.smtpPass === '***已配置***') {
newCfg.email.smtpPass = oldCfg.email?.smtpPass || '';
}
saveConfig(newCfg);
reloadScheduler(); reloadScheduler();
res.json({ success: true, message: '配置已保存' }); res.json({ success: true, message: '配置已保存' });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
// ========== 邮件 ==========
app.post('/api/send-email', async (req, res) => { app.post('/api/send-email', async (req, res) => {
try { try {
const { emailConfig, report } = req.body; const { emailConfig, report } = req.body;
if (!emailConfig?.smtpHost || !emailConfig?.smtpUser || !emailConfig?.smtpPass) if (!emailConfig?.smtpHost || !emailConfig?.smtpUser || !emailConfig?.smtpPass) {
return res.status(400).json({ success: false, error: '邮件配置不完整' }); return res.status(400).json({ success: false, error: '邮件配置不完整' });
if (!emailConfig.recipients?.trim()) }
if (!emailConfig.recipients?.trim()) {
return res.status(400).json({ success: false, error: '请指定收件人' }); return res.status(400).json({ success: false, error: '请指定收件人' });
if (!report) }
if (!report) {
return res.status(400).json({ success: false, error: '没有报告数据' }); return res.status(400).json({ success: false, error: '没有报告数据' });
}
const { sendReportEmail } = await import('./emailService.js'); const { sendReportEmail } = await import('./emailService.js');
const result = await sendReportEmail(emailConfig, report); const result = await sendReportEmail(emailConfig, report);
res.json({ success: true, message: '邮件发送成功', messageId: result.messageId }); res.json({ success: true, message: '邮件发送成功', messageId: result.messageId });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
// ========== 定时任务 ========== app.get('/api/scheduler/status', (_req, res) => {
app.get('/api/scheduler/status', (req, res) => {
try { try {
res.json({ success: true, data: getSchedulerStatus() }); res.json({ success: true, data: getSchedulerStatus() });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
app.post('/api/run-scheduled-task', (req, res) => { app.post('/api/run-scheduled-task', (_req, res) => {
try { try {
runTaskNow().catch(err => console.error('定时任务执行失败:', err)); runTaskNow().catch((error) => console.error('定时任务执行失败:', error));
res.json({ success: true, message: '定时任务已在后台触发' }); res.json({ success: true, message: '定时任务已在后台触发' });
} catch (e) { } catch (error) {
res.status(500).json({ success: false, error: e.message }); res.status(500).json({ success: false, error: error.message });
} }
}); });
initResultsStore();
app.listen(PORT, () => { app.listen(PORT, () => {
console.log(`Server running at http://localhost:${PORT}`); console.log(`Server running at http://localhost:${PORT}`);
initScheduler(); initScheduler();