Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2614af7808 | |||
| 7bfba04199 | |||
| d78dc655ee | |||
| bd46d8f907 | |||
| 0648770a6a | |||
| 40118ec508 | |||
| b9270428db | |||
| 4f504447a1 | |||
| a2408fa952 |
2
.env
2
.env
@@ -2,4 +2,4 @@
|
|||||||
PORT=5000
|
PORT=5000
|
||||||
|
|
||||||
# Firecrawl API Key（在 https://www.firecrawl.dev/app/api-keys 获取）
|
# Firecrawl API Key（在 https://www.firecrawl.dev/app/api-keys 获取）
|
||||||
FIRECRAWL_API_KEY=fc-354d1bbd965d482c977796ff534e15ca
|
FIRECRAWL_API_KEY=fc-595dd922780442f8a907202666a522ef
|
||||||
5
.gitignore
vendored
5
.gitignore
vendored
@@ -14,6 +14,11 @@ pnpm-debug.log*
|
|||||||
|
|
||||||
# 配置文件（包含敏感信息）
|
# 配置文件（包含敏感信息）
|
||||||
config.json
|
config.json
|
||||||
|
data/
|
||||||
|
*.sqlite
|
||||||
|
*.sqlite-shm
|
||||||
|
*.sqlite-wal
|
||||||
|
*.migrated.bak
|
||||||
|
|
||||||
# 编辑器目录和文件
|
# 编辑器目录和文件
|
||||||
.vscode/
|
.vscode/
|
||||||
|
|||||||
619
README.md
619
README.md
@@ -1,536 +1,191 @@
|
|||||||
# 南京公共工程建设中心 - 公告采集工具
|
# 公告抓取与分析工具
|
||||||
|
|
||||||
一个用于采集南京公共工程建设中心公告信息的 Web 可视化工具，支持定时任务和邮件推送。
|
一个用于管理公告抓取任务、查看抓取结果、查询项目数据并维护系统配置的工具。
|
||||||
|
|
||||||
## 功能特性
|
当前项目结构已经升级为：
|
||||||
|
|
||||||
- ✅ 采集公告列表(支持分页)
|
- 前端：Vue 3 + JavaScript + Vite + Vue Router + Axios + Element Plus
|
||||||
- ✅ 按时间范围智能采集
|
- 后端：Node.js + Express
|
||||||
- ✅ 采集公告详情内容
|
- 数据存储：better-sqlite3
|
||||||
- ✅ 智能提取预算金额
|
- 调度：node-cron
|
||||||
- ✅ 生成统计报告
|
- 邮件：nodemailer
|
||||||
- ✅ Web 可视化界面
|
|
||||||
- ✅ 导出 Word/Markdown 报告
|
|
||||||
- ✅ RESTful API 支持
|
|
||||||
- ✅ **定时任务自动采集**
|
|
||||||
- ✅ **邮件推送 HTML 报告**
|
|
||||||
- ✅ **Web 配置界面**
|
|
||||||
- ✅ **无需数据库，轻量部署**
|
|
||||||
|
|
||||||
## 快速开始
|
## 功能概览
|
||||||
|
|
||||||
### 1. 安装依赖
|
- 任务管理：新增、编辑、启用、禁用、删除任务
|
||||||
|
- 手动执行：支持单任务运行和批量运行
|
||||||
|
- 结果查看：按城市、板块、类型筛选抓取结果
|
||||||
|
- 项目查询：按项目名称、金额、日期范围过滤项目
|
||||||
|
- 系统设置：维护 Agent、定时任务、邮件配置
|
||||||
|
- 定时调度：支持 cron 表达式配置
|
||||||
|
- 数据持久化：任务与结果保存在 SQLite
|
||||||
|
|
||||||
|
## 环境要求
|
||||||
|
|
||||||
|
- Node.js 20 及以上可运行当前项目
|
||||||
|
- 建议使用 Node.js 22
|
||||||
|
|
||||||
|
说明：
|
||||||
|
|
||||||
|
- 当前依赖 `@mendable/firecrawl-js` 会提示要求 Node `>=22`
|
||||||
|
- 目前在 Node `20.19.4` 下可以安装、构建和启动
|
||||||
|
- 如果后续采集运行时出现环境兼容问题，优先升级到 Node 22
|
||||||
|
|
||||||
|
## 安装依赖
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm install
|
npm install
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. 配置文件
|
## 配置文件
|
||||||
|
|
||||||
首次使用需要创建配置文件：
|
项目根目录下使用 `config.json` 作为运行配置文件。
|
||||||
|
|
||||||
|
如果你还没有该文件，可以参考：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 复制示例配置文件
|
|
||||||
cp config.example.json config.json
|
cp config.example.json config.json
|
||||||
|
|
||||||
# 编辑配置文件（或通过 Web 界面配置）
|
|
||||||
# 填写邮件服务器信息和定时任务设置
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**配置文件说明：**
|
需要重点配置的内容包括：
|
||||||
|
|
||||||
- `config.example.json` - 配置模板（不含敏感信息，可提交到 Git）
|
- `agent.baseUrl`
|
||||||
- `config.json` - 实际配置（包含密码等敏感信息，已在 .gitignore 中忽略）
|
- `scheduler.enabled`
|
||||||
|
- `scheduler.cronTime`
|
||||||
|
- `email.smtpHost`
|
||||||
|
- `email.smtpUser`
|
||||||
|
- `email.smtpPass`
|
||||||
|
- `email.recipients`
|
||||||
|
|
||||||
## 使用方法
|
## 启动方式
|
||||||
|
|
||||||
### 1. 启动服务器
|
### 开发模式
|
||||||
|
|
||||||
|
前后端同时启动：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
启动后地址如下：
|
||||||
|
|
||||||
|
- 前端开发服务：`http://localhost:5173`
|
||||||
|
- 后端 API 服务：`http://localhost:5000`
|
||||||
|
|
||||||
|
说明：
|
||||||
|
|
||||||
|
- Vite 前端会通过代理把 `/api` 请求转发到 `5000`
|
||||||
|
- 开发时建议直接访问 `http://localhost:5173`
|
||||||
|
|
||||||
|
### 生产模式
|
||||||
|
|
||||||
|
先构建前端，再启动 Express：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run build
|
||||||
npm start
|
npm start
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. 访问界面
|
启动后访问：
|
||||||
|
|
||||||
打开浏览器访问: **http://localhost:5000** （或您配置的端口）
|
- `http://localhost:5000`
|
||||||
|
|
||||||
### 3. 功能介绍
|
说明：
|
||||||
|
|
||||||
**公告列表标签**
|
- `npm run build` 会把前端构建到 `dist/`
|
||||||
|
- `npm start` 会由 Express 托管 `dist/` 静态资源和 `/api`
|
||||||
|
|
||||||
- 快速查看所有公告
|
## 常用脚本
|
||||||
- 支持分页浏览
|
|
||||||
- 一键获取最新公告列表
|
|
||||||
|
|
||||||
**详情采集标签**
|
|
||||||
|
|
||||||
- 批量采集公告详情
|
|
||||||
- 支持按时间范围采集
|
|
||||||
- 自动提取预算金额
|
|
||||||
- 可自定义采集数量
|
|
||||||
|
|
||||||
**生成报告标签**
|
|
||||||
|
|
||||||
- 支持按时间范围生成报告
|
|
||||||
- 设置金额阈值筛选项目
|
|
||||||
- 实时统计项目信息
|
|
||||||
- 一键导出 Word/Markdown 报告
|
|
||||||
|
|
||||||
**定时任务标签** ⭐ 新增
|
|
||||||
|
|
||||||
- Web 界面配置定时任务
|
|
||||||
- 支持 Cron 表达式自定义执行时间
|
|
||||||
- 可选时间范围（今日/本周/本月）
|
|
||||||
- 设置金额阈值自动筛选
|
|
||||||
- 实时查看任务运行状态
|
|
||||||
- 立即测试运行功能
|
|
||||||
|
|
||||||
**邮件配置标签** ⭐ 新增
|
|
||||||
|
|
||||||
- Web 界面配置 SMTP 邮件服务
|
|
||||||
- 支持主流邮箱（QQ、163、Gmail 等）
|
|
||||||
- 测试连接功能验证配置
|
|
||||||
- 支持多个收件人
|
|
||||||
- 自动发送精美 HTML 报告
|
|
||||||
|
|
||||||
## 报告示例
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# 南京公共工程建设项目报告
|
|
||||||
|
|
||||||
**生成时间**: 2025/12/12 11:00:03
|
|
||||||
|
|
||||||
## 统计摘要
|
|
||||||
|
|
||||||
- 总项目数: 10
|
|
||||||
- 超过 50 万元的项目: 3
|
|
||||||
- 总金额: 5395.50 万元
|
|
||||||
|
|
||||||
## 项目列表
|
|
||||||
|
|
||||||
### 1. 项目名称
|
|
||||||
|
|
||||||
- **发布日期**: 2025-12-12
|
|
||||||
- **发布时间**: 2025-12-12 10:35:00
|
|
||||||
- **预算金额**: 5000 万元
|
|
||||||
- **链接**: https://...
|
|
||||||
```
|
|
||||||
|
|
||||||
## API 接口文档
|
|
||||||
|
|
||||||
服务器启动后提供以下 RESTful API 接口:
|
|
||||||
|
|
||||||
### 1. 获取公告列表
|
|
||||||
|
|
||||||
```
|
|
||||||
GET /api/list?url=<列表页URL>&page=<页码>
|
|
||||||
```
|
|
||||||
|
|
||||||
参数:
|
|
||||||
|
|
||||||
- `url` (可选): 列表页 URL,默认为官网首页
|
|
||||||
- `page` (可选): 页码,默认为 1
|
|
||||||
|
|
||||||
### 2. 按时间范围获取列表
|
|
||||||
|
|
||||||
```
|
|
||||||
POST /api/list-daterange
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"startDate": "2025-11-01",
|
|
||||||
"endDate": "2025-12-31",
|
|
||||||
"maxPages": 23
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. 批量获取详情
|
|
||||||
|
|
||||||
```
|
|
||||||
POST /api/details
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"items": [{ "title": "...", "href": "...", "date": "..." }],
|
|
||||||
"limit": 10
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. 生成报告
|
|
||||||
|
|
||||||
```
|
|
||||||
POST /api/report
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"url": "https://gjzx.nanjing.gov.cn/gggs/",
|
|
||||||
"limit": 15,
|
|
||||||
"threshold": 50
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. 按时间范围生成报告
|
|
||||||
|
|
||||||
```
|
|
||||||
POST /api/report-daterange
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"startDate": "2025-11-01",
|
|
||||||
"endDate": "2025-12-31",
|
|
||||||
"threshold": 50,
|
|
||||||
"maxPages": 23
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 6. 获取配置
|
|
||||||
|
|
||||||
```
|
|
||||||
GET /api/config
|
|
||||||
```
|
|
||||||
|
|
||||||
返回当前配置信息（密码会被隐藏）
|
|
||||||
|
|
||||||
### 7. 更新配置
|
|
||||||
|
|
||||||
```
|
|
||||||
POST /api/config
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"scheduler": {
|
|
||||||
"enabled": true,
|
|
||||||
"cronTime": "0 9 * * *",
|
|
||||||
"threshold": 100000,
|
|
||||||
"timeRange": "today"
|
|
||||||
},
|
|
||||||
"email": {
|
|
||||||
"smtpHost": "smtp.qq.com",
|
|
||||||
"smtpPort": 587,
|
|
||||||
"smtpUser": "your-email@qq.com",
|
|
||||||
"smtpPass": "your-password",
|
|
||||||
"recipients": "recipient@example.com"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 8. 获取定时任务状态
|
|
||||||
|
|
||||||
```
|
|
||||||
GET /api/scheduler/status
|
|
||||||
```
|
|
||||||
|
|
||||||
### 9. 手动触发定时任务
|
|
||||||
|
|
||||||
```
|
|
||||||
POST /api/run-scheduled-task
|
|
||||||
```
|
|
||||||
|
|
||||||
## 定时任务配置
|
|
||||||
|
|
||||||
### 通过 Web 界面配置（推荐）
|
|
||||||
|
|
||||||
1. 访问 `http://localhost:5000` （或您配置的端口）
|
|
||||||
2. 切换到 **"定时任务"** 标签
|
|
||||||
3. 配置以下选项：
|
|
||||||
- **启用定时任务**：勾选启用
|
|
||||||
- **执行时间**：选择预设时间或自定义 Cron 表达式
|
|
||||||
- **时间范围**：今日/本周/本月
|
|
||||||
- **金额阈值**：设置最低金额（单位：万元）
|
|
||||||
4. 切换到 **"邮件配置"** 标签
|
|
||||||
5. 配置 SMTP 邮件服务器信息
|
|
||||||
6. 点击"测试连接"验证配置
|
|
||||||
7. 点击"保存配置"，定时任务自动生效
|
|
||||||
|
|
||||||
### 通过配置文件
|
|
||||||
|
|
||||||
编辑 `config.json`：
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"scheduler": {
|
|
||||||
"enabled": true, // 是否启用
|
|
||||||
"cronTime": "0 9 * * *", // Cron表达式：每天9点
|
|
||||||
"threshold": 100000, // 金额阈值（万元，10亿）
|
|
||||||
"description": "每天9点采集大于10亿的项目",
|
|
||||||
"timeRange": "today" // 采集范围: today/thisWeek/thisMonth
|
|
||||||
},
|
|
||||||
"email": {
|
|
||||||
"smtpHost": "smtp.qq.com", // SMTP服务器
|
|
||||||
"smtpPort": 587, // 端口
|
|
||||||
"smtpUser": "your@qq.com", // 发件邮箱
|
|
||||||
"smtpPass": "授权码", // QQ邮箱授权码
|
|
||||||
"recipients": "to@qq.com" // 收件人（多个用逗号分隔）
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Cron 表达式说明
|
|
||||||
|
|
||||||
格式：`分 时 日 月 周`
|
|
||||||
|
|
||||||
常用示例：
|
|
||||||
- `0 9 * * *` - 每天 9:00
|
|
||||||
- `0 9,18 * * *` - 每天 9:00 和 18:00
|
|
||||||
- `0 9 * * 1` - 每周一 9:00
|
|
||||||
- `0 9 1 * *` - 每月1号 9:00
|
|
||||||
- `0 */6 * * *` - 每 6 小时
|
|
||||||
|
|
||||||
### 邮件服务配置参考
|
|
||||||
|
|
||||||
**QQ 邮箱：**
|
|
||||||
- SMTP: `smtp.qq.com`
|
|
||||||
- 端口: `587`
|
|
||||||
- 密码: 使用授权码（在 QQ 邮箱设置中生成）
|
|
||||||
|
|
||||||
**163 邮箱：**
|
|
||||||
- SMTP: `smtp.163.com`
|
|
||||||
- 端口: `465`
|
|
||||||
- 密码: 使用授权码
|
|
||||||
|
|
||||||
**Gmail：**
|
|
||||||
- SMTP: `smtp.gmail.com`
|
|
||||||
- 端口: `587`
|
|
||||||
- 需要开启"不够安全的应用访问"
|
|
||||||
|
|
||||||
## 服务器部署
|
|
||||||
|
|
||||||
### 方法 1：使用 PM2（推荐）
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 安装 PM2
|
npm run dev
|
||||||
npm install -g pm2
|
npm run build
|
||||||
|
npm run preview
|
||||||
# 启动服务
|
npm start
|
||||||
pm2 start src/server.js --name gjzx-scraper
|
|
||||||
|
|
||||||
# 查看状态
|
|
||||||
pm2 status
|
|
||||||
|
|
||||||
# 查看日志
|
|
||||||
pm2 logs gjzx-scraper
|
|
||||||
|
|
||||||
# 设置开机自启
|
|
||||||
pm2 startup
|
|
||||||
pm2 save
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 方法 2：使用 systemd
|
脚本说明：
|
||||||
|
|
||||||
创建服务文件 `/etc/systemd/system/gjzx-scraper.service`：
|
- `npm run dev`：开发模式，前后端同时启动
|
||||||
|
- `npm run build`：构建前端生产包
|
||||||
|
- `npm run preview`：本地预览前端构建结果
|
||||||
|
- `npm start`：启动后端服务，并托管前端构建产物
|
||||||
|
|
||||||
```ini
|
## 页面说明
|
||||||
[Unit]
|
|
||||||
Description=GJZX Scraper Service
|
|
||||||
After=network.target
|
|
||||||
|
|
||||||
[Service]
|
### 任务配置
|
||||||
Type=simple
|
|
||||||
User=your-user
|
|
||||||
WorkingDirectory=/path/to/tool-node
|
|
||||||
ExecStart=/usr/bin/node src/server.js
|
|
||||||
Restart=always
|
|
||||||
RestartSec=10
|
|
||||||
|
|
||||||
[Install]
|
- 管理抓取任务
|
||||||
WantedBy=multi-user.target
|
- 支持单独运行和批量运行
|
||||||
```
|
- 支持查看运行状态
|
||||||
|
|
||||||
启动服务：
|
### 抓取结果
|
||||||
|
|
||||||
```bash
|
- 查看每次抓取生成的记录
|
||||||
sudo systemctl enable gjzx-scraper
|
- 支持按条件筛选
|
||||||
sudo systemctl start gjzx-scraper
|
- 支持删除结果
|
||||||
sudo systemctl status gjzx-scraper
|
|
||||||
```
|
|
||||||
|
|
||||||
### 注意事项
|
### 项目管理
|
||||||
|
|
||||||
- ✅ **无需数据库**：项目采用无数据库架构，轻量部署
|
- 对抓取出的项目做去重查询
|
||||||
- ✅ **配置持久化**：所有配置保存在 `config.json` 文件中
|
- 支持金额和日期范围过滤
|
||||||
- ✅ **进程保活**：使用 PM2 或 systemd 确保进程持续运行
|
|
||||||
- ⚠️ **防火墙**：确保配置的端口可访问（默认 5000）
|
### 系统设置
|
||||||
- ⚠️ **配置安全**：不要将 `config.json` 提交到公开仓库
|
|
||||||
|
- 配置 Agent 服务地址
|
||||||
|
- 配置定时任务
|
||||||
|
- 配置邮件发送参数
|
||||||
|
|
||||||
## 技术栈
|
## 技术栈
|
||||||
|
|
||||||
- **后端**: Node.js + Express
|
- 前端：Vue 3、Vite、Vue Router、Axios、Element Plus
|
||||||
- **爬虫**: Axios + Cheerio
|
- 后端：Express
|
||||||
- **定时任务**: node-cron
|
- 数据库：better-sqlite3
|
||||||
- **邮件服务**: nodemailer
|
- 调度：node-cron
|
||||||
- **前端**: 原生 HTML/CSS/JavaScript
|
- 邮件：nodemailer
|
||||||
- **编码处理**: iconv-lite (支持 GBK/UTF-8)
|
|
||||||
- **文档导出**: docx.js
|
|
||||||
- **架构**: 无数据库设计
|
|
||||||
|
|
||||||
## 项目结构
|
## 项目结构
|
||||||
|
|
||||||
```
|
```text
|
||||||
.
|
.
|
||||||
├── src/
|
├─ client/ # Vue 3 前端源码
|
||||||
│ ├── server.js # Web服务器及API
|
│ ├─ index.html
|
||||||
│ ├── scheduler.js # 定时任务调度器
|
│ └─ src/
|
||||||
│ └── emailService.js # 邮件发送服务
|
│ ├─ api/
|
||||||
├── public/
|
│ ├─ components/
|
||||||
│ ├── index.html # Web界面
|
│ ├─ pages/
|
||||||
│ └── app.js # 前端逻辑
|
│ ├─ router/
|
||||||
├── config.json # 配置文件（不提交到Git）
|
│ ├─ App.vue
|
||||||
├── config.example.json # 配置示例
|
│ ├─ main.js
|
||||||
├── package.json
|
│ └─ styles.css
|
||||||
└── README.md
|
├─ dist/ # Vite 构建产物
|
||||||
|
├─ src/ # Express 服务端
|
||||||
|
│ ├─ server.js
|
||||||
|
│ ├─ scheduler.js
|
||||||
|
│ ├─ resultStore.js
|
||||||
|
│ ├─ agentService.js
|
||||||
|
│ └─ emailService.js
|
||||||
|
├─ data/ # SQLite 数据文件目录
|
||||||
|
├─ config.json # 运行配置
|
||||||
|
├─ config.example.json # 配置示例
|
||||||
|
├─ package.json
|
||||||
|
├─ vite.config.js
|
||||||
|
└─ README.md
|
||||||
```
|
```
|
||||||
|
|
||||||
## 架构特点
|
## 数据说明
|
||||||
|
|
||||||
### 无数据库设计
|
- 任务数据和抓取结果保存在 SQLite
|
||||||
|
- 默认数据库路径位于 `data/results.sqlite`
|
||||||
|
- 配置数据保存在根目录 `config.json`
|
||||||
|
|
||||||
本项目采用**无数据库架构**，具有以下特点：
|
## 部署建议
|
||||||
|
|
||||||
- ✅ **轻量部署**：无需安装和配置数据库
|
- 使用 PM2 或 systemd 保持进程常驻
|
||||||
- ✅ **实时数据**：每次从源站实时抓取最新数据
|
- 通过反向代理暴露 `5000` 端口
|
||||||
- ✅ **配置简单**：只需配置 config.json 文件
|
- 不要把 `config.json`、`data/`、`.env` 提交到公开仓库
|
||||||
- ✅ **邮件归档**：报告通过邮件发送，邮箱即为历史记录
|
|
||||||
- ✅ **低资源消耗**：内存占用小，适合小型服务器
|
|
||||||
|
|
||||||
### 数据流程
|
## 备注
|
||||||
|
|
||||||
```
|
- 旧版 `public/` 页面已不再作为当前主前端入口
|
||||||
定时触发 → 抓取网站数据 → 解析提取 → 筛选过滤 → 生成报告 → 发送邮件
|
- 当前主前端以 `client/` 目录为准
|
||||||
↑ ↓
|
|
||||||
└──────────────────── 配置文件 ──────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## 注意事项
|
|
||||||
|
|
||||||
1. **采集速度**：已限制为每条延迟 500ms-1s，避免请求过快
|
|
||||||
2. **域名支持**：仅支持 gjzx.nanjing.gov.cn 域名的详情页解析
|
|
||||||
3. **金额提取**：基于正则匹配，支持多种格式（预算金额、最高限价等）
|
|
||||||
4. **端口配置**：Web 服务器默认端口 5000，支持通过环境变量 PORT 修改
|
|
||||||
5. **智能停止**：按时间范围采集会在检测到所有公告早于起始日期时自动停止
|
|
||||||
6. **编码处理**：自动识别，支持 GBK 和 UTF-8 网页
|
|
||||||
7. **配置安全**：config.json 包含敏感信息，已加入 .gitignore，不要提交到公开仓库
|
|
||||||
8. **进程保活**：部署时使用 PM2 或 systemd 确保定时任务持续运行
|
|
||||||
|
|
||||||
## 核心功能说明
|
|
||||||
|
|
||||||
### 时间范围采集逻辑
|
|
||||||
|
|
||||||
按时间范围采集时,程序会:
|
|
||||||
|
|
||||||
1. 从第一页开始顺序采集
|
|
||||||
2. 检查每页公告的日期是否在指定范围内
|
|
||||||
3. 如果某页所有公告都早于起始日期,自动停止采集
|
|
||||||
4. 支持设置最大页数限制,避免过度采集
|
|
||||||
|
|
||||||
### 金额提取规则
|
|
||||||
|
|
||||||
支持识别以下格式:
|
|
||||||
|
|
||||||
- 预算金额: XX 万元
|
|
||||||
- 最高限价: XX 万元
|
|
||||||
- 预算: XX 万元
|
|
||||||
- 金额: XX 万元
|
|
||||||
- 直接数字: XX 万元
|
|
||||||
|
|
||||||
### 编码处理
|
|
||||||
|
|
||||||
自动识别网页编码:
|
|
||||||
|
|
||||||
- 优先读取 Content-Type 中的 charset
|
|
||||||
- 自动处理 GBK、GB2312 编码
|
|
||||||
- 默认使用 UTF-8
|
|
||||||
|
|
||||||
## 常见问题
|
|
||||||
|
|
||||||
### Q: 为什么采集速度比较慢？
|
|
||||||
|
|
||||||
A: 为了避免对服务器造成过大压力，程序限制了请求频率（每条延迟 500ms-1s）。这是一个负责任的爬虫设计。
|
|
||||||
|
|
||||||
### Q: 如何采集指定日期范围的公告？
|
|
||||||
|
|
||||||
A: 在 Web 界面的"详情采集"和"生成报告"标签中勾选"按时间范围采集"，然后输入起始和结束日期即可。
|
|
||||||
|
|
||||||
### Q: 导出的报告在哪里？
|
|
||||||
|
|
||||||
A: 点击"导出 Word"或"导出 Markdown"按钮后会自动下载到浏览器的默认下载目录。
|
|
||||||
|
|
||||||
### Q: 可以采集其他网站吗？
|
|
||||||
|
|
||||||
A: 需要修改 server.js 中的 BASE_URL 和相应的解析函数，因为不同网站的 HTML 结构不同。
|
|
||||||
|
|
||||||
### Q: 需要安装数据库吗？
|
|
||||||
|
|
||||||
A: **不需要！** 本项目采用无数据库架构，只需安装 Node.js 依赖即可运行。所有配置保存在 config.json 文件中，报告通过邮件发送。
|
|
||||||
|
|
||||||
### Q: 定时任务配置后不生效怎么办？
|
|
||||||
|
|
||||||
A: 请检查以下几点：
|
|
||||||
1. `config.json` 中 `scheduler.enabled` 是否为 `true`
|
|
||||||
2. 邮件配置是否正确（SMTP 服务器、用户名、密码）
|
|
||||||
3. Node.js 进程是否持续运行（使用 PM2 查看状态）
|
|
||||||
4. 查看服务器日志是否有错误信息
|
|
||||||
|
|
||||||
### Q: 如何修改定时任务的执行时间？
|
|
||||||
|
|
||||||
A: 有两种方式：
|
|
||||||
1. **Web 界面**（推荐）：访问网页，切换到"定时任务"标签，选择预设时间或自定义 Cron 表达式，点击保存
|
|
||||||
2. **配置文件**：编辑 `config.json` 中的 `scheduler.cronTime` 字段
|
|
||||||
|
|
||||||
### Q: 邮件发送失败怎么办？
|
|
||||||
|
|
||||||
A: 常见原因：
|
|
||||||
1. **QQ 邮箱**：需要使用授权码，不是登录密码（在 QQ 邮箱设置 → 账户 → POP3/SMTP 服务中生成）
|
|
||||||
2. **端口问题**：QQ 邮箱使用 587，某些邮箱可能需要 465（SSL）
|
|
||||||
3. **防火墙**：确保服务器可以访问 SMTP 端口
|
|
||||||
4. **测试连接**：在"邮件配置"标签中使用"测试连接"功能验证配置
|
|
||||||
|
|
||||||
### Q: 如何查看定时任务运行日志？
|
|
||||||
|
|
||||||
A: 如果使用 PM2 部署：
|
|
||||||
```bash
|
|
||||||
pm2 logs gjzx-scraper
|
|
||||||
```
|
|
||||||
|
|
||||||
如果使用 systemd：
|
|
||||||
```bash
|
|
||||||
sudo journalctl -u gjzx-scraper -f
|
|
||||||
```
|
|
||||||
|
|
||||||
### Q: 服务器重启后定时任务还会运行吗？
|
|
||||||
|
|
||||||
A: 需要设置开机自启：
|
|
||||||
- **PM2 方式**：执行 `pm2 startup` 和 `pm2 save`
|
|
||||||
- **systemd 方式**：执行 `sudo systemctl enable gjzx-scraper`
|
|
||||||
|
|
||||||
### Q: 可以设置多个定时任务吗？
|
|
||||||
|
|
||||||
A: 当前版本只支持一个定时任务。如需多个任务，可以：
|
|
||||||
1. 使用 Cron 表达式设置多个时间点（如 `0 9,18 * * *` 表示每天 9:00 和 18:00）
|
|
||||||
2. 部署多个实例，使用不同的配置文件和端口
|
|
||||||
|
|
||||||
## 更新日志
|
|
||||||
|
|
||||||
### v1.1.0 (2025-12-15)
|
|
||||||
|
|
||||||
- ✨ 新增定时任务功能（node-cron）
|
|
||||||
- ✨ 新增邮件推送服务（nodemailer）
|
|
||||||
- ✨ 新增 Web 配置界面（定时任务 + 邮件配置）
|
|
||||||
- ✨ 支持多时间范围（今日/本周/本月）
|
|
||||||
- ✨ 配置 API（读取/更新配置）
|
|
||||||
- ✨ 定时任务状态查询
|
|
||||||
- ✨ 手动触发任务功能
|
|
||||||
- 📝 完善文档，新增部署指南
|
|
||||||
|
|
||||||
### v1.0.0 (2025-12-12)
|
|
||||||
|
|
||||||
- Web 可视化界面
|
|
||||||
- 支持按时间范围采集
|
|
||||||
- 支持分页浏览
|
|
||||||
- 支持导出 Word/Markdown 报告
|
|
||||||
- RESTful API 接口
|
|
||||||
- 自动编码识别
|
|
||||||
- 智能金额提取
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
MIT
|
|
||||||
|
|||||||
12
client/index.html
Normal file
12
client/index.html
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="zh-CN">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<title>公告抓取与分析工具</title>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="app"></div>
|
||||||
|
<script type="module" src="/src/main.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
10
client/src/App.vue
Normal file
10
client/src/App.vue
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
<script setup>
|
||||||
|
import zhCn from 'element-plus/es/locale/lang/zh-cn';
|
||||||
|
import AppShell from './components/AppShell.vue';
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<el-config-provider :locale="zhCn">
|
||||||
|
<AppShell />
|
||||||
|
</el-config-provider>
|
||||||
|
</template>
|
||||||
24
client/src/api/http.js
Normal file
24
client/src/api/http.js
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
import axios from 'axios';
|
||||||
|
|
||||||
|
const http = axios.create({
|
||||||
|
baseURL: '/api',
|
||||||
|
timeout: 30000,
|
||||||
|
});
|
||||||
|
|
||||||
|
http.interceptors.response.use(
|
||||||
|
(response) => {
|
||||||
|
const payload = response.data;
|
||||||
|
|
||||||
|
if (payload && payload.success === false) {
|
||||||
|
return Promise.reject(new Error(payload.error || '请求失败'));
|
||||||
|
}
|
||||||
|
|
||||||
|
return payload;
|
||||||
|
},
|
||||||
|
(error) => {
|
||||||
|
const message = error.response?.data?.error || error.message || '请求失败';
|
||||||
|
return Promise.reject(new Error(message));
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
export default http;
|
||||||
25
client/src/api/results.js
Normal file
25
client/src/api/results.js
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import http from './http';
|
||||||
|
|
||||||
|
export function fetchResults(params = {}) {
|
||||||
|
return http.get('/results', { params }).then((payload) => payload);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fetchResultFilters(params = {}) {
|
||||||
|
return http.get('/results/filters', { params }).then((payload) => payload.data || {});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function deleteResult(id) {
|
||||||
|
return http.delete(`/results/${id}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function clearResults() {
|
||||||
|
return http.delete('/results');
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fetchProjects(params = {}) {
|
||||||
|
return http.get('/projects', { params }).then((payload) => payload);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fetchProjectFilters() {
|
||||||
|
return http.get('/projects/filters').then((payload) => payload.data || {});
|
||||||
|
}
|
||||||
17
client/src/api/settings.js
Normal file
17
client/src/api/settings.js
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
import http from './http';
|
||||||
|
|
||||||
|
export function fetchConfig() {
|
||||||
|
return http.get('/config').then((payload) => payload.data || {});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function saveConfig(payload) {
|
||||||
|
return http.post('/config', payload);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fetchSchedulerStatus() {
|
||||||
|
return http.get('/scheduler/status').then((payload) => payload.data || {});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function triggerScheduledTask() {
|
||||||
|
return http.post('/run-scheduled-task');
|
||||||
|
}
|
||||||
34
client/src/api/tasks.js
Normal file
34
client/src/api/tasks.js
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
import http from './http';
|
||||||
|
|
||||||
|
export function fetchTasks() {
|
||||||
|
return http.get('/tasks').then((payload) => payload.data || []);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createTask(payload) {
|
||||||
|
return http.post('/tasks', payload).then((response) => response.data);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function updateTask(id, payload) {
|
||||||
|
return http.put(`/tasks/${id}`, payload).then((response) => response.data);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function deleteTask(id) {
|
||||||
|
return http.delete(`/tasks/${id}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runTask(id) {
|
||||||
|
return http.post(`/tasks/${id}/run`);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runAllTasks(ids = []) {
|
||||||
|
const body = ids.length ? { ids } : {};
|
||||||
|
return http.post('/tasks/run', body);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toggleTask(id, enabled) {
|
||||||
|
return http.put(`/tasks/${id}`, { enabled }).then((response) => response.data);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fetchTaskStatus() {
|
||||||
|
return http.get('/tasks/status').then((payload) => payload.data || { isRunning: false });
|
||||||
|
}
|
||||||
41
client/src/components/AppShell.vue
Normal file
41
client/src/components/AppShell.vue
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
<script setup>
|
||||||
|
import { computed } from 'vue';
|
||||||
|
import { useRoute } from 'vue-router';
|
||||||
|
|
||||||
|
const route = useRoute();
|
||||||
|
|
||||||
|
const subtitle = computed(() => route.meta?.subtitle || '把抓取任务、分析结果和调度配置收在同一套前端工作台里。');
|
||||||
|
|
||||||
|
const menuItems = [
|
||||||
|
{ index: '/tasks', label: '任务配置' },
|
||||||
|
{ index: '/results', label: '抓取结果' },
|
||||||
|
{ index: '/projects', label: '项目管理' },
|
||||||
|
{ index: '/settings', label: '系统设置' },
|
||||||
|
];
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<div class="app-shell">
|
||||||
|
<div class="app-frame">
|
||||||
|
<header class="app-topbar">
|
||||||
|
<div class="app-brand">
|
||||||
|
<div class="app-brand__mark">公</div>
|
||||||
|
<div class="app-brand__text">
|
||||||
|
<h1>公告抓取与分析工具</h1>
|
||||||
|
<p>{{ subtitle }}</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<el-menu :default-active="route.path" class="app-menu" mode="horizontal" router>
|
||||||
|
<el-menu-item v-for="item in menuItems" :key="item.index" :index="item.index">
|
||||||
|
{{ item.label }}
|
||||||
|
</el-menu-item>
|
||||||
|
</el-menu>
|
||||||
|
|
||||||
|
<main class="app-main">
|
||||||
|
<RouterView />
|
||||||
|
</main>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
55
client/src/components/ResultRecordCard.vue
Normal file
55
client/src/components/ResultRecordCard.vue
Normal file
@@ -0,0 +1,55 @@
|
|||||||
|
<script setup>
|
||||||
|
import { computed } from 'vue';
|
||||||
|
import { formatDateTime, formatYuan, pickResultLink, pickResultName } from '@/utils/format';
|
||||||
|
|
||||||
|
const props = defineProps({
|
||||||
|
record: {
|
||||||
|
type: Object,
|
||||||
|
required: true,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const emit = defineEmits(['delete']);
|
||||||
|
|
||||||
|
const items = computed(() => props.record.data?.results || []);
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<el-card shadow="hover" class="result-card-plus">
|
||||||
|
<template #header>
|
||||||
|
<div class="result-card-plus__header">
|
||||||
|
<div>
|
||||||
|
<div class="result-card-plus__title">{{ record.city || '未命名城市' }}</div>
|
||||||
|
<div class="result-card-plus__meta">
|
||||||
|
<el-tag size="small" type="primary">{{ items.length }} 条结果</el-tag>
|
||||||
|
<el-tag v-if="record.error" size="small" type="danger">执行失败</el-tag>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="result-card-plus__time">{{ formatDateTime(record.scrapedAt) }}</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<el-alert v-if="record.error" :title="`错误信息：${record.error}`" type="error" :closable="false" show-icon />
|
||||||
|
|
||||||
|
<div v-else-if="items.length" class="result-card-plus__list">
|
||||||
|
<div v-for="(item, index) in items" :key="`${record.id}-${index}`" class="result-card-plus__item">
|
||||||
|
<div class="result-card-plus__type">{{ item.type || '-' }}</div>
|
||||||
|
<div class="result-card-plus__name">{{ pickResultName(item) }}</div>
|
||||||
|
<div class="result-card-plus__amount">{{ formatYuan(item.amount_yuan) }}</div>
|
||||||
|
<div class="result-card-plus__date">{{ item.date || '-' }}</div>
|
||||||
|
<div class="result-card-plus__action">
|
||||||
|
<el-link v-if="pickResultLink(item)" :href="pickResultLink(item)" target="_blank" type="primary">查看详情</el-link>
|
||||||
|
<span v-else>-</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<el-empty v-else description="当前记录里没有可展示的数据" :image-size="88" />
|
||||||
|
|
||||||
|
<template #footer>
|
||||||
|
<div class="result-card-plus__footer">
|
||||||
|
<el-button type="danger" plain @click="emit('delete', record)">删除记录</el-button>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
</el-card>
|
||||||
|
</template>
|
||||||
105
client/src/components/TaskDialog.vue
Normal file
105
client/src/components/TaskDialog.vue
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
<script setup>
|
||||||
|
import { computed, reactive, watch } from 'vue';
|
||||||
|
import { TASK_MODE_OPTIONS } from '@/constants/taskModes';
|
||||||
|
|
||||||
|
const props = defineProps({
|
||||||
|
modelValue: {
|
||||||
|
type: Boolean,
|
||||||
|
default: false,
|
||||||
|
},
|
||||||
|
task: {
|
||||||
|
type: Object,
|
||||||
|
default: null,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const emit = defineEmits(['update:modelValue', 'submit']);
|
||||||
|
|
||||||
|
const form = reactive({
|
||||||
|
city: '',
|
||||||
|
plateName: '',
|
||||||
|
prompt: '',
|
||||||
|
mode: TASK_MODE_OPTIONS[0],
|
||||||
|
enabled: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const dialogTitle = computed(() => (props.task?.id ? '编辑任务' : '新增任务'));
|
||||||
|
const modeOptions = computed(() => {
|
||||||
|
const set = new Set(TASK_MODE_OPTIONS);
|
||||||
|
if (props.task?.mode) set.add(props.task.mode);
|
||||||
|
return [...set];
|
||||||
|
});
|
||||||
|
|
||||||
|
watch(
|
||||||
|
() => [props.modelValue, props.task],
|
||||||
|
() => {
|
||||||
|
form.city = props.task?.city || '';
|
||||||
|
form.plateName = props.task?.plateName || '';
|
||||||
|
form.prompt = props.task?.prompt || '';
|
||||||
|
form.mode = props.task?.mode || TASK_MODE_OPTIONS[0];
|
||||||
|
form.enabled = props.task?.enabled ?? true;
|
||||||
|
},
|
||||||
|
{ immediate: true },
|
||||||
|
);
|
||||||
|
|
||||||
|
function close() {
|
||||||
|
emit('update:modelValue', false);
|
||||||
|
}
|
||||||
|
|
||||||
|
function submit() {
|
||||||
|
emit('submit', {
|
||||||
|
city: form.city.trim(),
|
||||||
|
plateName: form.plateName.trim(),
|
||||||
|
prompt: form.prompt.trim(),
|
||||||
|
mode: form.mode.trim() || TASK_MODE_OPTIONS[0],
|
||||||
|
enabled: form.enabled,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<el-dialog :model-value="modelValue" :title="dialogTitle" width="760px" destroy-on-close @close="close">
|
||||||
|
<el-form label-position="top">
|
||||||
|
<el-form-item label="城市名称">
|
||||||
|
<el-input v-model="form.city" placeholder="如：南京市" />
|
||||||
|
</el-form-item>
|
||||||
|
|
||||||
|
<el-form-item label="板块名称">
|
||||||
|
<el-input v-model="form.plateName" placeholder="如：工程" />
|
||||||
|
</el-form-item>
|
||||||
|
|
||||||
|
<el-form-item label="提示词">
|
||||||
|
<el-input
|
||||||
|
v-model="form.prompt"
|
||||||
|
type="textarea"
|
||||||
|
:autosize="{ minRows: 6, maxRows: 12 }"
|
||||||
|
placeholder="请访问目标网站，抓取今天的招标公告和中标公告信息……"
|
||||||
|
/>
|
||||||
|
</el-form-item>
|
||||||
|
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="模型">
|
||||||
|
<el-select v-model="form.mode" placeholder="选择模型" style="width: 100%;">
|
||||||
|
<el-option v-for="option in modeOptions" :key="option" :label="option" :value="option" />
|
||||||
|
</el-select>
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="状态">
|
||||||
|
<el-switch v-model="form.enabled" inline-prompt active-text="启用" inactive-text="禁用" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
|
||||||
|
<el-alert title="建议把目标网址、筛选规则和字段要求都写进提示词里，后续维护会更稳定。" type="info" :closable="false" show-icon />
|
||||||
|
|
||||||
|
<template #footer>
|
||||||
|
<div class="dialog-footer">
|
||||||
|
<el-button @click="close">取消</el-button>
|
||||||
|
<el-button type="primary" @click="submit">保存任务</el-button>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
</el-dialog>
|
||||||
|
</template>
|
||||||
10
client/src/constants/taskModes.js
Normal file
10
client/src/constants/taskModes.js
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
export const TASK_MODE_OPTIONS = [
|
||||||
|
'qwen3.5-plus',
|
||||||
|
'qwen3-max-2026-01-23',
|
||||||
|
'qwen3-coder-next',
|
||||||
|
'qwen3-coder-plus',
|
||||||
|
'glm-5',
|
||||||
|
'glm-4.7',
|
||||||
|
'kimi-k2.5',
|
||||||
|
'MiniMax-M2.5',
|
||||||
|
];
|
||||||
6
client/src/main.js
Normal file
6
client/src/main.js
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
import { createApp } from 'vue';
|
||||||
|
import App from './App.vue';
|
||||||
|
import router from './router';
|
||||||
|
import './styles.css';
|
||||||
|
|
||||||
|
createApp(App).use(router).mount('#app');
|
||||||
233
client/src/pages/ProjectsPage.vue
Normal file
233
client/src/pages/ProjectsPage.vue
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
<script setup>
|
||||||
|
import { onMounted, reactive, ref } from 'vue';
|
||||||
|
import { ElMessage } from 'element-plus';
|
||||||
|
import { fetchProjectFilters, fetchProjects } from '@/api/results';
|
||||||
|
import { formatYuan } from '@/utils/format';
|
||||||
|
|
||||||
|
const form = reactive({
|
||||||
|
city: '',
|
||||||
|
section: '',
|
||||||
|
projectName: '',
|
||||||
|
minAmount: '',
|
||||||
|
maxAmount: '',
|
||||||
|
startDate: '',
|
||||||
|
endDate: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
const activeFilters = reactive({
|
||||||
|
city: '',
|
||||||
|
section: '',
|
||||||
|
projectName: '',
|
||||||
|
minAmount: '',
|
||||||
|
maxAmount: '',
|
||||||
|
startDate: '',
|
||||||
|
endDate: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
const options = reactive({
|
||||||
|
cities: [],
|
||||||
|
sections: [],
|
||||||
|
});
|
||||||
|
|
||||||
|
const state = reactive({
|
||||||
|
records: [],
|
||||||
|
page: 1,
|
||||||
|
pageSize: 10,
|
||||||
|
total: 0,
|
||||||
|
loading: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const summaryText = ref('加载中…');
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
await Promise.all([loadFilters(), loadProjects()]);
|
||||||
|
});
|
||||||
|
|
||||||
|
async function loadFilters() {
|
||||||
|
try {
|
||||||
|
const data = await fetchProjectFilters();
|
||||||
|
options.cities = data.cities || [];
|
||||||
|
options.sections = data.sections || [];
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`项目筛选项加载失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function validateForm() {
|
||||||
|
if (form.minAmount && form.maxAmount && Number(form.minAmount) > Number(form.maxAmount)) {
|
||||||
|
throw new Error('最小金额不能大于最大金额');
|
||||||
|
}
|
||||||
|
|
||||||
|
if (form.startDate && form.endDate && form.startDate > form.endDate) {
|
||||||
|
throw new Error('开始日期不能晚于结束日期');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function syncActiveFilters() {
|
||||||
|
Object.assign(activeFilters, form);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadProjects() {
|
||||||
|
state.loading = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const payload = await fetchProjects({
|
||||||
|
...activeFilters,
|
||||||
|
page: state.page,
|
||||||
|
pageSize: state.pageSize,
|
||||||
|
});
|
||||||
|
|
||||||
|
state.records = payload.data || [];
|
||||||
|
state.total = payload.total || 0;
|
||||||
|
state.page = payload.page || 1;
|
||||||
|
|
||||||
|
if (!state.records.length) {
|
||||||
|
summaryText.value = '当前筛选条件下没有项目';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const startIndex = (state.page - 1) * state.pageSize + 1;
|
||||||
|
const endIndex = Math.min(state.page * state.pageSize, state.total);
|
||||||
|
summaryText.value = `显示第 ${startIndex} 至 ${endIndex} 条记录，共 ${state.total} 条`;
|
||||||
|
} catch (error) {
|
||||||
|
state.records = [];
|
||||||
|
state.total = 0;
|
||||||
|
summaryText.value = `项目加载失败：${error.message}`;
|
||||||
|
ElMessage.error(`项目加载失败：${error.message}`);
|
||||||
|
} finally {
|
||||||
|
state.loading = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function search() {
|
||||||
|
try {
|
||||||
|
validateForm();
|
||||||
|
syncActiveFilters();
|
||||||
|
state.page = 1;
|
||||||
|
await loadProjects();
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(error.message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function reset() {
|
||||||
|
Object.keys(form).forEach((key) => {
|
||||||
|
form[key] = '';
|
||||||
|
activeFilters[key] = '';
|
||||||
|
});
|
||||||
|
state.page = 1;
|
||||||
|
await loadProjects();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function changePage(page) {
|
||||||
|
state.page = page;
|
||||||
|
await loadProjects();
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<section class="page-grid">
|
||||||
|
<header class="page-head">
|
||||||
|
<div>
|
||||||
|
<h2>项目管理</h2>
|
||||||
|
<p>用 Element Plus 表单和表格组织项目去重查询，适合继续扩展导出、排序和列配置。</p>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<el-form label-position="top">
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="城市">
|
||||||
|
<el-select v-model="form.city" placeholder="全部城市" clearable style="width: 100%;">
|
||||||
|
<el-option v-for="item in options.cities" :key="item" :label="item" :value="item" />
|
||||||
|
</el-select>
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="板块">
|
||||||
|
<el-select v-model="form.section" placeholder="全部板块" clearable style="width: 100%;">
|
||||||
|
<el-option v-for="item in options.sections" :key="item" :label="item" :value="item" />
|
||||||
|
</el-select>
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="项目名称">
|
||||||
|
<el-input v-model="form.projectName" placeholder="输入项目名关键字" clearable />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="最小金额">
|
||||||
|
<el-input-number v-model="form.minAmount" :min="0" :precision="2" :controls="false" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="最大金额">
|
||||||
|
<el-input-number v-model="form.maxAmount" :min="0" :precision="2" :controls="false" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="开始日期">
|
||||||
|
<el-date-picker v-model="form.startDate" type="date" value-format="YYYY-MM-DD" placeholder="选择日期" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-form-item label="结束日期">
|
||||||
|
<el-date-picker v-model="form.endDate" type="date" value-format="YYYY-MM-DD" placeholder="选择日期" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
|
||||||
|
<div class="toolbar">
|
||||||
|
<div class="summary-text">按项目名称去重后展示，方便快速判断有哪些有效项目。</div>
|
||||||
|
<div class="toolbar__group">
|
||||||
|
<el-button @click="reset">重置</el-button>
|
||||||
|
<el-button type="primary" @click="search">查询项目</el-button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</el-card>
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<template #header>
|
||||||
|
<div class="toolbar">
|
||||||
|
<span>{{ summaryText }}</span>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<el-table v-loading="state.loading" :data="state.records" style="width: 100%;" empty-text="暂无项目数据">
|
||||||
|
<el-table-column prop="city" label="城市" min-width="120" />
|
||||||
|
<el-table-column label="板块" min-width="140">
|
||||||
|
<template #default="{ row }">
|
||||||
|
{{ row.section || row.type || '-' }}
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column prop="projectName" label="项目名称" min-width="280" show-overflow-tooltip />
|
||||||
|
<el-table-column label="金额" min-width="150">
|
||||||
|
<template #default="{ row }">
|
||||||
|
{{ formatYuan(row.amountYuan) }}
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column prop="date" label="发布日期" min-width="140" />
|
||||||
|
<el-table-column label="详情" min-width="120" fixed="right">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<el-link v-if="row.detailLink" :href="row.detailLink" type="primary" target="_blank">查看详情</el-link>
|
||||||
|
<span v-else>-</span>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
</el-table>
|
||||||
|
|
||||||
|
<div class="pagination-wrap">
|
||||||
|
<el-pagination
|
||||||
|
v-if="state.total"
|
||||||
|
background
|
||||||
|
layout="total, prev, pager, next"
|
||||||
|
:current-page="state.page"
|
||||||
|
:page-size="state.pageSize"
|
||||||
|
:total="state.total"
|
||||||
|
@current-change="changePage"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</el-card>
|
||||||
|
</section>
|
||||||
|
</template>
|
||||||
200
client/src/pages/ResultsPage.vue
Normal file
200
client/src/pages/ResultsPage.vue
Normal file
@@ -0,0 +1,200 @@
|
|||||||
|
<script setup>
|
||||||
|
import { computed, onMounted, reactive } from 'vue';
|
||||||
|
import { ElMessage, ElMessageBox } from 'element-plus';
|
||||||
|
import ResultRecordCard from '@/components/ResultRecordCard.vue';
|
||||||
|
import { deleteResult, fetchResultFilters, fetchResults } from '@/api/results';
|
||||||
|
|
||||||
|
const filters = reactive({
|
||||||
|
city: '',
|
||||||
|
section: '',
|
||||||
|
type: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
const options = reactive({
|
||||||
|
cities: [],
|
||||||
|
sections: [],
|
||||||
|
types: [],
|
||||||
|
});
|
||||||
|
|
||||||
|
const stats = reactive({
|
||||||
|
total: 0,
|
||||||
|
success: 0,
|
||||||
|
failed: 0,
|
||||||
|
cities: 0,
|
||||||
|
});
|
||||||
|
|
||||||
|
const state = reactive({
|
||||||
|
records: [],
|
||||||
|
page: 1,
|
||||||
|
pageSize: 10,
|
||||||
|
total: 0,
|
||||||
|
loading: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const hasRecords = computed(() => state.records.length > 0);
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
await refreshAll();
|
||||||
|
});
|
||||||
|
|
||||||
|
async function loadFilters() {
|
||||||
|
const data = await fetchResultFilters();
|
||||||
|
options.cities = data.cities || [];
|
||||||
|
options.sections = data.sections || [];
|
||||||
|
options.types = data.types || [];
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadStats() {
|
||||||
|
const payload = await fetchResults({ page: 1, pageSize: 500 });
|
||||||
|
const items = payload.data || [];
|
||||||
|
stats.total = payload.total || 0;
|
||||||
|
stats.success = items.filter((item) => !item.error).length;
|
||||||
|
stats.failed = items.filter((item) => item.error).length;
|
||||||
|
stats.cities = new Set(items.map((item) => item.city).filter(Boolean)).size;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadRecords() {
|
||||||
|
state.loading = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const payload = await fetchResults({
|
||||||
|
page: state.page,
|
||||||
|
pageSize: state.pageSize,
|
||||||
|
city: filters.city || undefined,
|
||||||
|
section: filters.section || undefined,
|
||||||
|
type: filters.type || undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
state.records = payload.data || [];
|
||||||
|
state.total = payload.total || 0;
|
||||||
|
state.page = payload.page || 1;
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`结果加载失败：${error.message}`);
|
||||||
|
state.records = [];
|
||||||
|
state.total = 0;
|
||||||
|
} finally {
|
||||||
|
state.loading = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshAll() {
|
||||||
|
try {
|
||||||
|
await Promise.all([loadFilters(), loadStats(), loadRecords()]);
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`页面刷新失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyFilters() {
|
||||||
|
state.page = 1;
|
||||||
|
await loadRecords();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function changePage(page) {
|
||||||
|
state.page = page;
|
||||||
|
await loadRecords();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleDelete(record) {
|
||||||
|
try {
|
||||||
|
await ElMessageBox.confirm('确定要删除这条抓取记录吗？', '删除确认', {
|
||||||
|
type: 'warning',
|
||||||
|
});
|
||||||
|
await deleteResult(record.id);
|
||||||
|
ElMessage.success('抓取记录已删除');
|
||||||
|
await refreshAll();
|
||||||
|
} catch (error) {
|
||||||
|
if (error !== 'cancel' && error !== 'close') {
|
||||||
|
ElMessage.error(`删除失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function resetFilters() {
|
||||||
|
filters.city = '';
|
||||||
|
filters.section = '';
|
||||||
|
filters.type = '';
|
||||||
|
applyFilters();
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<section class="page-grid">
|
||||||
|
<header class="page-head">
|
||||||
|
<div>
|
||||||
|
<h2>抓取结果</h2>
|
||||||
|
<p>结果页已经切到 Element Plus 组件风格，适合继续加更复杂的筛选、导出和批量操作。</p>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-statistic title="总记录数" :value="stats.total" />
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-statistic title="成功条数" :value="stats.success" />
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-statistic title="失败条数" :value="stats.failed" />
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12" :lg="6">
|
||||||
|
<el-statistic title="来源城市" :value="stats.cities" />
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<el-form label-position="top">
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="8">
|
||||||
|
<el-form-item label="城市">
|
||||||
|
<el-select v-model="filters.city" placeholder="全部城市" clearable style="width: 100%;" @change="applyFilters">
|
||||||
|
<el-option v-for="item in options.cities" :key="item" :label="item" :value="item" />
|
||||||
|
</el-select>
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="8">
|
||||||
|
<el-form-item label="板块">
|
||||||
|
<el-select v-model="filters.section" placeholder="全部板块" clearable style="width: 100%;" @change="applyFilters">
|
||||||
|
<el-option v-for="item in options.sections" :key="item" :label="item" :value="item" />
|
||||||
|
</el-select>
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="8">
|
||||||
|
<el-form-item label="类型">
|
||||||
|
<el-select v-model="filters.type" placeholder="全部类型" clearable style="width: 100%;" @change="applyFilters">
|
||||||
|
<el-option v-for="item in options.types" :key="item" :label="item" :value="item" />
|
||||||
|
</el-select>
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
|
||||||
|
<div class="toolbar">
|
||||||
|
<div class="summary-text">当前页展示 {{ state.records.length }} 条记录。</div>
|
||||||
|
<div class="toolbar__group">
|
||||||
|
<el-button @click="resetFilters">重置筛选</el-button>
|
||||||
|
<el-button type="primary" @click="refreshAll">刷新数据</el-button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</el-card>
|
||||||
|
|
||||||
|
<div v-loading="state.loading" class="page-grid">
|
||||||
|
<template v-if="hasRecords">
|
||||||
|
<ResultRecordCard v-for="item in state.records" :key="item.id" :record="item" @delete="handleDelete" />
|
||||||
|
</template>
|
||||||
|
<el-empty v-else description="暂无抓取结果" :image-size="96" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="pagination-wrap">
|
||||||
|
<el-pagination
|
||||||
|
v-if="state.total"
|
||||||
|
background
|
||||||
|
layout="total, prev, pager, next"
|
||||||
|
:current-page="state.page"
|
||||||
|
:page-size="state.pageSize"
|
||||||
|
:total="state.total"
|
||||||
|
@current-change="changePage"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
</template>
|
||||||
228
client/src/pages/SettingsPage.vue
Normal file
228
client/src/pages/SettingsPage.vue
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
<script setup>
|
||||||
|
import { onMounted, reactive, ref } from 'vue';
|
||||||
|
import { ElMessage, ElMessageBox } from 'element-plus';
|
||||||
|
import { fetchConfig, fetchSchedulerStatus, saveConfig, triggerScheduledTask } from '@/api/settings';
|
||||||
|
|
||||||
|
const loading = ref(true);
|
||||||
|
const saveStatus = ref('');
|
||||||
|
const schedulerText = ref('未获取');
|
||||||
|
|
||||||
|
const form = reactive({
|
||||||
|
agent: {
|
||||||
|
baseUrl: '',
|
||||||
|
pollInterval: 3000,
|
||||||
|
timeout: 300000,
|
||||||
|
},
|
||||||
|
scheduler: {
|
||||||
|
enabled: false,
|
||||||
|
cronTime: '0 9 * * *',
|
||||||
|
description: '',
|
||||||
|
},
|
||||||
|
email: {
|
||||||
|
smtpHost: '',
|
||||||
|
smtpPort: 587,
|
||||||
|
smtpUser: '',
|
||||||
|
smtpPass: '',
|
||||||
|
recipients: '',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
await refresh();
|
||||||
|
});
|
||||||
|
|
||||||
|
async function refresh() {
|
||||||
|
loading.value = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const [config, schedulerStatus] = await Promise.all([fetchConfig(), fetchSchedulerStatus()]);
|
||||||
|
|
||||||
|
form.agent.baseUrl = config.agent?.baseUrl || '';
|
||||||
|
form.agent.pollInterval = config.agent?.pollInterval || 3000;
|
||||||
|
form.agent.timeout = config.agent?.timeout || 300000;
|
||||||
|
|
||||||
|
form.scheduler.enabled = config.scheduler?.enabled ?? false;
|
||||||
|
form.scheduler.cronTime = config.scheduler?.cronTime || '0 9 * * *';
|
||||||
|
form.scheduler.description = config.scheduler?.description || '';
|
||||||
|
|
||||||
|
form.email.smtpHost = config.email?.smtpHost || '';
|
||||||
|
form.email.smtpPort = config.email?.smtpPort || 587;
|
||||||
|
form.email.smtpUser = config.email?.smtpUser || '';
|
||||||
|
form.email.smtpPass = config.email?.smtpPass || '';
|
||||||
|
form.email.recipients = config.email?.recipients || '';
|
||||||
|
|
||||||
|
schedulerText.value = schedulerStatus.isRunning
|
||||||
|
? `运行中（启用任务 ${schedulerStatus.enabledTasks || 0} 个）`
|
||||||
|
: `未运行（启用任务 ${schedulerStatus.enabledTasks || 0} 个）`;
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`设置加载失败：${error.message}`);
|
||||||
|
} finally {
|
||||||
|
loading.value = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleSave() {
|
||||||
|
try {
|
||||||
|
await saveConfig({
|
||||||
|
agent: {
|
||||||
|
baseUrl: form.agent.baseUrl.trim(),
|
||||||
|
pollInterval: Number(form.agent.pollInterval) || 3000,
|
||||||
|
timeout: Number(form.agent.timeout) || 300000,
|
||||||
|
},
|
||||||
|
scheduler: {
|
||||||
|
enabled: Boolean(form.scheduler.enabled),
|
||||||
|
cronTime: form.scheduler.cronTime.trim() || '0 9 * * *',
|
||||||
|
description: form.scheduler.description.trim(),
|
||||||
|
},
|
||||||
|
email: {
|
||||||
|
smtpHost: form.email.smtpHost.trim(),
|
||||||
|
smtpPort: Number(form.email.smtpPort) || 587,
|
||||||
|
smtpUser: form.email.smtpUser.trim(),
|
||||||
|
smtpPass: form.email.smtpPass.trim(),
|
||||||
|
recipients: form.email.recipients.trim(),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
saveStatus.value = '设置已保存';
|
||||||
|
ElMessage.success('系统设置已保存');
|
||||||
|
await refresh();
|
||||||
|
setTimeout(() => {
|
||||||
|
saveStatus.value = '';
|
||||||
|
}, 3000);
|
||||||
|
} catch (error) {
|
||||||
|
saveStatus.value = '';
|
||||||
|
ElMessage.error(`保存失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleTriggerScheduler() {
|
||||||
|
try {
|
||||||
|
await ElMessageBox.confirm('确定要立即执行一次定时任务吗？', '执行确认', {
|
||||||
|
type: 'warning',
|
||||||
|
});
|
||||||
|
const payload = await triggerScheduledTask();
|
||||||
|
ElMessage.success(payload.message || '定时任务已在后台触发');
|
||||||
|
await refresh();
|
||||||
|
} catch (error) {
|
||||||
|
if (error !== 'cancel' && error !== 'close') {
|
||||||
|
ElMessage.error(`触发失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<section class="page-grid">
|
||||||
|
<header class="page-head">
|
||||||
|
<div>
|
||||||
|
<h2>系统设置</h2>
|
||||||
|
<p>这一页已经改用 Element Plus 表单和卡片来管理 Agent、定时任务和邮件配置。</p>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<div v-loading="loading" class="page-grid">
|
||||||
|
<el-card shadow="never">
|
||||||
|
<template #header>
|
||||||
|
<span>Agent 服务配置</span>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<el-form label-position="top">
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="24">
|
||||||
|
<el-form-item label="Agent 服务地址">
|
||||||
|
<el-input v-model="form.agent.baseUrl" placeholder="http://127.0.0.1:18625" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="轮询间隔（毫秒）">
|
||||||
|
<el-input-number v-model="form.agent.pollInterval" :min="1000" :step="100" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="超时时间（毫秒）">
|
||||||
|
<el-input-number v-model="form.agent.timeout" :min="1000" :step="1000" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
</el-card>
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<template #header>
|
||||||
|
<div class="toolbar">
|
||||||
|
<span>定时任务</span>
|
||||||
|
<el-tag type="info">{{ schedulerText }}</el-tag>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<el-form label-position="top">
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="8">
|
||||||
|
<el-form-item label="是否启用">
|
||||||
|
<el-switch v-model="form.scheduler.enabled" inline-prompt active-text="启用" inactive-text="禁用" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="8">
|
||||||
|
<el-form-item label="Cron 表达式">
|
||||||
|
<el-input v-model="form.scheduler.cronTime" placeholder="0 9 * * *" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="8">
|
||||||
|
<el-form-item label="描述">
|
||||||
|
<el-input v-model="form.scheduler.description" placeholder="每天 9 点执行" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
|
||||||
|
<el-button @click="handleTriggerScheduler">立即执行一次</el-button>
|
||||||
|
</el-card>
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<template #header>
|
||||||
|
<span>邮件配置</span>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<el-form label-position="top">
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="SMTP 服务地址">
|
||||||
|
<el-input v-model="form.email.smtpHost" placeholder="smtp.qq.com" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="端口">
|
||||||
|
<el-input-number v-model="form.email.smtpPort" :min="1" style="width: 100%;" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="用户名">
|
||||||
|
<el-input v-model="form.email.smtpUser" placeholder="your-email@example.com" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="密码或授权码">
|
||||||
|
<el-input v-model="form.email.smtpPass" type="password" show-password placeholder="留空会沿用已保存的掩码值" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="24">
|
||||||
|
<el-form-item label="收件人">
|
||||||
|
<el-input v-model="form.email.recipients" placeholder="a@example.com, b@example.com" />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
</el-card>
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<div class="toolbar">
|
||||||
|
<div class="summary-text">保存后会自动刷新定时任务配置。</div>
|
||||||
|
<div class="toolbar__group">
|
||||||
|
<span v-if="saveStatus" class="summary-text" style="color: var(--el-color-success);">{{ saveStatus }}</span>
|
||||||
|
<el-button type="primary" @click="handleSave">保存全部设置</el-button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</el-card>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
</template>
|
||||||
347
client/src/pages/TasksPage.vue
Normal file
347
client/src/pages/TasksPage.vue
Normal file
@@ -0,0 +1,347 @@
|
|||||||
|
<script setup>
|
||||||
|
import { computed, onBeforeUnmount, onMounted, reactive, ref, watch } from 'vue';
|
||||||
|
import { ElMessage, ElMessageBox } from 'element-plus';
|
||||||
|
import TaskDialog from '@/components/TaskDialog.vue';
|
||||||
|
import {
|
||||||
|
createTask,
|
||||||
|
deleteTask,
|
||||||
|
fetchTasks,
|
||||||
|
fetchTaskStatus,
|
||||||
|
runAllTasks,
|
||||||
|
runTask,
|
||||||
|
toggleTask,
|
||||||
|
updateTask,
|
||||||
|
} from '@/api/tasks';
|
||||||
|
import { formatElapsed } from '@/utils/format';
|
||||||
|
|
||||||
|
const tasks = ref([]);
|
||||||
|
const isLoading = ref(true);
|
||||||
|
const dialogOpen = ref(false);
|
||||||
|
const editingTask = ref(null);
|
||||||
|
|
||||||
|
const filters = reactive({
|
||||||
|
city: '',
|
||||||
|
plate: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
const pagination = reactive({
|
||||||
|
page: 1,
|
||||||
|
pageSize: 10,
|
||||||
|
});
|
||||||
|
|
||||||
|
const runner = reactive({
|
||||||
|
timer: null,
|
||||||
|
polling: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
const status = reactive({
|
||||||
|
visible: false,
|
||||||
|
type: 'info',
|
||||||
|
message: '',
|
||||||
|
});
|
||||||
|
|
||||||
|
const filteredTasks = computed(() => {
|
||||||
|
const cityKeyword = filters.city.trim().toLowerCase();
|
||||||
|
const plateKeyword = filters.plate.trim().toLowerCase();
|
||||||
|
|
||||||
|
return tasks.value.filter((task) => {
|
||||||
|
const city = (task.city || '').toLowerCase();
|
||||||
|
const plate = (task.plateName || '').toLowerCase();
|
||||||
|
return (!cityKeyword || city.includes(cityKeyword)) && (!plateKeyword || plate.includes(plateKeyword));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
const enabledCount = computed(() => tasks.value.filter((item) => item.enabled).length);
|
||||||
|
const pagedTasks = computed(() => {
|
||||||
|
const start = (pagination.page - 1) * pagination.pageSize;
|
||||||
|
return filteredTasks.value.slice(start, start + pagination.pageSize);
|
||||||
|
});
|
||||||
|
|
||||||
|
const summaryText = computed(
|
||||||
|
() => `共 ${tasks.value.length} 个任务，已启用 ${enabledCount.value} 个，筛选后 ${filteredTasks.value.length} 个`,
|
||||||
|
);
|
||||||
|
|
||||||
|
watch(
|
||||||
|
() => [filters.city, filters.plate],
|
||||||
|
() => {
|
||||||
|
pagination.page = 1;
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
onMounted(async () => {
|
||||||
|
await loadTasks();
|
||||||
|
await restoreRunningStatus();
|
||||||
|
});
|
||||||
|
|
||||||
|
onBeforeUnmount(() => {
|
||||||
|
stopPolling();
|
||||||
|
});
|
||||||
|
|
||||||
|
async function loadTasks() {
|
||||||
|
isLoading.value = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
tasks.value = await fetchTasks();
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`任务加载失败：${error.message}`);
|
||||||
|
} finally {
|
||||||
|
isLoading.value = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function openCreateDialog() {
|
||||||
|
editingTask.value = null;
|
||||||
|
dialogOpen.value = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
function openEditDialog(task) {
|
||||||
|
editingTask.value = task;
|
||||||
|
dialogOpen.value = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateStatus(type, message) {
|
||||||
|
status.visible = true;
|
||||||
|
status.type = type;
|
||||||
|
status.message = message;
|
||||||
|
}
|
||||||
|
|
||||||
|
function stopPolling() {
|
||||||
|
if (runner.timer) {
|
||||||
|
clearInterval(runner.timer);
|
||||||
|
runner.timer = null;
|
||||||
|
}
|
||||||
|
runner.polling = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyRunStatus(data) {
|
||||||
|
if (!data?.isRunning && !data?.finished) {
|
||||||
|
stopPolling();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (data.finished) {
|
||||||
|
stopPolling();
|
||||||
|
|
||||||
|
if (data.error) {
|
||||||
|
updateStatus('error', `运行失败：${data.error}`);
|
||||||
|
ElMessage.error(`任务执行失败：${data.error}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const results = data.results || [];
|
||||||
|
if (results.length <= 1) {
|
||||||
|
const total = results[0]?.data?.total ?? 0;
|
||||||
|
updateStatus('success', `运行完成，抓取到 ${total} 条结果。`);
|
||||||
|
ElMessage.success(`任务完成，抓取到 ${total} 条结果`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const successCount = results.filter((item) => !item.error).length;
|
||||||
|
const failCount = results.filter((item) => item.error).length;
|
||||||
|
updateStatus('success', `批量运行完成，成功 ${successCount} 个，失败 ${failCount} 个。`);
|
||||||
|
ElMessage.success(`批量运行完成，成功 ${successCount} 个`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
updateStatus(
|
||||||
|
'info',
|
||||||
|
`正在执行：${data.city || '任务'}（${data.current || 0}/${data.total || 0}），已用时 ${formatElapsed(data.elapsed)}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function pollStatus() {
|
||||||
|
try {
|
||||||
|
const data = await fetchTaskStatus();
|
||||||
|
applyRunStatus(data);
|
||||||
|
if (data.finished) {
|
||||||
|
await loadTasks();
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
stopPolling();
|
||||||
|
updateStatus('error', `状态轮询失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function startPolling() {
|
||||||
|
stopPolling();
|
||||||
|
runner.polling = true;
|
||||||
|
runner.timer = setInterval(pollStatus, 2000);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function restoreRunningStatus() {
|
||||||
|
try {
|
||||||
|
const data = await fetchTaskStatus();
|
||||||
|
applyRunStatus(data);
|
||||||
|
if (data.isRunning) {
|
||||||
|
startPolling();
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`任务状态恢复失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function submitTask(payload) {
|
||||||
|
try {
|
||||||
|
if (editingTask.value?.id) {
|
||||||
|
await updateTask(editingTask.value.id, payload);
|
||||||
|
ElMessage.success('任务已更新');
|
||||||
|
} else {
|
||||||
|
await createTask(payload);
|
||||||
|
ElMessage.success('任务已创建');
|
||||||
|
}
|
||||||
|
|
||||||
|
dialogOpen.value = false;
|
||||||
|
await loadTasks();
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`保存失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleDelete(task) {
|
||||||
|
try {
|
||||||
|
await ElMessageBox.confirm(`确定要删除“${task.city || '未命名任务'}”吗？`, '删除确认', {
|
||||||
|
type: 'warning',
|
||||||
|
});
|
||||||
|
await deleteTask(task.id);
|
||||||
|
ElMessage.success('任务已删除');
|
||||||
|
await loadTasks();
|
||||||
|
} catch (error) {
|
||||||
|
if (error !== 'cancel' && error !== 'close') {
|
||||||
|
ElMessage.error(`删除失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleToggle(task, enabled) {
|
||||||
|
try {
|
||||||
|
await toggleTask(task.id, enabled);
|
||||||
|
ElMessage.success(enabled ? '任务已启用' : '任务已禁用');
|
||||||
|
await loadTasks();
|
||||||
|
} catch (error) {
|
||||||
|
ElMessage.error(`状态更新失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleRun(task) {
|
||||||
|
if (runner.polling) {
|
||||||
|
ElMessage.info('当前已有任务在运行，请稍后再试');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await runTask(task.id);
|
||||||
|
updateStatus('info', `已开始运行任务：${task.city}`);
|
||||||
|
ElMessage.success(`已开始运行：${task.city}`);
|
||||||
|
startPolling();
|
||||||
|
} catch (error) {
|
||||||
|
updateStatus('error', `启动失败：${error.message}`);
|
||||||
|
ElMessage.error(`启动失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleRunAll() {
|
||||||
|
if (!enabledCount.value) {
|
||||||
|
ElMessage.info('没有已启用的任务可以运行');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await ElMessageBox.confirm(`确定要运行全部 ${enabledCount.value} 个已启用任务吗？`, '批量执行确认', {
|
||||||
|
type: 'warning',
|
||||||
|
});
|
||||||
|
await runAllTasks();
|
||||||
|
updateStatus('info', '已开始运行全部启用任务，请稍候…');
|
||||||
|
ElMessage.success('已开始运行全部启用任务');
|
||||||
|
startPolling();
|
||||||
|
} catch (error) {
|
||||||
|
if (error !== 'cancel' && error !== 'close') {
|
||||||
|
updateStatus('error', `批量启动失败：${error.message}`);
|
||||||
|
ElMessage.error(`批量启动失败：${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<template>
|
||||||
|
<section class="page-grid">
|
||||||
|
<header class="page-head">
|
||||||
|
<div>
|
||||||
|
<h2>任务配置</h2>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<el-card shadow="never">
|
||||||
|
<div class="toolbar">
|
||||||
|
<div class="toolbar__group">
|
||||||
|
<el-button type="primary" @click="openCreateDialog">新增任务</el-button>
|
||||||
|
<el-button type="success" :disabled="runner.polling" @click="handleRunAll">运行全部启用</el-button>
|
||||||
|
</div>
|
||||||
|
<div class="summary-text">{{ summaryText }}</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<el-form label-position="top" class="filter-form">
|
||||||
|
<el-row :gutter="16">
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="搜索城市">
|
||||||
|
<el-input v-model="filters.city" placeholder="输入城市关键字" clearable />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
<el-col :xs="24" :sm="12">
|
||||||
|
<el-form-item label="搜索板块">
|
||||||
|
<el-input v-model="filters.plate" placeholder="输入板块关键字" clearable />
|
||||||
|
</el-form-item>
|
||||||
|
</el-col>
|
||||||
|
</el-row>
|
||||||
|
</el-form>
|
||||||
|
|
||||||
|
<el-alert v-if="status.visible" :title="status.message" :type="status.type" :closable="false" show-icon />
|
||||||
|
|
||||||
|
<el-table v-loading="isLoading" :data="pagedTasks" style="width: 100%; margin-top: 18px;" empty-text="没有匹配的任务">
|
||||||
|
<el-table-column prop="city" label="城市" min-width="120" />
|
||||||
|
<el-table-column prop="plateName" label="板块" min-width="120" />
|
||||||
|
<el-table-column label="提示词" min-width="280">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<div class="table-prompt">{{ row.prompt || '-' }}</div>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column label="模型" min-width="160">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<el-tag type="info">{{ row.mode || 'qwen3.5-plus' }}</el-tag>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column label="状态" width="100">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<el-tag :type="row.enabled ? 'success' : 'danger'">{{ row.enabled ? '启用' : '禁用' }}</el-tag>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column label="操作" min-width="260" fixed="right">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<div class="table-actions">
|
||||||
|
<el-button size="small" @click="openEditDialog(row)">编辑</el-button>
|
||||||
|
<el-button size="small" type="primary" :disabled="runner.polling" @click="handleRun(row)">运行</el-button>
|
||||||
|
<el-button size="small" plain @click="handleToggle(row, !row.enabled)">{{ row.enabled ? '禁用' : '启用' }}</el-button>
|
||||||
|
<el-button size="small" type="danger" plain @click="handleDelete(row)">删除</el-button>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
</el-table>
|
||||||
|
|
||||||
|
<div class="pagination-wrap">
|
||||||
|
<el-pagination
|
||||||
|
v-if="filteredTasks.length"
|
||||||
|
background
|
||||||
|
layout="total, prev, pager, next"
|
||||||
|
:current-page="pagination.page"
|
||||||
|
:page-size="pagination.pageSize"
|
||||||
|
:total="filteredTasks.length"
|
||||||
|
@current-change="pagination.page = $event"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</el-card>
|
||||||
|
|
||||||
|
<TaskDialog v-model="dialogOpen" :task="editingTask" @submit="submitTask" />
|
||||||
|
</section>
|
||||||
|
</template>
|
||||||
56
client/src/router/index.js
Normal file
56
client/src/router/index.js
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
import { createRouter, createWebHistory } from 'vue-router';
|
||||||
|
|
||||||
|
const routes = [
|
||||||
|
{
|
||||||
|
path: '/',
|
||||||
|
redirect: '/tasks',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: '/tasks',
|
||||||
|
name: 'tasks',
|
||||||
|
component: () => import('@/pages/TasksPage.vue'),
|
||||||
|
meta: {
|
||||||
|
title: '任务配置',
|
||||||
|
subtitle: '管理城市任务、切换模型，并手动触发抓取流程。',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: '/results',
|
||||||
|
name: 'results',
|
||||||
|
component: () => import('@/pages/ResultsPage.vue'),
|
||||||
|
meta: {
|
||||||
|
title: '抓取结果',
|
||||||
|
subtitle: '查看抓取记录、按条件筛选，并快速定位失败任务。',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: '/projects',
|
||||||
|
name: 'projects',
|
||||||
|
component: () => import('@/pages/ProjectsPage.vue'),
|
||||||
|
meta: {
|
||||||
|
title: '项目管理',
|
||||||
|
subtitle: '对抓取出的项目去重、筛选与金额范围查询。',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
path: '/settings',
|
||||||
|
name: 'settings',
|
||||||
|
component: () => import('@/pages/SettingsPage.vue'),
|
||||||
|
meta: {
|
||||||
|
title: '系统设置',
|
||||||
|
subtitle: '维护 Agent、定时任务与邮件推送配置。',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const router = createRouter({
|
||||||
|
history: createWebHistory(),
|
||||||
|
routes,
|
||||||
|
});
|
||||||
|
|
||||||
|
router.afterEach((to) => {
|
||||||
|
const title = to.meta?.title ? `${to.meta.title} | 公告抓取与分析工具` : '公告抓取与分析工具';
|
||||||
|
document.title = title;
|
||||||
|
});
|
||||||
|
|
||||||
|
export default router;
|
||||||
383
client/src/styles.css
Normal file
383
client/src/styles.css
Normal file
@@ -0,0 +1,383 @@
|
|||||||
|
@import url('https://fonts.googleapis.com/css2?family=Plus+Jakarta+Sans:wght@400;500;600;700;800&family=Noto+Sans+SC:wght@400;500;700&family=IBM+Plex+Mono:wght@500&display=swap');
|
||||||
|
|
||||||
|
:root {
|
||||||
|
--bg-top: #edf7ff;
|
||||||
|
--bg-mid: #f8fcf8;
|
||||||
|
--bg-bottom: #fff8ef;
|
||||||
|
--line: rgba(15, 34, 52, 0.1);
|
||||||
|
--text: #11263a;
|
||||||
|
--muted: #60758a;
|
||||||
|
--shadow-lg: 0 24px 54px rgba(11, 42, 68, 0.14);
|
||||||
|
--shadow-md: 0 12px 28px rgba(11, 42, 68, 0.08);
|
||||||
|
--radius-lg: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
html,
|
||||||
|
body,
|
||||||
|
#app {
|
||||||
|
min-height: 100%;
|
||||||
|
}
|
||||||
|
|
||||||
|
* {
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
margin: 0;
|
||||||
|
font-family: 'Plus Jakarta Sans', 'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||||
|
color: var(--text);
|
||||||
|
background:
|
||||||
|
radial-gradient(880px 520px at -8% -12%, rgba(13, 108, 184, 0.2), transparent 58%),
|
||||||
|
radial-gradient(760px 440px at 110% -10%, rgba(15, 157, 140, 0.18), transparent 52%),
|
||||||
|
radial-gradient(900px 520px at 60% 118%, rgba(215, 139, 30, 0.12), transparent 56%),
|
||||||
|
linear-gradient(160deg, var(--bg-top) 0%, var(--bg-mid) 52%, var(--bg-bottom) 100%);
|
||||||
|
--el-color-primary: #0d6cb8;
|
||||||
|
--el-border-radius-base: 12px;
|
||||||
|
--el-font-family: 'Plus Jakarta Sans', 'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||||
|
}
|
||||||
|
|
||||||
|
a {
|
||||||
|
text-decoration: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-shell {
|
||||||
|
min-height: 100vh;
|
||||||
|
padding: 24px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-frame {
|
||||||
|
max-width: 1480px;
|
||||||
|
margin: 0 auto;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 28px;
|
||||||
|
overflow: hidden;
|
||||||
|
background: rgba(255, 255, 255, 0.68);
|
||||||
|
box-shadow: var(--shadow-lg);
|
||||||
|
backdrop-filter: blur(22px);
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-topbar {
|
||||||
|
padding: 24px 28px;
|
||||||
|
color: #eff8ff;
|
||||||
|
background:
|
||||||
|
radial-gradient(circle at top right, rgba(255, 255, 255, 0.2), transparent 26%),
|
||||||
|
linear-gradient(125deg, #0c639f 0%, #0f85a2 55%, #17a86b 100%);
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-brand {
|
||||||
|
display: flex;
|
||||||
|
gap: 16px;
|
||||||
|
align-items: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-brand__mark {
|
||||||
|
width: 54px;
|
||||||
|
height: 54px;
|
||||||
|
border-radius: 16px;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
color: #0d4c6e;
|
||||||
|
font-size: 24px;
|
||||||
|
font-weight: 800;
|
||||||
|
background: linear-gradient(145deg, rgba(255, 255, 255, 0.96), rgba(232, 248, 255, 0.84));
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-brand__text h1 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 26px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-brand__text p {
|
||||||
|
margin: 6px 0 0;
|
||||||
|
font-size: 13px;
|
||||||
|
opacity: 0.92;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-menu.el-menu {
|
||||||
|
padding: 10px 10px 0;
|
||||||
|
border-bottom: 1px solid var(--line);
|
||||||
|
background: rgba(255, 255, 255, 0.26);
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-menu.el-menu--horizontal > .el-menu-item {
|
||||||
|
height: 48px;
|
||||||
|
line-height: 48px;
|
||||||
|
border-bottom: 0;
|
||||||
|
margin: 0 8px 10px 0;
|
||||||
|
border-radius: 14px;
|
||||||
|
color: var(--muted);
|
||||||
|
font-weight: 700;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-menu.el-menu--horizontal > .el-menu-item.is-active {
|
||||||
|
color: var(--el-color-primary);
|
||||||
|
background: rgba(255, 255, 255, 0.9);
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-main {
|
||||||
|
padding: 28px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-grid {
|
||||||
|
display: grid;
|
||||||
|
gap: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-head {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 20px;
|
||||||
|
align-items: flex-end;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-head h2 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 28px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-head p {
|
||||||
|
margin: 8px 0 0;
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.toolbar {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
align-items: center;
|
||||||
|
gap: 12px;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.toolbar__group {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 10px;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-text {
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero-panel.el-card {
|
||||||
|
border: 0;
|
||||||
|
color: #f7fcff;
|
||||||
|
background: linear-gradient(140deg, rgba(13, 108, 184, 0.95), rgba(15, 157, 140, 0.92));
|
||||||
|
box-shadow: var(--shadow-md);
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero-panel .el-card__body {
|
||||||
|
position: relative;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero-panel .el-card__body::after {
|
||||||
|
content: '';
|
||||||
|
position: absolute;
|
||||||
|
right: -70px;
|
||||||
|
top: -80px;
|
||||||
|
width: 240px;
|
||||||
|
height: 240px;
|
||||||
|
border-radius: 50%;
|
||||||
|
background: rgba(255, 255, 255, 0.14);
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero-panel h3,
|
||||||
|
.hero-panel p {
|
||||||
|
position: relative;
|
||||||
|
z-index: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero-panel h3 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero-panel p {
|
||||||
|
margin: 8px 0 0;
|
||||||
|
max-width: 760px;
|
||||||
|
opacity: 0.92;
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-form {
|
||||||
|
margin-top: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.table-prompt {
|
||||||
|
display: -webkit-box;
|
||||||
|
overflow: hidden;
|
||||||
|
-webkit-box-orient: vertical;
|
||||||
|
-webkit-line-clamp: 2;
|
||||||
|
line-height: 1.6;
|
||||||
|
}
|
||||||
|
|
||||||
|
.table-actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 8px;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.pagination-wrap {
|
||||||
|
display: flex;
|
||||||
|
justify-content: flex-end;
|
||||||
|
margin-top: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.dialog-footer {
|
||||||
|
display: flex;
|
||||||
|
justify-content: flex-end;
|
||||||
|
gap: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus {
|
||||||
|
border-radius: var(--radius-lg);
|
||||||
|
box-shadow: var(--shadow-md);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__header {
|
||||||
|
display: flex;
|
||||||
|
align-items: flex-start;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__title {
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: 700;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__meta {
|
||||||
|
display: flex;
|
||||||
|
gap: 8px;
|
||||||
|
margin-top: 8px;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__time {
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 13px;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__list {
|
||||||
|
display: grid;
|
||||||
|
gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__item {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 110px 1.6fr 150px 120px auto;
|
||||||
|
gap: 12px;
|
||||||
|
align-items: center;
|
||||||
|
padding-bottom: 12px;
|
||||||
|
border-bottom: 1px solid rgba(15, 34, 52, 0.08);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__item:last-child {
|
||||||
|
border-bottom: 0;
|
||||||
|
padding-bottom: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__type {
|
||||||
|
color: var(--el-color-primary);
|
||||||
|
font-size: 13px;
|
||||||
|
font-weight: 800;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__name {
|
||||||
|
min-width: 0;
|
||||||
|
word-break: break-word;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__amount {
|
||||||
|
color: #99591a;
|
||||||
|
font-weight: 700;
|
||||||
|
text-align: right;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__date {
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 13px;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__action {
|
||||||
|
text-align: right;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__footer {
|
||||||
|
display: flex;
|
||||||
|
justify-content: flex-end;
|
||||||
|
}
|
||||||
|
|
||||||
|
.el-card {
|
||||||
|
--el-card-border-color: rgba(15, 34, 52, 0.08);
|
||||||
|
border-radius: var(--radius-lg);
|
||||||
|
box-shadow: var(--shadow-md);
|
||||||
|
}
|
||||||
|
|
||||||
|
.el-statistic {
|
||||||
|
padding: 18px;
|
||||||
|
border-radius: var(--radius-lg);
|
||||||
|
background: rgba(255, 255, 255, 0.76);
|
||||||
|
border: 1px solid rgba(15, 34, 52, 0.08);
|
||||||
|
box-shadow: var(--shadow-md);
|
||||||
|
}
|
||||||
|
|
||||||
|
.el-statistic__head {
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.el-table {
|
||||||
|
margin-top: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.el-table,
|
||||||
|
.el-table tr,
|
||||||
|
.el-table th.el-table__cell,
|
||||||
|
.el-table td.el-table__cell {
|
||||||
|
background: transparent;
|
||||||
|
}
|
||||||
|
|
||||||
|
.el-empty {
|
||||||
|
padding: 36px 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 1120px) {
|
||||||
|
.result-card-plus__item {
|
||||||
|
grid-template-columns: 90px 1fr;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-plus__amount,
|
||||||
|
.result-card-plus__date,
|
||||||
|
.result-card-plus__action {
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 840px) {
|
||||||
|
.app-shell {
|
||||||
|
padding: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-main,
|
||||||
|
.app-topbar {
|
||||||
|
padding: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-head {
|
||||||
|
align-items: flex-start;
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-menu.el-menu--horizontal {
|
||||||
|
display: block;
|
||||||
|
height: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-menu.el-menu--horizontal > .el-menu-item {
|
||||||
|
display: block;
|
||||||
|
}
|
||||||
|
}
|
||||||
26
client/src/utils/format.js
Normal file
26
client/src/utils/format.js
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
export function formatDateTime(value) {
|
||||||
|
if (!value) return '-';
|
||||||
|
const date = new Date(value);
|
||||||
|
if (Number.isNaN(date.getTime())) return value;
|
||||||
|
return date.toLocaleString('zh-CN');
|
||||||
|
}
|
||||||
|
|
||||||
|
export function formatYuan(value) {
|
||||||
|
if (typeof value !== 'number' || !Number.isFinite(value)) return '-';
|
||||||
|
return `${value.toLocaleString('zh-CN', { maximumFractionDigits: 2 })} 元`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function formatElapsed(totalSeconds) {
|
||||||
|
const seconds = Number(totalSeconds) || 0;
|
||||||
|
const minutes = Math.floor(seconds / 60);
|
||||||
|
const remainder = seconds % 60;
|
||||||
|
return minutes > 0 ? `${minutes} 分 ${remainder} 秒` : `${remainder} 秒`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function pickResultName(item = {}) {
|
||||||
|
return item.project_name || item.projectName || item.title || item.name || item.bidName || '-';
|
||||||
|
}
|
||||||
|
|
||||||
|
export function pickResultLink(item = {}) {
|
||||||
|
return item.detail_link || item.target_link || item.url || item.href || '';
|
||||||
|
}
|
||||||
898
config.json
898
config.json
@@ -1,897 +1,21 @@
|
|||||||
{
|
{
|
||||||
|
"agent": {
|
||||||
|
"baseUrl": "http://192.168.3.65:18777",
|
||||||
|
"pollInterval": 3000,
|
||||||
|
"timeout": 3600000
|
||||||
|
},
|
||||||
"scheduler": {
|
"scheduler": {
|
||||||
"enabled": false,
|
"enabled": true,
|
||||||
"cronTime": "0 9 * * *",
|
"cronTime": "0 9 * * *",
|
||||||
"threshold": 0,
|
"threshold": 100000,
|
||||||
"description": "每天9点采集当日项目"
|
"description": "每天9点采集当日项目",
|
||||||
|
"timeRange": "thisMonth"
|
||||||
},
|
},
|
||||||
"email": {
|
"email": {
|
||||||
"smtpHost": "smtp.qq.com",
|
"smtpHost": "smtp.qq.com",
|
||||||
"smtpPort": 587,
|
"smtpPort": 587,
|
||||||
"smtpUser": "1076597680@qq.com",
|
"smtpUser": "1076597680@qq.com",
|
||||||
"smtpPass": "nfrjdiraqddsjeeh",
|
"smtpPass": "nfrjdiraqddsjeeh",
|
||||||
"recipients": "5482498@qq.com"
|
"recipients": "1650243281@qq.com"
|
||||||
},
|
}
|
||||||
"scrapers": [
|
|
||||||
{
|
|
||||||
"id": "scraper-1772762494299",
|
|
||||||
"city": "南京市",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/buildService1.html",
|
|
||||||
"section": "房建市政",
|
|
||||||
"subsection": "工程类、服务类",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今天的招标公告信息，包括:标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期(YYYY-MM-DD格式)、详情页完整URL，如果没有该日期数据，直接忽略并输出结果",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958569164",
|
|
||||||
"city": "南京市",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/jtsw/traffic.html",
|
|
||||||
"section": "交通水务",
|
|
||||||
"subsection": "交通",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958590218",
|
|
||||||
"city": "南京市",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/jtsw/069005/traffic5.html",
|
|
||||||
"section": "交通水务",
|
|
||||||
"subsection": "水务",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958710168",
|
|
||||||
"city": "无锡市",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/jsgc/index.shtml",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "工程类、非工程类",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958756969",
|
|
||||||
"city": "无锡市",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上2026-03-05招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958789571",
|
|
||||||
"city": "无锡市",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/jtgc/index.shtml",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958889688",
|
|
||||||
"city": "徐州市",
|
|
||||||
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003001/003001014/list.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标文件提前公示",
|
|
||||||
"prompt": "从页面中提取今日的招标文件提前公示信息，具体字段包括：标题、项目金额（包含合同预估价、最高投标限价等所有涉及金额的信息）、发布日期（严格按照 YYYY-MM-DD 格式）、详情页完整 URL。若当前为分页第一页且未检索到任何符合 “今日发布” 条件的公示信息，直接返回 “无数据”，无需执行后续翻页及提取操作；若第一页存在有效数据，则正常提取对应字段信息",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958933270",
|
|
||||||
"city": "徐州市",
|
|
||||||
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003001/003001001/list.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "从页面中提取今日的招标公告/资审公告提前公示信息，具体字段包括：标题、项目金额（包含合同预估价、最高投标限价等所有涉及金额的信息）、发布日期（严格按照 YYYY-MM-DD 格式）、详情页完整 URL。若当前为分页第一页且未检索到任何符合 “今日发布” 条件的公示信息，直接返回 “无数据”，无需执行后续翻页及提取操作；若第一页存在有效数据，则正常提取对应字段信息",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772958989431",
|
|
||||||
"city": "徐州市",
|
|
||||||
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003002/003002005/list.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959084636",
|
|
||||||
"city": "徐州市",
|
|
||||||
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003002/003002001/list.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959127769",
|
|
||||||
"city": "徐州市",
|
|
||||||
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003003/003003005/list.html",
|
|
||||||
"section": "水务工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959152540",
|
|
||||||
"city": "徐州市",
|
|
||||||
"url": "https://ggzy.zwb.xz.gov.cn/jyxx/003003/003003001/list.html",
|
|
||||||
"section": "水务工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959320386",
|
|
||||||
"city": "常州市",
|
|
||||||
"url": "http://ggzy.xzsp.changzhou.gov.cn/jyzx/001001/tradeInfonew.html?category=001001",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959351660",
|
|
||||||
"city": "常州市",
|
|
||||||
"url": "http://ggzy.xzsp.changzhou.gov.cn/jyzx/001001/tradeInfonew.html?category=001001",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959481049",
|
|
||||||
"city": "常州市",
|
|
||||||
"url": "http://ggzy.xzsp.changzhou.gov.cn/jyzx/001001/tradeInfonew.html?category=001001",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959722015",
|
|
||||||
"city": "苏州市",
|
|
||||||
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标预计划",
|
|
||||||
"prompt": "提取页面上今日的招标预计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959755349",
|
|
||||||
"city": "苏州市",
|
|
||||||
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959817179",
|
|
||||||
"city": "苏州市",
|
|
||||||
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标预计划",
|
|
||||||
"prompt": "提取页面上今日的招标预计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959842489",
|
|
||||||
"city": "苏州市",
|
|
||||||
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959877226",
|
|
||||||
"city": "苏州市",
|
|
||||||
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标预计划",
|
|
||||||
"prompt": "提取页面上今日的招标预计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959894660",
|
|
||||||
"city": "苏州市",
|
|
||||||
"url": "http://ggzy.suzhou.gov.cn/jyxx/003001/tradeInfo.html",
|
|
||||||
"section": "水务工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959960153",
|
|
||||||
"city": "南通市",
|
|
||||||
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003001/003001009/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772959988900",
|
|
||||||
"city": "南通市",
|
|
||||||
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003001/003001001/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960042784",
|
|
||||||
"city": "南通市",
|
|
||||||
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003005/003005001/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标公告/招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960066902",
|
|
||||||
"city": "南通市",
|
|
||||||
"url": "https://ggzyjy.nantong.gov.cn/jyxx/003006/003006001/tradeInfo.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标公告/招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960133857",
|
|
||||||
"city": "连云港市",
|
|
||||||
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001001/001001008/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960157269",
|
|
||||||
"city": "连云港市",
|
|
||||||
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001001/001001001/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960194684",
|
|
||||||
"city": "连云港市",
|
|
||||||
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001003/001003004/tradeInfo.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960218045",
|
|
||||||
"city": "连云港市",
|
|
||||||
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001003/001003001/tradeInfo.html",
|
|
||||||
"section": "水务工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960262581",
|
|
||||||
"city": "连云港市",
|
|
||||||
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001002/001002004/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960280984",
|
|
||||||
"city": "连云港市",
|
|
||||||
"url": "https://ggzy.lyg.gov.cn/lygweb/jyxx/001002/001002001/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960353235",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001001/001001009/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960376613",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001001/001001001/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960419655",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001002/001002009/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960532565",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001002/001002001/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960574486",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001003/001003006/tradeInfo.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960597160",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001003/001003001/tradeInfo.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960644255",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001009/001009005/tradeInfo.html",
|
|
||||||
"section": "土地整治",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960674724",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001009/001009001/tradeInfo.html",
|
|
||||||
"section": "土地整治",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960714492",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001010/001010005/tradeInfo.html",
|
|
||||||
"section": "农田建设",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960745082",
|
|
||||||
"city": "淮安市",
|
|
||||||
"url": "https://ggzy.huaian.gov.cn/jyxx/001010/001010001/tradeInfo.html",
|
|
||||||
"section": "农田建设",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960817101",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "工程建设",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772960871290",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "工程建设",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772961673500",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772961722432",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772961991141",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962017043",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962073734",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "农业农村",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962143448",
|
|
||||||
"city": "盐城市",
|
|
||||||
"url": "https://ycggzy.jszwfw.gov.cn/tradeInfor?secondId=19&secondCode=transactionInfo",
|
|
||||||
"section": "农业农村",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962220152",
|
|
||||||
"city": "扬州市",
|
|
||||||
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/fjsz/zbgg/index.html",
|
|
||||||
"section": "房建市政",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962257442",
|
|
||||||
"city": "扬州市",
|
|
||||||
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/jtgc/zbgg/index.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962297502",
|
|
||||||
"city": "扬州市",
|
|
||||||
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/slgc/zbgg/index.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962349287",
|
|
||||||
"city": "扬州市",
|
|
||||||
"url": "https://ggzyjyzx.yangzhou.gov.cn/jyxx/nygc/nyzbgg/index.html",
|
|
||||||
"section": "农业工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "农业招标公告",
|
|
||||||
"prompt": "提取页面上今日的农业招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962408213",
|
|
||||||
"city": "镇江市",
|
|
||||||
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962433571",
|
|
||||||
"city": "镇江市",
|
|
||||||
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962491815",
|
|
||||||
"city": "镇江市",
|
|
||||||
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962512616",
|
|
||||||
"city": "镇江市",
|
|
||||||
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962542402",
|
|
||||||
"city": "镇江市",
|
|
||||||
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划",
|
|
||||||
"prompt": "提取页面上今日的招标计划信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962560473",
|
|
||||||
"city": "镇江市",
|
|
||||||
"url": "https://ggzy.zhenjiang.gov.cn/jyxx/tradeInfonew.html?type=gcjs",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962599558",
|
|
||||||
"city": "泰州市",
|
|
||||||
"url": "http://ggzy.taizhou.gov.cn/jyxx/001001/001001001/secondPagejyxx.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962630223",
|
|
||||||
"city": "泰州市",
|
|
||||||
"url": "http://ggzy.taizhou.gov.cn/jyxx/001013/001013001/secondPagejyxx.html",
|
|
||||||
"section": "能源工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962664564",
|
|
||||||
"city": "泰州市",
|
|
||||||
"url": "http://ggzy.taizhou.gov.cn/jyxx/001002/001002001/secondPagejyxx.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962689903",
|
|
||||||
"city": "泰州市",
|
|
||||||
"url": "http://ggzy.taizhou.gov.cn/jyxx/001003/001003001/secondPagejyxx.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962734224",
|
|
||||||
"city": "泰州市",
|
|
||||||
"url": "http://ggzy.taizhou.gov.cn/jyxx/001012/001012001/secondPagejyxx.html",
|
|
||||||
"section": "农业工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962859477",
|
|
||||||
"city": "宿迁市",
|
|
||||||
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001010/tradeInfo.html",
|
|
||||||
"section": "招标计划提前发布",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划提前发布",
|
|
||||||
"prompt": "提取页面上今日的招标计划提前发布信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962901103",
|
|
||||||
"city": "宿迁市",
|
|
||||||
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001001/001001010/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标预公告",
|
|
||||||
"prompt": "提取页面上今日的招标预公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962919812",
|
|
||||||
"city": "宿迁市",
|
|
||||||
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001001/001001001/tradeInfo.html",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772962967658",
|
|
||||||
"city": "宿迁市",
|
|
||||||
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001002/001002001/tradeInfo.html",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963005494",
|
|
||||||
"city": "宿迁市",
|
|
||||||
"url": "https://ggzy.xzspj.suqian.gov.cn/jyxx/001003/001003001/tradeInfo.html",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963398342",
|
|
||||||
"city": "江苏省公共资源交易平台",
|
|
||||||
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划/招标计划变更公告",
|
|
||||||
"prompt": "提取页面上今日的招标计划/招标计划变更公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963423954",
|
|
||||||
"city": "江苏省公共资源交易平台",
|
|
||||||
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
|
|
||||||
"section": "建设工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告/资审公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告/资审公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963466949",
|
|
||||||
"city": "江苏省公共资源交易平台",
|
|
||||||
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划/招标计划变更公告",
|
|
||||||
"prompt": "提取页面上今日的招标计划/招标计划变更公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963511551",
|
|
||||||
"city": "江苏省公共资源交易平台",
|
|
||||||
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
|
|
||||||
"section": "交通工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963581266",
|
|
||||||
"city": "江苏省公共资源交易平台",
|
|
||||||
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标计划/招标计划变更公告",
|
|
||||||
"prompt": "提取页面上今日的招标计划/招标计划变更公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963597905",
|
|
||||||
"city": "江苏省公共资源交易平台",
|
|
||||||
"url": "http://jsggzy.jszwfw.gov.cn/jyxx/tradeInfonew.html?type=jsgc",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963721156",
|
|
||||||
"city": "江苏省建设工程招标网",
|
|
||||||
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%B7%BF%CE%DD%BD%A8%D6%FE%CA%A9%B9%A4",
|
|
||||||
"section": "房屋建筑施工",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772963750768",
|
|
||||||
"city": "江苏省建设工程招标网",
|
|
||||||
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%CA%D0%D5%FE%B9%A4%B3%CC%CA%A9%B9%A4",
|
|
||||||
"section": "市政工程施工",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772965504453",
|
|
||||||
"city": "江苏省建设工程招标网",
|
|
||||||
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%B5%A5%B6%C0%D7%B0%CA%CE%D7%B0%D0%DE%CA%A9%B9%A4",
|
|
||||||
"section": "单独装饰装修施工",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772965525132",
|
|
||||||
"city": "江苏省建设工程招标网",
|
|
||||||
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%C9%E8%BC%C6",
|
|
||||||
"section": "设计",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "scraper-1772965538927",
|
|
||||||
"city": "江苏省建设工程招标网",
|
|
||||||
"url": "http://www.jszb.com.cn/JSZB/YW_info/ZhaoBiaoGG/MoreInfo_ZBGG.aspx?Type=%CB%AE%C0%FB",
|
|
||||||
"section": "水利",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"prompt": "提取页面上2026年3月3日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "spark-1-mini"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
}
|
||||||
3026
package-lock.json
generated
3026
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
19
package.json
19
package.json
@@ -2,18 +2,35 @@
|
|||||||
"name": "njggzy-scraper",
|
"name": "njggzy-scraper",
|
||||||
"version": "2.0.0",
|
"version": "2.0.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"description": "南京公共资源交易平台 - 合同估算价采集工具",
|
"description": "公告抓取与分析工具",
|
||||||
"main": "src/server.js",
|
"main": "src/server.js",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
"dev": "concurrently \"npm:dev:server\" \"npm:dev:client\"",
|
||||||
|
"dev:server": "node src/server.js",
|
||||||
|
"dev:client": "vite --config vite.config.js",
|
||||||
|
"build": "vite build --config vite.config.js",
|
||||||
|
"preview": "vite preview --config vite.config.js",
|
||||||
"start": "node src/server.js"
|
"start": "node src/server.js"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@mendable/firecrawl-js": "latest",
|
"@mendable/firecrawl-js": "latest",
|
||||||
|
"axios": "^1.11.0",
|
||||||
|
"better-sqlite3": "^12.8.0",
|
||||||
"cors": "^2.8.5",
|
"cors": "^2.8.5",
|
||||||
"dotenv": "^17.2.3",
|
"dotenv": "^17.2.3",
|
||||||
|
"element-plus": "^2.11.4",
|
||||||
"express": "^5.2.1",
|
"express": "^5.2.1",
|
||||||
"node-cron": "^4.2.1",
|
"node-cron": "^4.2.1",
|
||||||
"nodemailer": "^7.0.11",
|
"nodemailer": "^7.0.11",
|
||||||
|
"vue": "^3.5.18",
|
||||||
|
"vue-router": "^4.5.1",
|
||||||
"zod": "^3.24.2"
|
"zod": "^3.24.2"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@vitejs/plugin-vue": "^6.0.1",
|
||||||
|
"concurrently": "^9.2.1",
|
||||||
|
"unplugin-auto-import": "^20.2.0",
|
||||||
|
"unplugin-vue-components": "^29.0.0",
|
||||||
|
"vite": "^7.1.3"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1063,7 +1063,7 @@ async function generateCombinedReport() {
|
|||||||
function displayCombinedReport(winningReport, bidReport, container) {
|
function displayCombinedReport(winningReport, bidReport, container) {
|
||||||
const html = `
|
const html = `
|
||||||
<!-- 中标公示部分 -->
|
<!-- 中标公示部分 -->
|
||||||
<div class="summary" style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);">
|
<div class="summary" style="background: linear-gradient(135deg, #0f6ecd 0%, #0ea5a4 100%);">
|
||||||
<h2>中标公示报告</h2>
|
<h2>中标公示报告</h2>
|
||||||
<div class="stat">
|
<div class="stat">
|
||||||
<div class="stat-label">总项目数</div>
|
<div class="stat-label">总项目数</div>
|
||||||
|
|||||||
2164
public/index.html
2164
public/index.html
File diff suppressed because it is too large
Load Diff
@@ -6,6 +6,8 @@
|
|||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
<title>抓取结果查看 - 公告采集工具</title>
|
<title>抓取结果查看 - 公告采集工具</title>
|
||||||
<style>
|
<style>
|
||||||
|
@import url('https://fonts.googleapis.com/css2?family=Fira+Sans:wght@400;500;600;700&family=Fira+Code:wght@500;600&family=Noto+Sans+SC:wght@400;500;700&display=swap');
|
||||||
|
|
||||||
* {
|
* {
|
||||||
margin: 0;
|
margin: 0;
|
||||||
padding: 0;
|
padding: 0;
|
||||||
@@ -44,7 +46,7 @@
|
|||||||
.topbar-logo {
|
.topbar-logo {
|
||||||
width: 36px;
|
width: 36px;
|
||||||
height: 36px;
|
height: 36px;
|
||||||
background: linear-gradient(135deg, #667eea, #764ba2);
|
background: linear-gradient(135deg, #0f6ecd, #0ea5a4);
|
||||||
border-radius: 8px;
|
border-radius: 8px;
|
||||||
display: flex;
|
display: flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
@@ -79,8 +81,8 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.topbar-link.active {
|
.topbar-link.active {
|
||||||
background: rgba(102, 126, 234, 0.25);
|
background: rgba(15, 110, 205, 0.25);
|
||||||
color: #8fa8f8;
|
color: #5ea2e8;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ===== 主体内容 ===== */
|
/* ===== 主体内容 ===== */
|
||||||
@@ -128,7 +130,7 @@
|
|||||||
|
|
||||||
.stat-card:hover {
|
.stat-card:hover {
|
||||||
background: rgba(255, 255, 255, 0.08);
|
background: rgba(255, 255, 255, 0.08);
|
||||||
border-color: rgba(102, 126, 234, 0.4);
|
border-color: rgba(15, 110, 205, 0.4);
|
||||||
}
|
}
|
||||||
|
|
||||||
.stat-card .label {
|
.stat-card .label {
|
||||||
@@ -192,8 +194,8 @@
|
|||||||
|
|
||||||
.filter-select:focus,
|
.filter-select:focus,
|
||||||
.filter-input:focus {
|
.filter-input:focus {
|
||||||
border-color: #667eea;
|
border-color: #0f6ecd;
|
||||||
box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.15);
|
box-shadow: 0 0 0 3px rgba(15, 110, 205, 0.15);
|
||||||
}
|
}
|
||||||
|
|
||||||
.filter-select option {
|
.filter-select option {
|
||||||
@@ -224,14 +226,14 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.btn-primary {
|
.btn-primary {
|
||||||
background: linear-gradient(135deg, #667eea, #764ba2);
|
background: linear-gradient(135deg, #0f6ecd, #0ea5a4);
|
||||||
color: #fff;
|
color: #fff;
|
||||||
}
|
}
|
||||||
|
|
||||||
.btn-primary:hover {
|
.btn-primary:hover {
|
||||||
opacity: 0.9;
|
opacity: 0.9;
|
||||||
transform: translateY(-1px);
|
transform: translateY(-1px);
|
||||||
box-shadow: 0 4px 14px rgba(102, 126, 234, 0.4);
|
box-shadow: 0 4px 14px rgba(15, 110, 205, 0.4);
|
||||||
}
|
}
|
||||||
|
|
||||||
.btn-danger {
|
.btn-danger {
|
||||||
@@ -286,7 +288,7 @@
|
|||||||
top: 0;
|
top: 0;
|
||||||
bottom: 0;
|
bottom: 0;
|
||||||
width: 4px;
|
width: 4px;
|
||||||
background: linear-gradient(180deg, #667eea, #764ba2);
|
background: linear-gradient(180deg, #0f6ecd, #0ea5a4);
|
||||||
border-radius: 14px 0 0 14px;
|
border-radius: 14px 0 0 14px;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -296,7 +298,7 @@
|
|||||||
|
|
||||||
.result-card:hover {
|
.result-card:hover {
|
||||||
background: rgba(255, 255, 255, 0.07);
|
background: rgba(255, 255, 255, 0.07);
|
||||||
border-color: rgba(102, 126, 234, 0.3);
|
border-color: rgba(15, 110, 205, 0.3);
|
||||||
}
|
}
|
||||||
|
|
||||||
.result-card-header {
|
.result-card-header {
|
||||||
@@ -323,9 +325,9 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.tag-city {
|
.tag-city {
|
||||||
background: rgba(102, 126, 234, 0.2);
|
background: rgba(15, 110, 205, 0.2);
|
||||||
color: #8fa8f8;
|
color: #5ea2e8;
|
||||||
border: 1px solid rgba(102, 126, 234, 0.3);
|
border: 1px solid rgba(15, 110, 205, 0.3);
|
||||||
}
|
}
|
||||||
|
|
||||||
.tag-section {
|
.tag-section {
|
||||||
@@ -353,7 +355,7 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.result-url a {
|
.result-url a {
|
||||||
color: #6fa3ff;
|
color: #3c88d6;
|
||||||
text-decoration: none;
|
text-decoration: none;
|
||||||
font-size: 13px;
|
font-size: 13px;
|
||||||
word-break: break-all;
|
word-break: break-all;
|
||||||
@@ -437,8 +439,8 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.data-table th {
|
.data-table th {
|
||||||
background: rgba(102, 126, 234, 0.15);
|
background: rgba(15, 110, 205, 0.15);
|
||||||
color: #8fa8f8;
|
color: #5ea2e8;
|
||||||
font-weight: 600;
|
font-weight: 600;
|
||||||
white-space: nowrap;
|
white-space: nowrap;
|
||||||
}
|
}
|
||||||
@@ -452,7 +454,7 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.data-table td a {
|
.data-table td a {
|
||||||
color: #6fa3ff;
|
color: #3c88d6;
|
||||||
text-decoration: none;
|
text-decoration: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -519,15 +521,15 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.page-btn:hover:not(:disabled) {
|
.page-btn:hover:not(:disabled) {
|
||||||
background: rgba(102, 126, 234, 0.25);
|
background: rgba(15, 110, 205, 0.25);
|
||||||
color: #8fa8f8;
|
color: #5ea2e8;
|
||||||
border-color: rgba(102, 126, 234, 0.4);
|
border-color: rgba(15, 110, 205, 0.4);
|
||||||
}
|
}
|
||||||
|
|
||||||
.page-btn.active-page {
|
.page-btn.active-page {
|
||||||
background: rgba(102, 126, 234, 0.3);
|
background: rgba(15, 110, 205, 0.3);
|
||||||
color: #8fa8f8;
|
color: #5ea2e8;
|
||||||
border-color: rgba(102, 126, 234, 0.5);
|
border-color: rgba(15, 110, 205, 0.5);
|
||||||
font-weight: 700;
|
font-weight: 700;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -580,8 +582,8 @@
|
|||||||
display: inline-block;
|
display: inline-block;
|
||||||
width: 40px;
|
width: 40px;
|
||||||
height: 40px;
|
height: 40px;
|
||||||
border: 3px solid rgba(102, 126, 234, 0.2);
|
border: 3px solid rgba(15, 110, 205, 0.2);
|
||||||
border-top-color: #667eea;
|
border-top-color: #0f6ecd;
|
||||||
border-radius: 50%;
|
border-radius: 50%;
|
||||||
animation: spin 0.8s linear infinite;
|
animation: spin 0.8s linear infinite;
|
||||||
margin-bottom: 12px;
|
margin-bottom: 12px;
|
||||||
@@ -790,26 +792,330 @@
|
|||||||
grid-template-columns: repeat(2, 1fr);
|
grid-template-columns: repeat(2, 1fr);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
</style>
|
|
||||||
|
|
||||||
|
/* ===== Glass Theme Override (No Purple) ===== */
|
||||||
|
|
||||||
|
:root {
|
||||||
|
--primary: #0f6ecd;
|
||||||
|
--primary-soft: rgba(15, 110, 205, 0.18);
|
||||||
|
--secondary: #0ea5a4;
|
||||||
|
--accent: #f59e0b;
|
||||||
|
--text: #112941;
|
||||||
|
--muted: #536b86;
|
||||||
|
--line: rgba(17, 41, 65, 0.14);
|
||||||
|
--glass: rgba(255, 255, 255, 0.62);
|
||||||
|
--glass-strong: rgba(255, 255, 255, 0.82);
|
||||||
|
--danger: #d94848;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
font-family: 'Fira Sans', 'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||||
|
color: var(--text);
|
||||||
|
background:
|
||||||
|
radial-gradient(900px 500px at -5% -5%, rgba(15, 110, 205, 0.20), transparent 55%),
|
||||||
|
radial-gradient(900px 520px at 108% -10%, rgba(14, 165, 164, 0.18), transparent 52%),
|
||||||
|
linear-gradient(145deg, #edf6ff 0%, #e8fbf6 50%, #f6fbff 100%);
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar {
|
||||||
|
background: var(--glass);
|
||||||
|
border-bottom: 1px solid var(--line);
|
||||||
|
backdrop-filter: blur(14px);
|
||||||
|
-webkit-backdrop-filter: blur(14px);
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar-logo {
|
||||||
|
background: linear-gradient(125deg, var(--primary), var(--secondary));
|
||||||
|
color: #fff;
|
||||||
|
font-family: 'Fira Code', sans-serif;
|
||||||
|
font-size: 14px;
|
||||||
|
font-weight: 700;
|
||||||
|
letter-spacing: 0.4px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar-title { color: #1a3f65; }
|
||||||
|
|
||||||
|
.topbar-link {
|
||||||
|
color: var(--muted);
|
||||||
|
border: 1px solid transparent;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar-link:hover {
|
||||||
|
color: var(--text);
|
||||||
|
background: rgba(255, 255, 255, 0.7);
|
||||||
|
border-color: var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.topbar-link.active {
|
||||||
|
color: var(--primary);
|
||||||
|
background: rgba(15, 110, 205, 0.12);
|
||||||
|
border-color: rgba(15, 110, 205, 0.24);
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-header h1 { color: #173d62; }
|
||||||
|
.page-header p { color: var(--muted); }
|
||||||
|
|
||||||
|
.stat-card,
|
||||||
|
.toolbar,
|
||||||
|
.result-card,
|
||||||
|
.dialog-box,
|
||||||
|
.detail-box {
|
||||||
|
background: var(--glass);
|
||||||
|
border-color: var(--line);
|
||||||
|
backdrop-filter: blur(14px);
|
||||||
|
-webkit-backdrop-filter: blur(14px);
|
||||||
|
box-shadow: 0 16px 36px rgba(15, 42, 74, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat-card:hover,
|
||||||
|
.result-card:hover {
|
||||||
|
background: var(--glass-strong);
|
||||||
|
border-color: rgba(15, 110, 205, 0.26);
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat-card .label,
|
||||||
|
.stat-card .sub,
|
||||||
|
.page-info,
|
||||||
|
.result-time,
|
||||||
|
.dialog-msg {
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat-card .value,
|
||||||
|
.detail-header h3,
|
||||||
|
.dialog-title,
|
||||||
|
.result-data-body h4,
|
||||||
|
.detail-body h4 {
|
||||||
|
color: #183f65;
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-group label { color: #304d6b; }
|
||||||
|
|
||||||
|
.filter-select,
|
||||||
|
.filter-input {
|
||||||
|
background: rgba(255, 255, 255, 0.76);
|
||||||
|
border-color: var(--line);
|
||||||
|
color: var(--text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-select:focus,
|
||||||
|
.filter-input:focus {
|
||||||
|
border-color: rgba(15, 110, 205, 0.5);
|
||||||
|
box-shadow: 0 0 0 3px rgba(15, 110, 205, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.filter-select option {
|
||||||
|
background: #ffffff;
|
||||||
|
color: #1b3f62;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn {
|
||||||
|
border: 1px solid transparent;
|
||||||
|
transition: all 0.2s ease;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-primary {
|
||||||
|
background: linear-gradient(125deg, var(--primary), #178ac2);
|
||||||
|
color: #fff;
|
||||||
|
box-shadow: 0 8px 16px rgba(15, 110, 205, 0.24);
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-primary:hover {
|
||||||
|
transform: translateY(-1px);
|
||||||
|
box-shadow: 0 12px 24px rgba(15, 110, 205, 0.26);
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-ghost {
|
||||||
|
color: #284867;
|
||||||
|
background: rgba(255, 255, 255, 0.76);
|
||||||
|
border-color: var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-ghost:hover {
|
||||||
|
color: #163a60;
|
||||||
|
background: #ffffff;
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-danger {
|
||||||
|
color: #b73d3d;
|
||||||
|
background: rgba(217, 72, 72, 0.10);
|
||||||
|
border-color: rgba(217, 72, 72, 0.22);
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-danger:hover {
|
||||||
|
color: #fff;
|
||||||
|
background: linear-gradient(125deg, #d94848, #e36d61);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card::before {
|
||||||
|
background: linear-gradient(180deg, var(--primary), var(--secondary));
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card.has-error::before {
|
||||||
|
background: linear-gradient(180deg, #d94848, #ed7a62);
|
||||||
|
}
|
||||||
|
|
||||||
|
.tag-city {
|
||||||
|
background: rgba(15, 110, 205, 0.12);
|
||||||
|
color: #0f5fae;
|
||||||
|
border-color: rgba(15, 110, 205, 0.24);
|
||||||
|
}
|
||||||
|
|
||||||
|
.tag-section {
|
||||||
|
background: rgba(14, 165, 164, 0.12);
|
||||||
|
color: #0b7a79;
|
||||||
|
border-color: rgba(14, 165, 164, 0.24);
|
||||||
|
}
|
||||||
|
|
||||||
|
.tag-type {
|
||||||
|
background: rgba(245, 158, 11, 0.16);
|
||||||
|
color: #b96f06;
|
||||||
|
border-color: rgba(245, 158, 11, 0.28);
|
||||||
|
}
|
||||||
|
|
||||||
|
.tag-error {
|
||||||
|
background: rgba(217, 72, 72, 0.14);
|
||||||
|
color: #bf3b3b;
|
||||||
|
border-color: rgba(217, 72, 72, 0.28);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-url a,
|
||||||
|
.data-table td a {
|
||||||
|
color: var(--primary);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-data {
|
||||||
|
background: rgba(255, 255, 255, 0.68);
|
||||||
|
border-color: var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-data-toggle {
|
||||||
|
color: #375677;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-data-toggle:hover {
|
||||||
|
color: #1b4168;
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-data-body pre,
|
||||||
|
.detail-body pre {
|
||||||
|
color: #225068;
|
||||||
|
}
|
||||||
|
|
||||||
|
.data-table {
|
||||||
|
color: #1f3f5d;
|
||||||
|
}
|
||||||
|
|
||||||
|
.data-table th,
|
||||||
|
.data-table td {
|
||||||
|
border-color: rgba(17, 41, 65, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
|
.data-table th {
|
||||||
|
background: rgba(15, 110, 205, 0.12);
|
||||||
|
color: #185a9f;
|
||||||
|
}
|
||||||
|
|
||||||
|
.data-table tr:nth-child(even) {
|
||||||
|
background: rgba(255, 255, 255, 0.42);
|
||||||
|
}
|
||||||
|
|
||||||
|
.data-table tr:hover {
|
||||||
|
background: rgba(255, 255, 255, 0.72);
|
||||||
|
}
|
||||||
|
|
||||||
|
.result-card-footer {
|
||||||
|
border-top-color: rgba(17, 41, 65, 0.10);
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-btn {
|
||||||
|
background: rgba(255, 255, 255, 0.72);
|
||||||
|
border-color: var(--line);
|
||||||
|
color: #2a4b6c;
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-btn:hover:not(:disabled) {
|
||||||
|
background: rgba(15, 110, 205, 0.14);
|
||||||
|
color: var(--primary);
|
||||||
|
border-color: rgba(15, 110, 205, 0.25);
|
||||||
|
}
|
||||||
|
|
||||||
|
.page-btn.active-page {
|
||||||
|
background: rgba(15, 110, 205, 0.2);
|
||||||
|
color: #0f5fae;
|
||||||
|
border-color: rgba(15, 110, 205, 0.35);
|
||||||
|
}
|
||||||
|
|
||||||
|
.empty-state,
|
||||||
|
.empty-state h3,
|
||||||
|
.empty-state p {
|
||||||
|
color: #4a6481;
|
||||||
|
}
|
||||||
|
|
||||||
|
.spinner {
|
||||||
|
border-color: rgba(15, 110, 205, 0.18);
|
||||||
|
border-top-color: var(--primary);
|
||||||
|
}
|
||||||
|
|
||||||
|
.toast {
|
||||||
|
color: #fff;
|
||||||
|
box-shadow: 0 8px 24px rgba(17, 41, 65, 0.24);
|
||||||
|
}
|
||||||
|
|
||||||
|
.overlay,
|
||||||
|
.detail-overlay {
|
||||||
|
background: rgba(8, 20, 36, 0.34);
|
||||||
|
backdrop-filter: blur(4px);
|
||||||
|
-webkit-backdrop-filter: blur(4px);
|
||||||
|
}
|
||||||
|
|
||||||
|
.dialog-title,
|
||||||
|
.detail-header h3 {
|
||||||
|
font-family: 'Fira Code', 'Noto Sans SC', sans-serif;
|
||||||
|
}
|
||||||
|
|
||||||
|
.detail-header {
|
||||||
|
border-bottom-color: rgba(17, 41, 65, 0.10);
|
||||||
|
}
|
||||||
|
|
||||||
|
.close-btn {
|
||||||
|
background: rgba(255, 255, 255, 0.75);
|
||||||
|
color: #345777;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.close-btn:hover {
|
||||||
|
background: #ffffff;
|
||||||
|
color: #1b4269;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (prefers-reduced-motion: reduce) {
|
||||||
|
* {
|
||||||
|
transition: none !important;
|
||||||
|
animation: none !important;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</style>
|
||||||
</head>
|
</head>
|
||||||
|
|
||||||
<body>
|
<body>
|
||||||
<!-- 顶部导航 -->
|
<!-- 顶部导航 -->
|
||||||
<div class="topbar">
|
<div class="topbar">
|
||||||
<a href="/" class="topbar-brand">
|
<a href="/" class="topbar-brand">
|
||||||
<div class="topbar-logo">📋</div>
|
<div class="topbar-logo">AG</div>
|
||||||
<span class="topbar-title">公告采集工具</span>
|
<span class="topbar-title">公告采集工具</span>
|
||||||
</a>
|
</a>
|
||||||
<div class="topbar-nav">
|
<div class="topbar-nav">
|
||||||
<a href="/" class="topbar-link">⚙️ 配置管理</a>
|
<a href="/" class="topbar-link"> 配置管理</a>
|
||||||
<a href="/results.html" class="topbar-link active">📊 抓取结果</a>
|
<a href="/results.html" class="topbar-link active"> 抓取结果</a>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="main">
|
<div class="main">
|
||||||
<!-- 页头 -->
|
<!-- 页头 -->
|
||||||
<div class="page-header">
|
<div class="page-header">
|
||||||
<h1>📊 抓取结果</h1>
|
<h1> 抓取结果</h1>
|
||||||
<p>所有抓取来源的历史结果，按抓取时间倒序展示</p>
|
<p>所有抓取来源的历史结果，按抓取时间倒序展示</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -859,8 +1165,8 @@
|
|||||||
</div>
|
</div>
|
||||||
<div class="toolbar-actions">
|
<div class="toolbar-actions">
|
||||||
<button class="btn btn-ghost" onclick="resetFilters()">↺ 重置</button>
|
<button class="btn btn-ghost" onclick="resetFilters()">↺ 重置</button>
|
||||||
<button class="btn btn-primary" onclick="loadResults()">🔄 刷新</button>
|
<button class="btn btn-primary" onclick="loadResults()"> 刷新</button>
|
||||||
<button class="btn btn-danger" onclick="confirmClearAll()">🗑 清空全部</button>
|
<!-- <button class="btn btn-danger" onclick="confirmClearAll()"> 清空全部</button> -->
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -883,7 +1189,7 @@
|
|||||||
<!-- 确认清空弹窗 -->
|
<!-- 确认清空弹窗 -->
|
||||||
<div class="overlay" id="confirmOverlay">
|
<div class="overlay" id="confirmOverlay">
|
||||||
<div class="dialog-box">
|
<div class="dialog-box">
|
||||||
<div class="dialog-title">⚠️ 确认清空</div>
|
<div class="dialog-title"> 确认清空</div>
|
||||||
<div class="dialog-msg" id="confirmMsg">确定要清空所有抓取结果吗？此操作不可撤销。</div>
|
<div class="dialog-msg" id="confirmMsg">确定要清空所有抓取结果吗？此操作不可撤销。</div>
|
||||||
<div class="dialog-actions">
|
<div class="dialog-actions">
|
||||||
<button class="btn btn-ghost" onclick="closeConfirm()">取消</button>
|
<button class="btn btn-ghost" onclick="closeConfirm()">取消</button>
|
||||||
@@ -993,7 +1299,7 @@
|
|||||||
renderResults(json.data || []);
|
renderResults(json.data || []);
|
||||||
renderPagination(json.total, json.page, json.pageSize);
|
renderPagination(json.total, json.page, json.pageSize);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
listEl.innerHTML = `<div class="empty-state"><div class="icon">⚠️</div><h3>加载失败</h3><p>${e.message}</p></div>`;
|
listEl.innerHTML = `<div class="empty-state"><div class="icon"></div><h3>加载失败</h3><p>${e.message}</p></div>`;
|
||||||
} finally {
|
} finally {
|
||||||
maskEl.classList.remove('show');
|
maskEl.classList.remove('show');
|
||||||
}
|
}
|
||||||
@@ -1015,7 +1321,7 @@
|
|||||||
if (data.length === 0) {
|
if (data.length === 0) {
|
||||||
listEl.innerHTML = `
|
listEl.innerHTML = `
|
||||||
<div class="empty-state">
|
<div class="empty-state">
|
||||||
<div class="icon">🔍</div>
|
<div class="icon"></div>
|
||||||
<h3>暂无抓取记录</h3>
|
<h3>暂无抓取记录</h3>
|
||||||
<p>运行抓取来源后，结果将自动保存在这里</p>
|
<p>运行抓取来源后，结果将自动保存在这里</p>
|
||||||
</div>`;
|
</div>`;
|
||||||
@@ -1029,13 +1335,13 @@
|
|||||||
${r.city ? `<span class="tag tag-city">${r.city}</span>` : ''}
|
${r.city ? `<span class="tag tag-city">${r.city}</span>` : ''}
|
||||||
${r.section ? `<span class="tag tag-section">${r.section}${r.subsection ? ' · ' + r.subsection : ''}</span>` : ''}
|
${r.section ? `<span class="tag tag-section">${r.section}${r.subsection ? ' · ' + r.subsection : ''}</span>` : ''}
|
||||||
${r.type ? `<span class="tag tag-type">${r.type}</span>` : ''}
|
${r.type ? `<span class="tag tag-type">${r.type}</span>` : ''}
|
||||||
${r.error ? `<span class="tag tag-error">❌ 失败</span>` : ''}
|
${r.error ? `<span class="tag tag-error"> 失败</span>` : ''}
|
||||||
</div>
|
</div>
|
||||||
<span class="result-time">${formatTime(r.scrapedAt)}</span>
|
<span class="result-time">${formatTime(r.scrapedAt)}</span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="result-url">
|
<div class="result-url">
|
||||||
<a href="${r.url}" target="_blank" title="${r.url}">🔗 ${r.url}</a>
|
<a href="${r.url}" target="_blank" title="${r.url}"> ${r.url}</a>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
${r.error ? `<div class="result-error">错误信息：${r.error}</div>` : ''}
|
${r.error ? `<div class="result-error">错误信息：${r.error}</div>` : ''}
|
||||||
@@ -1043,7 +1349,7 @@
|
|||||||
${!r.error && r.data ? `
|
${!r.error && r.data ? `
|
||||||
<div class="result-data">
|
<div class="result-data">
|
||||||
<button class="result-data-toggle" onclick="toggleData(this)">
|
<button class="result-data-toggle" onclick="toggleData(this)">
|
||||||
<span>📄 查看数据</span>
|
<span> 查看数据</span>
|
||||||
<span>▼</span>
|
<span>▼</span>
|
||||||
</button>
|
</button>
|
||||||
<div class="result-data-body">
|
<div class="result-data-body">
|
||||||
@@ -1052,7 +1358,7 @@
|
|||||||
</div>` : ''}
|
</div>` : ''}
|
||||||
|
|
||||||
<div class="result-card-footer">
|
<div class="result-card-footer">
|
||||||
${!r.error && r.data ? `<button class="btn btn-ghost btn-sm" onclick="openDetail('${r.id}')">🔎 全屏查看</button>` : ''}
|
${!r.error && r.data ? `<button class="btn btn-ghost btn-sm" onclick="openDetail('${r.id}')"> 全屏查看</button>` : ''}
|
||||||
<button class="btn btn-danger btn-sm" onclick="deleteResult('${r.id}')">删除</button>
|
<button class="btn btn-danger btn-sm" onclick="deleteResult('${r.id}')">删除</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -1245,7 +1551,7 @@
|
|||||||
else val = String(val);
|
else val = String(val);
|
||||||
|
|
||||||
if ((val.startsWith('http://') || val.startsWith('https://')) && !val.includes(' ')) {
|
if ((val.startsWith('http://') || val.startsWith('https://')) && !val.includes(' ')) {
|
||||||
html += `<td><a href="${escHtml(val)}" target="_blank" title="${escHtml(val)}">🔗 链接</a></td>`;
|
html += `<td><a href="${escHtml(val)}" target="_blank" title="${escHtml(val)}"> 链接</a></td>`;
|
||||||
} else {
|
} else {
|
||||||
html += `<td>${escHtml(val)}</td>`;
|
html += `<td>${escHtml(val)}</td>`;
|
||||||
}
|
}
|
||||||
|
|||||||
459
results.json
459
results.json
@@ -1,459 +0,0 @@
|
|||||||
[
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762494299",
|
|
||||||
"city": "南京市",
|
|
||||||
"section": "房建市政",
|
|
||||||
"subsection": "工程类、服务类",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/buildService1.html",
|
|
||||||
"scrapedAt": "2026-03-09T03:19:39.057Z",
|
|
||||||
"data": {
|
|
||||||
"result": [
|
|
||||||
{
|
|
||||||
"title": "【澄清公告】文化谷东路 (东吉大道-创新大道)、创新大道(研发二路-文化谷东路)一期道路建设项目 施工",
|
|
||||||
"amount": "3180",
|
|
||||||
"date": "2026-03-09",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260309/d8483e91-9a7b-4425-a860-c5c9b45365f0.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "【澄清公告】南京大学仙林校区学生宿舍楼第28-30幢 20KV变电所工程",
|
|
||||||
"amount": "528",
|
|
||||||
"date": "2026-03-09",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260309/5a14fb16-fbd1-44d0-9f3f-90823f3639dd.html"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"total": 2
|
|
||||||
},
|
|
||||||
"id": "result-1773026379058-wd4gj"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762354799",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-06T06:57:46.881Z",
|
|
||||||
"data": {
|
|
||||||
"result": [
|
|
||||||
{
|
|
||||||
"title": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"amount": "5,923,797元",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "http://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"total": 1
|
|
||||||
},
|
|
||||||
"id": "result-1772780266881-odaof"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762354799",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-06T06:42:40.619Z",
|
|
||||||
"data": {
|
|
||||||
"result": [
|
|
||||||
{
|
|
||||||
"title": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"amount": "5923797元",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"total": 1
|
|
||||||
},
|
|
||||||
"id": "result-1772779360620-xr7ue"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762354799",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-06T04:02:43.530Z",
|
|
||||||
"data": {
|
|
||||||
"items": [
|
|
||||||
{
|
|
||||||
"title": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"amount": "5923797元",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"total": 1
|
|
||||||
},
|
|
||||||
"id": "result-1772769763530-3axw2"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762354799",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-06T02:51:39.452Z",
|
|
||||||
"error": "Insufficient credits to perform this request. For more credits, you can upgrade your plan at https://firecrawl.dev/pricing or try changing the request limit to a lower value.",
|
|
||||||
"data": null,
|
|
||||||
"id": "result-1772765499452-ynhn0"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762494299",
|
|
||||||
"city": "南京市",
|
|
||||||
"section": "房建市政",
|
|
||||||
"subsection": "工程类",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/buildService1.html",
|
|
||||||
"scrapedAt": "2026-03-06T02:32:03.818Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": {
|
|
||||||
"target_date": "2026-03-06",
|
|
||||||
"notice_count": 0,
|
|
||||||
"notices": [],
|
|
||||||
"message": "截至当前时间（2026-03-06 02:19），网站尚未发布今日（2026-03-06）的招标公告。最新公告日期为2026-03-05。",
|
|
||||||
"recent_notices_fallback": [
|
|
||||||
{
|
|
||||||
"title": "麒麟科创园具身智能训练场装修项目",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/a20ee94f-b76e-4f88-b8df-2847c2f35ce1.html",
|
|
||||||
"amount": "5660000.00"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "站东13号（MCd080-07-08）地块10kV电力杆线迁改工程",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/f0b99840-e8de-4a08-b2ba-3e57a347864c.html",
|
|
||||||
"amount": "9543100.00"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "【澄清公告】螺丝桥大街北延(月安街至应天大街段)道路工程",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/1b3da624-fe86-4755-a268-a1967cd9d489.html",
|
|
||||||
"amount": "900万元"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "建邺路150-164号等9个地块城中村改造项目",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/6f4fcf2f-d198-4814-acd8-9817ef559a0c.html",
|
|
||||||
"amount": "1,900,000.00"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "【澄清公告】南京市溧水区柘塘街道供水管网及配套设施提升改造工程",
|
|
||||||
"date": "2026-03-05",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/11ec2263-4ed1-4115-bdd1-0a6dcbf1d6c1.html",
|
|
||||||
"amount": "11320.01万元"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-07T02:32:00.316Z",
|
|
||||||
"creditsUsed": 0
|
|
||||||
},
|
|
||||||
"id": "result-1772764323818-mj8km"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772762354799",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/zbgg/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-06T02:19:27.580Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": [
|
|
||||||
{
|
|
||||||
"title": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"project_amount": "5,923,797元 (最高投标限价)",
|
|
||||||
"publish_date": "2026-03-05",
|
|
||||||
"detail_url": "http://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-07T02:19:24.631Z",
|
|
||||||
"creditsUsed": 0
|
|
||||||
},
|
|
||||||
"id": "result-1772763567581-ahz62"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772699302521",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/jyxx/slgc/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-05T10:05:46.148Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": {
|
|
||||||
"announcements": [
|
|
||||||
{
|
|
||||||
"title": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"project_amount": "最高投标限价：5923797元",
|
|
||||||
"publish_date": "2026-03-05",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-06T10:05:45.297Z",
|
|
||||||
"creditsUsed": 180
|
|
||||||
},
|
|
||||||
"id": "result-1772705146148-kn0ko"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772699302521",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/index.shtml",
|
|
||||||
"scrapedAt": "2026-03-05T10:02:01.153Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": [
|
|
||||||
{
|
|
||||||
"标题": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"项目金额": "5,923,797元",
|
|
||||||
"发布日期": "2026-03-05",
|
|
||||||
"详情页完整URL": "http://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-06T10:02:00.100Z",
|
|
||||||
"creditsUsed": 769
|
|
||||||
},
|
|
||||||
"id": "result-1772704921153-jx48m"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772699302521",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/",
|
|
||||||
"scrapedAt": "2026-03-05T09:23:03.452Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": {
|
|
||||||
"announcements": [
|
|
||||||
{
|
|
||||||
"title": "高新区三级防控系统工程周三房浜闸站工程施工招标公告",
|
|
||||||
"amount": "5,923,797元",
|
|
||||||
"publish_date": "2026-03-05",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741071.shtml"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "[WXHS202603001-X01]惠山区紧密型县域医共体服务能力提标扩能建设项目（惠山区人民医院紧密型医共体资源共享中心建设项目）勘察设计",
|
|
||||||
"amount": "570.00万元",
|
|
||||||
"publish_date": "2026-03-05",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/03/05/4741246.shtml"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-06T09:23:01.561Z",
|
|
||||||
"creditsUsed": 0
|
|
||||||
},
|
|
||||||
"id": "result-1772702583452-9t3b8"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "scraper-1772699302521",
|
|
||||||
"city": "无锡市",
|
|
||||||
"section": "水利工程",
|
|
||||||
"subsection": "",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/wxsggzyjyzxzl/",
|
|
||||||
"scrapedAt": "2026-03-05T08:39:45.736Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": [
|
|
||||||
{
|
|
||||||
"title": "[WXJY202601013-X01]江阴市长泾镇蒲市村区域性综合农事服务中心江阴市",
|
|
||||||
"amount": "874.0万元",
|
|
||||||
"date": "2026-01-30",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/01/30/4726538.shtml"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "[WXXS202406006-X02]中共锡山区委党校异地新建项目施工总承包",
|
|
||||||
"amount": "10350.0万元",
|
|
||||||
"date": "2026-01-30",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/01/30/4726721.shtml"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "[WXXQ202601010-X01]无锡交响音乐厅“一厅”及“两中心”品牌商户用房",
|
|
||||||
"amount": "400.0万元",
|
|
||||||
"date": "2026-01-30",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/01/30/4726619.shtml"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "[WXXQ202601008-X01]生命园三期2号楼、3号楼改造项目工程总承包",
|
|
||||||
"amount": "3650.0万元",
|
|
||||||
"date": "2026-01-30",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/01/30/4726675.shtml"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"title": "[WXBH202601007-X01]军嶂山显山透绿工程-吴杨路郊野覆绿工程施工",
|
|
||||||
"amount": "440.0万元",
|
|
||||||
"date": "2026-01-30",
|
|
||||||
"url": "https://ggzyjy.wuxi.gov.cn/doc/2026/01/30/4726726.shtml"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-06T08:39:45.265Z",
|
|
||||||
"creditsUsed": 0
|
|
||||||
},
|
|
||||||
"id": "result-1772699985736-b3nca"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"scraperId": "nj-jtsw-zbgg",
|
|
||||||
"city": "南京市",
|
|
||||||
"section": "房建市政",
|
|
||||||
"subsection": "工程类",
|
|
||||||
"type": "招标公告",
|
|
||||||
"url": "https://njggzy.nanjing.gov.cn/njweb/",
|
|
||||||
"scrapedAt": "2026-03-05T08:05:33.097Z",
|
|
||||||
"data": {
|
|
||||||
"success": true,
|
|
||||||
"status": "completed",
|
|
||||||
"data": {
|
|
||||||
"招标公告": [
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】螺丝桥大街北延(月安街至应天大街段)道路工程 - 施工",
|
|
||||||
"项目金额": "900 万元",
|
|
||||||
"发布日期": "2026-03-05",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/1b3da624-fe86-4755-a268-a1967cd9d489.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "建邺路150-164号等9个地块城中村改造项目 - 施工",
|
|
||||||
"项目金额": "190 万元",
|
|
||||||
"发布日期": "2026-03-05",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/6f4fcf2f-d198-4814-acd8-9817ef559a0c.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】南京市溧水区柘塘街道供水管网及配套设施提升改造工程 - 施工",
|
|
||||||
"项目金额": "11320.01 万元",
|
|
||||||
"发布日期": "2026-03-05",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/11ec2263-4ed1-4115-bdd1-0a6dcbf1d6c1.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "栖霞区百水芊城春水坊等5个片区排水管网改造工程 - 施工",
|
|
||||||
"项目金额": "435.86 万元",
|
|
||||||
"发布日期": "2026-03-05",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260305/d69e5640-d549-4638-a64a-d1f9df58a903.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】兰桥八期保障性住房项目 - 新建居住区供配电工程",
|
|
||||||
"项目金额": "6000 万元",
|
|
||||||
"发布日期": "2026-03-04",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260304/33e25a55-42c4-471e-9a3c-f8e792957141.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "青云巷10号危房整治工程 - SG1施工",
|
|
||||||
"项目金额": "375 万元",
|
|
||||||
"发布日期": "2026-03-04",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260304/e821f82c-39d8-479e-9457-b6bf5d101d80.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "百水工业园地块保障房一期项目 - D地块1#楼（公安编号）室内装饰工程",
|
|
||||||
"项目金额": "600 万元",
|
|
||||||
"发布日期": "2026-03-04",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260304/5f8f2183-e26f-4c03-a76a-8b4d61b0011c.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "青云巷10号危房整治工程 - SG1施工",
|
|
||||||
"项目金额": "375 万元",
|
|
||||||
"发布日期": "2026-03-04",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260304/9aa2d916-c0c3-4fb6-afa4-37457f0d2ceb.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】全国高校区域技术转移转化中心生物药物创新平台 - 施工",
|
|
||||||
"项目金额": "11000 万元",
|
|
||||||
"发布日期": "2026-03-03",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260303/2d1fe57f-fe0e-42f9-a99a-c345683aed3f.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "轻质耐热合金制造基地项目 - 施工",
|
|
||||||
"项目金额": "11000 万元",
|
|
||||||
"发布日期": "2026-03-03",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001002/20260303/78b81308-1389-42fc-a8de-23b6b2b40be1.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】润埠花园二期项目 - 监理",
|
|
||||||
"项目金额": "111.37 万元",
|
|
||||||
"发布日期": "2026-03-05",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260305/acb0010f-dcbc-4ea4-a988-e4dc75670999.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "轻质耐热合金制造基地项目 - 监理",
|
|
||||||
"项目金额": "188 万元",
|
|
||||||
"发布日期": "2026-03-04",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260304/93ee4804-5a5e-4524-92a3-b6c367803bd1.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】南京江北新区无人机制造共享工厂项目 - 监理",
|
|
||||||
"项目金额": "212.44 万元",
|
|
||||||
"发布日期": "2026-03-04",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260304/e44a1d28-0f43-494e-8daf-2f81252ed06a.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "2026年四项环卫设施大中修项目 - 设计",
|
|
||||||
"项目金额": "25.58 万元",
|
|
||||||
"发布日期": "2026-03-03",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260303/225961f4-08c8-4398-99c9-7777bf0d16b7.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】南京市溧水区柘塘街道供水管网及配套设施提升改造工程 - 监理",
|
|
||||||
"项目金额": "164.33 万元",
|
|
||||||
"发布日期": "2026-03-03",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260303/a827d48e-8e1f-42c9-bd07-09ce369c20c6.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "江苏银行金融科技中心建设项目 - 勘察",
|
|
||||||
"项目金额": "170 万元",
|
|
||||||
"发布日期": "2026-03-02",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260302/0ead5303-03db-4d95-b8ea-b32070a39dfa.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】南京高新区溧水园和凤园区改扩建项目 - 精诚电工地块及惠诚工具地块扩建厂房设计",
|
|
||||||
"项目金额": "140.68 万元",
|
|
||||||
"发布日期": "2026-03-02",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260302/d8df73f9-88d0-4f5d-8831-f9857a1a4ebc.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】NO.新区2025G11房地产开发项目 - 全过程工程咨询服务",
|
|
||||||
"项目金额": "950 万元",
|
|
||||||
"发布日期": "2026-03-02",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260302/348f6add-d17e-406d-9690-b637762175d7.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "江苏省六合高级中学新建食堂体育馆项目 - 渣土运输处置",
|
|
||||||
"项目金额": "242.97917 万元",
|
|
||||||
"发布日期": "2026-02-28",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260228/2099a860-b3c2-411f-8580-72cbb55fef42.html"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"标题": "【澄清公告】药谷产业区药谷大道(华宝路-汤盘公路)建设工程 - 勘察设计",
|
|
||||||
"项目金额": "194 万元",
|
|
||||||
"发布日期": "2026-02-28",
|
|
||||||
"详情页完整URL": "https://njggzy.nanjing.gov.cn/njweb/fjsz/068001/068001001/20260228/ffee9562-374d-43fd-8829-bf51c5b3cb46.html"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"model": "spark-1-mini",
|
|
||||||
"expiresAt": "2026-03-06T08:05:31.995Z",
|
|
||||||
"creditsUsed": 0
|
|
||||||
},
|
|
||||||
"id": "result-1772697933097-7hm4v"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
151
src/agentService.js
Normal file
151
src/agentService.js
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
/**
|
||||||
|
* Agent API 服务封装
|
||||||
|
* 调用本地部署的 agent 进行公告抓取
|
||||||
|
*/
|
||||||
|
|
||||||
|
const DEFAULT_BASE_URL = 'http://192.168.3.65:18625';
|
||||||
|
const DEFAULT_POLL_INTERVAL = 3000; // 3秒轮询
|
||||||
|
const DEFAULT_TIMEOUT = 3600000; // 1小时超时
|
||||||
|
const FETCH_TIMEOUT = 30000; // 单次 fetch 30秒超时
|
||||||
|
const MAX_FETCH_RETRIES = 5; // 网络错误最多重试5次
|
||||||
|
const DEFAULT_MODE = 'qwen3.5-plus';
|
||||||
|
|
||||||
|
function normalizeMode(value) {
|
||||||
|
if (typeof value === 'string' && value.trim()) return value.trim();
|
||||||
|
return DEFAULT_MODE;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
function generateTaskId() {
|
||||||
|
return `task-${Date.now()}-${Math.random().toString(36).slice(2, 7)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sleep(ms) {
|
||||||
|
return new Promise(resolve => setTimeout(resolve, ms));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 带超时和重试的 fetch
|
||||||
|
*/
|
||||||
|
async function fetchWithRetry(url, fetchOptions, retries = MAX_FETCH_RETRIES, logPrefix = '[Agent]') {
|
||||||
|
for (let attempt = 1; attempt <= retries; attempt++) {
|
||||||
|
try {
|
||||||
|
const controller = new AbortController();
|
||||||
|
const timer = setTimeout(() => controller.abort(), FETCH_TIMEOUT);
|
||||||
|
const res = await fetch(url, { ...fetchOptions, signal: controller.signal });
|
||||||
|
clearTimeout(timer);
|
||||||
|
return res;
|
||||||
|
} catch (err) {
|
||||||
|
const isLast = attempt === retries;
|
||||||
|
console.warn(`${logPrefix} fetch 失败 (${attempt}/${retries}): ${err.message}`);
|
||||||
|
if (isLast) throw err;
|
||||||
|
await sleep(3000 * attempt); // 递增等待
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 创建 agent 任务
|
||||||
|
*/
|
||||||
|
async function createTask(prompt, options = {}) {
|
||||||
|
const baseUrl = options.baseUrl || DEFAULT_BASE_URL;
|
||||||
|
const mode = normalizeMode(options.mode);
|
||||||
|
const taskId = generateTaskId();
|
||||||
|
const logPrefix = options.logPrefix || '[Agent]';
|
||||||
|
|
||||||
|
const res = await fetchWithRetry(`${baseUrl}/agent/createTask`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ taskId, prompt, mode }),
|
||||||
|
}, MAX_FETCH_RETRIES, logPrefix);
|
||||||
|
|
||||||
|
if (!res.ok) {
|
||||||
|
throw new Error(`创建任务失败: HTTP ${res.status}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return { taskId };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 检查任务状态
|
||||||
|
* 返回空/null 表示任务还在运行，返回 { success, message, data } 表示完成
|
||||||
|
*/
|
||||||
|
async function checkTask(taskId, options = {}) {
|
||||||
|
const baseUrl = options.baseUrl || DEFAULT_BASE_URL;
|
||||||
|
const logPrefix = options.logPrefix || '[Agent]';
|
||||||
|
|
||||||
|
const res = await fetchWithRetry(`${baseUrl}/agent/checkTask`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ taskId }),
|
||||||
|
}, MAX_FETCH_RETRIES, logPrefix);
|
||||||
|
|
||||||
|
if (!res.ok) {
|
||||||
|
throw new Error(`检查任务失败: HTTP ${res.status}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const text = await res.text();
|
||||||
|
console.log(`${logPrefix} checkTask(${taskId}) 返回:`, text ? text.substring(0, 500) : '(空)');
|
||||||
|
if (!text || text.trim() === '' || text.trim() === 'null') {
|
||||||
|
return null; // 任务还在运行
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
return JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 运行 agent 任务：创建 + 轮询直到完成
|
||||||
|
* 返回 { results: [{ type, project_name, amount_yuan, date, detail_link }] }
|
||||||
|
*/
|
||||||
|
export async function runAgentTask(prompt, options = {}) {
|
||||||
|
const baseUrl = options.baseUrl || DEFAULT_BASE_URL;
|
||||||
|
const mode = normalizeMode(options.mode);
|
||||||
|
const pollInterval = options.pollInterval || DEFAULT_POLL_INTERVAL;
|
||||||
|
const timeout = options.timeout || DEFAULT_TIMEOUT;
|
||||||
|
const logPrefix = options.logPrefix || '[Agent]';
|
||||||
|
|
||||||
|
console.log(`${logPrefix} 创建任务...`);
|
||||||
|
console.log(`${logPrefix} 使用 mode: ${mode}`);
|
||||||
|
const { taskId } = await createTask(prompt, { baseUrl, mode, logPrefix });
|
||||||
|
console.log(`${logPrefix} 任务已创建: ${taskId}`);
|
||||||
|
|
||||||
|
const startTime = Date.now();
|
||||||
|
|
||||||
|
while (true) {
|
||||||
|
if (Date.now() - startTime > timeout) {
|
||||||
|
throw new Error(`任务超时 (${timeout / 1000}秒): ${taskId}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
await sleep(pollInterval);
|
||||||
|
|
||||||
|
const result = await checkTask(taskId, { baseUrl, logPrefix });
|
||||||
|
|
||||||
|
if (result === null) {
|
||||||
|
const elapsed = Math.round((Date.now() - startTime) / 1000);
|
||||||
|
console.log(`${logPrefix} 任务进行中... (${elapsed}秒)`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (result.success) {
|
||||||
|
console.log(`${logPrefix} 任务完成: ${result.message}`);
|
||||||
|
const data = result.data || {};
|
||||||
|
const rawResults = Array.isArray(data.results) ? data.results : [];
|
||||||
|
const results = rawResults.map(item => {
|
||||||
|
if (!item || typeof item !== 'object') return item;
|
||||||
|
const detailLink = item.detail_link || item.target_link;
|
||||||
|
const { target_link, ...rest } = item;
|
||||||
|
return detailLink ? { ...rest, detail_link: detailLink } : rest;
|
||||||
|
});
|
||||||
|
console.log(`${logPrefix} 获取到 ${results.length} 条结果`);
|
||||||
|
return { results };
|
||||||
|
} else {
|
||||||
|
throw new Error(`任务失败: ${result.message || '未知错误'}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export { generateTaskId, createTask, checkTask };
|
||||||
@@ -767,23 +767,29 @@ function generateScraperResultsHtml(results) {
|
|||||||
const failResults = results.filter(r => r.error);
|
const failResults = results.filter(r => r.error);
|
||||||
const generatedAt = new Date().toLocaleString('zh-CN');
|
const generatedAt = new Date().toLocaleString('zh-CN');
|
||||||
|
|
||||||
// 把所有成功来源的 items 展开，附带来源信息
|
// Flatten all successful source items into one table.
|
||||||
const allRows = [];
|
const allRows = [];
|
||||||
for (const r of successResults) {
|
for (const r of successResults) {
|
||||||
const items = r.data?.result || [];
|
const items = r.data?.results || r.data?.result || [];
|
||||||
for (const item of items) {
|
for (const item of items) {
|
||||||
|
const hasAmount = typeof item.amount_yuan === 'number' || !!item.amount;
|
||||||
|
const amountText =
|
||||||
|
typeof item.amount_yuan === 'number'
|
||||||
|
? `${item.amount_yuan.toLocaleString('zh-CN')} CNY`
|
||||||
|
: (item.amount || 'N/A');
|
||||||
|
|
||||||
allRows.push({
|
allRows.push({
|
||||||
section: [r.section, r.subsection].filter(Boolean).join(' · ') || r.city || '-',
|
section: [r.section, r.subsection].filter(Boolean).join(' / ') || r.city || '-',
|
||||||
type: r.type || '-',
|
type: item.type || r.type || '-',
|
||||||
title: item.title || '-',
|
title: item.project_name || item.title || '-',
|
||||||
date: item.date || '-',
|
date: item.date || '-',
|
||||||
amount: item.amount || '未公开',
|
amount: amountText,
|
||||||
url: item.url || '',
|
hasAmount,
|
||||||
|
url: item.detail_link || item.target_link || item.url || '',
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 按日期降序排列
|
|
||||||
allRows.sort((a, b) => {
|
allRows.sort((a, b) => {
|
||||||
if (a.date === b.date) return 0;
|
if (a.date === b.date) return 0;
|
||||||
return a.date > b.date ? -1 : 1;
|
return a.date > b.date ? -1 : 1;
|
||||||
@@ -802,7 +808,7 @@ function generateScraperResultsHtml(results) {
|
|||||||
</td>
|
</td>
|
||||||
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;font-size:13px;max-width:320px;">${row.title}</td>
|
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;font-size:13px;max-width:320px;">${row.title}</td>
|
||||||
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;color:#555;">${row.date}</td>
|
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;color:#555;">${row.date}</td>
|
||||||
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;font-weight:600;color:${row.amount === '未公开' ? '#aaa' : '#e67e22'};">${row.amount}</td>
|
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;white-space:nowrap;font-size:13px;font-weight:600;color:${row.hasAmount ? '#e67e22' : '#aaa'};">${row.amount}</td>
|
||||||
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;text-align:center;">
|
<td style="padding:9px 12px;border-bottom:1px solid #eaecf5;text-align:center;">
|
||||||
${row.url
|
${row.url
|
||||||
? `<a href="${row.url}" target="_blank" style="color:#667eea;font-size:12px;text-decoration:none;white-space:nowrap;">查看 →</a>`
|
? `<a href="${row.url}" target="_blank" style="color:#667eea;font-size:12px;text-decoration:none;white-space:nowrap;">查看 →</a>`
|
||||||
@@ -851,7 +857,7 @@ function generateScraperResultsHtml(results) {
|
|||||||
<div style="font-size:12px;color:#888;margin-top:2px;">成功来源</div>
|
<div style="font-size:12px;color:#888;margin-top:2px;">成功来源</div>
|
||||||
</div>
|
</div>
|
||||||
<div style="flex:1;padding:16px 24px;text-align:center;border-right:1px solid #eaecf5;">
|
<div style="flex:1;padding:16px 24px;text-align:center;border-right:1px solid #eaecf5;">
|
||||||
<div style="font-size:28px;font-weight:700;color:#e67e22;">${allRows.filter(r => r.amount && r.amount !== '未公开').length}</div>
|
<div style="font-size:28px;font-weight:700;color:#e67e22;">${allRows.filter(r => r.hasAmount).length}</div>
|
||||||
<div style="font-size:12px;color:#888;margin-top:2px;">有金额</div>
|
<div style="font-size:12px;color:#888;margin-top:2px;">有金额</div>
|
||||||
</div>
|
</div>
|
||||||
<div style="flex:1;padding:16px 24px;text-align:center;">
|
<div style="flex:1;padding:16px 24px;text-align:center;">
|
||||||
|
|||||||
275
src/firecrawlBrowserScraper.js
Normal file
275
src/firecrawlBrowserScraper.js
Normal file
@@ -0,0 +1,275 @@
|
|||||||
|
const DEFAULT_SCRAPER_PROMPT = '提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等）、发布日期（YYYY-MM-DD格式）、详情页完整URL';
|
||||||
|
const PAYLOAD_MARKER = '__FC_PAYLOAD__';
|
||||||
|
|
||||||
|
function pad2(value) {
|
||||||
|
return String(value).padStart(2, '0');
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatDate(year, month, day) {
|
||||||
|
return `${year}-${pad2(month)}-${pad2(day)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function getTodayInShanghai() {
|
||||||
|
return new Intl.DateTimeFormat('en-CA', {
|
||||||
|
timeZone: 'Asia/Shanghai',
|
||||||
|
year: 'numeric',
|
||||||
|
month: '2-digit',
|
||||||
|
day: '2-digit',
|
||||||
|
}).format(new Date());
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseTargetDate(prompt) {
|
||||||
|
const text = String(prompt || '');
|
||||||
|
if (!text) return null;
|
||||||
|
|
||||||
|
const fullDate = text.match(/(20\d{2})[-/.年](\d{1,2})[-/.月](\d{1,2})日?/);
|
||||||
|
if (fullDate) {
|
||||||
|
return formatDate(fullDate[1], fullDate[2], fullDate[3]);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (/(今天|今日|当日)/.test(text)) {
|
||||||
|
return getTodayInShanghai();
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeDate(input) {
|
||||||
|
if (!input) return '';
|
||||||
|
const text = String(input).trim();
|
||||||
|
if (!text) return '';
|
||||||
|
|
||||||
|
let m = text.match(/(20\d{2})[-/.年](\d{1,2})[-/.月](\d{1,2})日?/);
|
||||||
|
if (m) return formatDate(m[1], m[2], m[3]);
|
||||||
|
|
||||||
|
m = text.match(/(\d{1,2})[-/.月](\d{1,2})日?/);
|
||||||
|
if (m) {
|
||||||
|
const currentYear = Number(getTodayInShanghai().slice(0, 4));
|
||||||
|
return formatDate(currentYear, m[1], m[2]);
|
||||||
|
}
|
||||||
|
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function extractDateFromText(text) {
|
||||||
|
if (!text) return '';
|
||||||
|
const m = String(text).match(/(20\d{2}[-/.年]\d{1,2}[-/.月]\d{1,2}日?)|(\d{1,2}[-/.月]\d{1,2}日?)/);
|
||||||
|
return m ? normalizeDate(m[0]) : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function extractAmountFromText(text) {
|
||||||
|
if (!text) return null;
|
||||||
|
const m = String(text).match(/([0-9][0-9,.\s]*(?:亿元|万元|万|元))/);
|
||||||
|
if (!m) return null;
|
||||||
|
return m[1].replace(/\s+/g, '').trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
function cleanText(text) {
|
||||||
|
return String(text || '').replace(/\s+/g, ' ').trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
function toFiniteNumber(value, fallback) {
|
||||||
|
const n = Number(value);
|
||||||
|
return Number.isFinite(n) ? n : fallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parsePayloadFromText(rawText) {
|
||||||
|
if (!rawText) return null;
|
||||||
|
const text = String(rawText);
|
||||||
|
|
||||||
|
const markerIndex = text.lastIndexOf(PAYLOAD_MARKER);
|
||||||
|
if (markerIndex >= 0) {
|
||||||
|
const tail = text.slice(markerIndex + PAYLOAD_MARKER.length);
|
||||||
|
const firstLine = tail.split(/\r?\n/).find(line => line.trim());
|
||||||
|
if (firstLine) {
|
||||||
|
try {
|
||||||
|
return JSON.parse(firstLine.trim());
|
||||||
|
} catch {
|
||||||
|
// Continue fallback parsing.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
return JSON.parse(text.trim());
|
||||||
|
} catch {
|
||||||
|
// Continue fallback parsing.
|
||||||
|
}
|
||||||
|
|
||||||
|
const lines = text.split(/\r?\n/).map(line => line.trim()).filter(Boolean).reverse();
|
||||||
|
for (const line of lines) {
|
||||||
|
try {
|
||||||
|
return JSON.parse(line);
|
||||||
|
} catch {
|
||||||
|
// Try next line.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseBrowserExecutePayload(executeResult) {
|
||||||
|
const sources = [executeResult?.result, executeResult?.stdout]
|
||||||
|
.filter(value => typeof value === 'string' && value.trim().length > 0);
|
||||||
|
|
||||||
|
for (const source of sources) {
|
||||||
|
const payload = parsePayloadFromText(source);
|
||||||
|
if (payload && typeof payload === 'object') return payload;
|
||||||
|
}
|
||||||
|
|
||||||
|
return { items: [] };
|
||||||
|
}
|
||||||
|
|
||||||
|
function splitKeywords(input) {
|
||||||
|
return String(input || '')
|
||||||
|
.split(/[、/，,|\s]+/)
|
||||||
|
.map(item => item.trim())
|
||||||
|
.filter(item => item.length >= 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
function filterByTypeIfPossible(items, type) {
|
||||||
|
const keywords = splitKeywords(type);
|
||||||
|
if (keywords.length === 0) return items;
|
||||||
|
|
||||||
|
const filtered = items.filter(item => {
|
||||||
|
const haystack = `${item.title} ${item.context || ''}`;
|
||||||
|
return keywords.some(keyword => haystack.includes(keyword));
|
||||||
|
});
|
||||||
|
|
||||||
|
return filtered.length > 0 ? filtered : items;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeItems(rawItems, targetDate, scraperType) {
|
||||||
|
const dedup = new Map();
|
||||||
|
|
||||||
|
for (const raw of rawItems) {
|
||||||
|
const title = cleanText(raw?.title);
|
||||||
|
const url = cleanText(raw?.url);
|
||||||
|
if (!title || !url) continue;
|
||||||
|
|
||||||
|
const context = cleanText(raw?.context);
|
||||||
|
const date = normalizeDate(raw?.date) || extractDateFromText(context);
|
||||||
|
const amount = cleanText(raw?.amount) || extractAmountFromText(context) || null;
|
||||||
|
const key = `${title}@@${url}`;
|
||||||
|
|
||||||
|
if (!dedup.has(key)) {
|
||||||
|
dedup.set(key, { title, amount, date, url, context });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let items = Array.from(dedup.values());
|
||||||
|
items = filterByTypeIfPossible(items, scraperType);
|
||||||
|
|
||||||
|
if (targetDate) {
|
||||||
|
items = items.filter(item => item.date === targetDate);
|
||||||
|
}
|
||||||
|
|
||||||
|
return items
|
||||||
|
.map(({ title, amount, date, url }) => ({ title, amount, date, url }))
|
||||||
|
.slice(0, 100);
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildBrowserScript(url) {
|
||||||
|
return `
|
||||||
|
const targetUrl = ${JSON.stringify(url)};
|
||||||
|
await page.goto(targetUrl, { waitUntil: 'domcontentloaded', timeout: 60000 });
|
||||||
|
await page.waitForTimeout(1500);
|
||||||
|
|
||||||
|
const payload = await page.evaluate(() => {
|
||||||
|
const normalize = (value) => String(value || '').replace(/\\s+/g, ' ').trim();
|
||||||
|
const blockedTitles = new Set(['首页', '尾页', '上一页', '下一页', '更多', '详情', '查看', '返回', '跳转']);
|
||||||
|
|
||||||
|
const links = Array.from(document.querySelectorAll('a[href]'));
|
||||||
|
const rows = [];
|
||||||
|
const seen = new Set();
|
||||||
|
|
||||||
|
for (const a of links) {
|
||||||
|
const href = a.getAttribute('href') || '';
|
||||||
|
if (!href || href.startsWith('javascript:') || href.startsWith('#')) continue;
|
||||||
|
|
||||||
|
const title = normalize(a.textContent);
|
||||||
|
if (!title || title.length < 6 || title.length > 180) continue;
|
||||||
|
if (blockedTitles.has(title)) continue;
|
||||||
|
|
||||||
|
let absoluteUrl = '';
|
||||||
|
try {
|
||||||
|
absoluteUrl = new URL(href, location.href).href;
|
||||||
|
} catch {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const container = a.closest('tr,li,article,section,div,p,dd,dt') || a.parentElement;
|
||||||
|
const context = normalize(container ? container.textContent : title);
|
||||||
|
|
||||||
|
const dateMatch = context.match(/(20\\d{2}[-/.年]\\d{1,2}[-/.月]\\d{1,2}日?)|(\\d{1,2}[-/.月]\\d{1,2}日?)/);
|
||||||
|
const amountMatch = context.match(/([0-9][0-9,.\\s]*(?:亿元|万元|万|元))/);
|
||||||
|
|
||||||
|
const key = (title + '@@' + absoluteUrl).toLowerCase();
|
||||||
|
if (seen.has(key)) continue;
|
||||||
|
seen.add(key);
|
||||||
|
|
||||||
|
rows.push({
|
||||||
|
title,
|
||||||
|
url: absoluteUrl,
|
||||||
|
date: dateMatch ? dateMatch[0] : '',
|
||||||
|
amount: amountMatch ? amountMatch[0].replace(/\\s+/g, '') : null,
|
||||||
|
context,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
pageUrl: location.href,
|
||||||
|
items: rows.slice(0, 300),
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log('${PAYLOAD_MARKER}' + JSON.stringify(payload));
|
||||||
|
JSON.stringify(payload);
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function runScraperWithBrowser(firecrawl, scraper, options = {}) {
|
||||||
|
const prefix = options.logPrefix || '[Browser]';
|
||||||
|
if (!scraper?.url) throw new Error('抓取 URL 不能为空');
|
||||||
|
|
||||||
|
const prompt = scraper.prompt || DEFAULT_SCRAPER_PROMPT;
|
||||||
|
const targetDate = parseTargetDate(prompt);
|
||||||
|
|
||||||
|
const ttl = toFiniteNumber(scraper.browserTtl, 180);
|
||||||
|
const activityTtl = toFiniteNumber(scraper.browserActivityTtl, 90);
|
||||||
|
|
||||||
|
const session = await firecrawl.browser({ ttl, activityTtl });
|
||||||
|
if (!session?.success || !session.id) {
|
||||||
|
throw new Error(session?.error || '创建 Browser 会话失败');
|
||||||
|
}
|
||||||
|
|
||||||
|
let executeResult;
|
||||||
|
try {
|
||||||
|
executeResult = await firecrawl.browserExecute(session.id, {
|
||||||
|
code: buildBrowserScript(scraper.url),
|
||||||
|
language: 'node',
|
||||||
|
});
|
||||||
|
} finally {
|
||||||
|
try {
|
||||||
|
await firecrawl.deleteBrowser(session.id);
|
||||||
|
} catch (closeError) {
|
||||||
|
console.warn(`${prefix} 会话关闭失败: ${closeError.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!executeResult?.success) {
|
||||||
|
throw new Error(executeResult?.error || executeResult?.stderr || 'Browser 执行失败');
|
||||||
|
}
|
||||||
|
|
||||||
|
const payload = parseBrowserExecutePayload(executeResult);
|
||||||
|
const rawItems = Array.isArray(payload.items) ? payload.items : [];
|
||||||
|
const items = normalizeItems(rawItems, targetDate, scraper.type);
|
||||||
|
|
||||||
|
console.log(`${prefix} URL=${scraper.url} raw=${rawItems.length} normalized=${items.length}${targetDate ? ` targetDate=${targetDate}` : ''}`);
|
||||||
|
|
||||||
|
return {
|
||||||
|
items,
|
||||||
|
targetDate,
|
||||||
|
pageUrl: payload.pageUrl || scraper.url,
|
||||||
|
};
|
||||||
|
}
|
||||||
731
src/resultStore.js
Normal file
731
src/resultStore.js
Normal file
@@ -0,0 +1,731 @@
|
|||||||
|
import Database from 'better-sqlite3';
|
||||||
|
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'fs';
|
||||||
|
import { fileURLToPath } from 'url';
|
||||||
|
import { dirname, join } from 'path';
|
||||||
|
|
||||||
|
const __filename = fileURLToPath(import.meta.url);
|
||||||
|
const __dirname = dirname(__filename);
|
||||||
|
|
||||||
|
const DB_PATH =
|
||||||
|
process.env.APP_DB_PATH ||
|
||||||
|
process.env.RESULTS_DB_PATH ||
|
||||||
|
join(__dirname, '..', 'data', 'results.sqlite');
|
||||||
|
const CONFIG_PATH = join(__dirname, '..', 'config.json');
|
||||||
|
const MAX_RESULT_RECORDS = 500;
|
||||||
|
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
|
||||||
|
const TASK_COLUMNS = ['id', 'city', 'plate_name', 'prompt', 'enabled', 'mode', 'created_at', 'updated_at'];
|
||||||
|
|
||||||
|
let db = null;
|
||||||
|
let initialized = false;
|
||||||
|
|
||||||
|
function clone(value) {
|
||||||
|
return JSON.parse(JSON.stringify(value));
|
||||||
|
}
|
||||||
|
|
||||||
|
function generateResultId() {
|
||||||
|
return `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function generateTaskId() {
|
||||||
|
return `task-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function getDefaultJsonConfig() {
|
||||||
|
return {
|
||||||
|
agent: {
|
||||||
|
baseUrl: '',
|
||||||
|
pollInterval: 3000,
|
||||||
|
timeout: 300000,
|
||||||
|
},
|
||||||
|
scheduler: {
|
||||||
|
enabled: false,
|
||||||
|
cronTime: '0 9 * * *',
|
||||||
|
threshold: 100000,
|
||||||
|
description: '',
|
||||||
|
timeRange: 'thisMonth',
|
||||||
|
},
|
||||||
|
email: {
|
||||||
|
smtpHost: '',
|
||||||
|
smtpPort: 587,
|
||||||
|
smtpUser: '',
|
||||||
|
smtpPass: '',
|
||||||
|
recipients: '',
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeJsonConfig(input = {}) {
|
||||||
|
const defaults = getDefaultJsonConfig();
|
||||||
|
const agent = input.agent || {};
|
||||||
|
const scheduler = input.scheduler || {};
|
||||||
|
const email = input.email || {};
|
||||||
|
|
||||||
|
return {
|
||||||
|
agent: {
|
||||||
|
baseUrl: typeof agent.baseUrl === 'string' ? agent.baseUrl : defaults.agent.baseUrl,
|
||||||
|
pollInterval: Number.isFinite(agent.pollInterval) ? agent.pollInterval : defaults.agent.pollInterval,
|
||||||
|
timeout: Number.isFinite(agent.timeout) ? agent.timeout : defaults.agent.timeout,
|
||||||
|
},
|
||||||
|
scheduler: {
|
||||||
|
enabled: scheduler.enabled === true,
|
||||||
|
cronTime: typeof scheduler.cronTime === 'string' && scheduler.cronTime.trim()
|
||||||
|
? scheduler.cronTime
|
||||||
|
: defaults.scheduler.cronTime,
|
||||||
|
threshold: Number.isFinite(scheduler.threshold) ? scheduler.threshold : defaults.scheduler.threshold,
|
||||||
|
description: typeof scheduler.description === 'string' ? scheduler.description : defaults.scheduler.description,
|
||||||
|
timeRange: typeof scheduler.timeRange === 'string' && scheduler.timeRange.trim()
|
||||||
|
? scheduler.timeRange
|
||||||
|
: defaults.scheduler.timeRange,
|
||||||
|
},
|
||||||
|
email: {
|
||||||
|
smtpHost: typeof email.smtpHost === 'string' ? email.smtpHost : defaults.email.smtpHost,
|
||||||
|
smtpPort: Number.isFinite(email.smtpPort) ? email.smtpPort : defaults.email.smtpPort,
|
||||||
|
smtpUser: typeof email.smtpUser === 'string' ? email.smtpUser : defaults.email.smtpUser,
|
||||||
|
smtpPass: typeof email.smtpPass === 'string' ? email.smtpPass : defaults.email.smtpPass,
|
||||||
|
recipients: typeof email.recipients === 'string' ? email.recipients : defaults.email.recipients,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeTaskMode(value) {
|
||||||
|
if (typeof value === 'string' && value.trim()) return value.trim();
|
||||||
|
return DEFAULT_TASK_MODE;
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildTaskRecord(task = {}) {
|
||||||
|
return {
|
||||||
|
id: task.id || generateTaskId(),
|
||||||
|
city: task.city || '',
|
||||||
|
plateName: task.plateName || '',
|
||||||
|
prompt: task.prompt || '',
|
||||||
|
enabled: task.enabled !== false,
|
||||||
|
mode: normalizeTaskMode(task.mode),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildResultRecord(result = {}) {
|
||||||
|
return {
|
||||||
|
id: result.id || generateResultId(),
|
||||||
|
taskId: result.taskId || null,
|
||||||
|
city: result.city || null,
|
||||||
|
scrapedAt: result.scrapedAt || new Date().toISOString(),
|
||||||
|
error: result.error || null,
|
||||||
|
data: result.data ?? null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseTaskRow(row) {
|
||||||
|
return {
|
||||||
|
id: row.id,
|
||||||
|
city: row.city,
|
||||||
|
plateName: row.plate_name,
|
||||||
|
prompt: row.prompt,
|
||||||
|
enabled: row.enabled === 1,
|
||||||
|
mode: normalizeTaskMode(row.mode),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseResultRow(row) {
|
||||||
|
return {
|
||||||
|
id: row.id,
|
||||||
|
taskId: row.task_id,
|
||||||
|
city: row.city,
|
||||||
|
scrapedAt: row.scraped_at,
|
||||||
|
error: row.error,
|
||||||
|
data: row.data_json ? JSON.parse(row.data_json) : null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function getDb() {
|
||||||
|
if (db) return db;
|
||||||
|
|
||||||
|
mkdirSync(dirname(DB_PATH), { recursive: true });
|
||||||
|
db = new Database(DB_PATH);
|
||||||
|
db.pragma('journal_mode = WAL');
|
||||||
|
return db;
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureSchema() {
|
||||||
|
getDb().exec(`
|
||||||
|
CREATE TABLE IF NOT EXISTS results (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
task_id TEXT,
|
||||||
|
city TEXT,
|
||||||
|
scraped_at TEXT NOT NULL,
|
||||||
|
error TEXT,
|
||||||
|
data_json TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_results_scraped_at ON results (scraped_at DESC);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_results_city ON results (city);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_results_task_id ON results (task_id);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS tasks (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
city TEXT,
|
||||||
|
plate_name TEXT,
|
||||||
|
prompt TEXT,
|
||||||
|
enabled INTEGER NOT NULL DEFAULT 1,
|
||||||
|
mode TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureTasksTableShape() {
|
||||||
|
const columns = getDb().prepare(`PRAGMA table_info(tasks)`).all();
|
||||||
|
const columnNames = columns.map((column) => column.name);
|
||||||
|
const hasLegacyBrowserColumn = columnNames.includes('use_browser');
|
||||||
|
const matchesExpectedShape =
|
||||||
|
columnNames.length === TASK_COLUMNS.length &&
|
||||||
|
TASK_COLUMNS.every((column, index) => columnNames[index] === column);
|
||||||
|
|
||||||
|
if (!hasLegacyBrowserColumn && matchesExpectedShape) return;
|
||||||
|
|
||||||
|
getDb().exec(`
|
||||||
|
BEGIN;
|
||||||
|
|
||||||
|
ALTER TABLE tasks RENAME TO tasks_legacy;
|
||||||
|
|
||||||
|
CREATE TABLE tasks (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
city TEXT,
|
||||||
|
plate_name TEXT,
|
||||||
|
prompt TEXT,
|
||||||
|
enabled INTEGER NOT NULL DEFAULT 1,
|
||||||
|
mode TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO tasks (id, city, plate_name, prompt, enabled, mode, created_at, updated_at)
|
||||||
|
SELECT
|
||||||
|
id,
|
||||||
|
city,
|
||||||
|
plate_name,
|
||||||
|
prompt,
|
||||||
|
COALESCE(enabled, 1),
|
||||||
|
COALESCE(mode, '${DEFAULT_TASK_MODE}'),
|
||||||
|
COALESCE(created_at, datetime('now')),
|
||||||
|
COALESCE(updated_at, datetime('now'))
|
||||||
|
FROM tasks_legacy;
|
||||||
|
|
||||||
|
DROP TABLE tasks_legacy;
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function trimResults(limit = MAX_RESULT_RECORDS) {
|
||||||
|
getDb().prepare(`
|
||||||
|
DELETE FROM results
|
||||||
|
WHERE id NOT IN (
|
||||||
|
SELECT id
|
||||||
|
FROM results
|
||||||
|
ORDER BY scraped_at DESC, rowid DESC
|
||||||
|
LIMIT ?
|
||||||
|
)
|
||||||
|
`).run(limit);
|
||||||
|
}
|
||||||
|
|
||||||
|
function readJsonIfExists(filePath) {
|
||||||
|
if (!existsSync(filePath)) return null;
|
||||||
|
return JSON.parse(readFileSync(filePath, 'utf-8'));
|
||||||
|
}
|
||||||
|
|
||||||
|
function stripTasksFromConfig(config) {
|
||||||
|
if (!config || typeof config !== 'object') return getDefaultJsonConfig();
|
||||||
|
const { agent, scheduler, email } = config;
|
||||||
|
return normalizeJsonConfig({ agent, scheduler, email });
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureJsonConfigExists() {
|
||||||
|
if (existsSync(CONFIG_PATH)) return;
|
||||||
|
writeFileSync(CONFIG_PATH, JSON.stringify(getDefaultJsonConfig(), null, 2), 'utf-8');
|
||||||
|
}
|
||||||
|
|
||||||
|
function queryBaseRows({ city, taskId }) {
|
||||||
|
const clauses = [];
|
||||||
|
const params = [];
|
||||||
|
|
||||||
|
if (city) {
|
||||||
|
clauses.push('city = ?');
|
||||||
|
params.push(city);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (taskId) {
|
||||||
|
clauses.push('task_id = ?');
|
||||||
|
params.push(taskId);
|
||||||
|
}
|
||||||
|
|
||||||
|
const whereSql = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : '';
|
||||||
|
const sql = `
|
||||||
|
SELECT id, task_id, city, scraped_at, error, data_json
|
||||||
|
FROM results
|
||||||
|
${whereSql}
|
||||||
|
ORDER BY scraped_at DESC, rowid DESC
|
||||||
|
`;
|
||||||
|
|
||||||
|
return getDb().prepare(sql).all(...params).map(parseResultRow);
|
||||||
|
}
|
||||||
|
|
||||||
|
function matchSection(record, section) {
|
||||||
|
if (!section) return true;
|
||||||
|
if (record.section === section || record.subsection === section) return true;
|
||||||
|
|
||||||
|
const items = record.data?.results || [];
|
||||||
|
return items.some((item) => item.section === section || item.subsection === section);
|
||||||
|
}
|
||||||
|
|
||||||
|
function matchType(record, type) {
|
||||||
|
if (!type) return true;
|
||||||
|
if (record.type === type) return true;
|
||||||
|
|
||||||
|
const items = record.data?.results || [];
|
||||||
|
return items.some((item) => item.type === type);
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeProjectName(item) {
|
||||||
|
if (!item || typeof item !== 'object') return '';
|
||||||
|
|
||||||
|
const candidates = [
|
||||||
|
item.project_name,
|
||||||
|
item.projectName,
|
||||||
|
item.title,
|
||||||
|
item.name,
|
||||||
|
item.bidName,
|
||||||
|
];
|
||||||
|
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
if (typeof candidate === 'string' && candidate.trim()) {
|
||||||
|
return candidate.replace(/\s+/g, ' ').trim();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function dedupeRowsByProjectName(rows) {
|
||||||
|
const seenProjectNames = new Set();
|
||||||
|
const dedupedRows = [];
|
||||||
|
|
||||||
|
for (const row of rows) {
|
||||||
|
const items = Array.isArray(row.data?.results) ? row.data.results : [];
|
||||||
|
if (items.length === 0) continue;
|
||||||
|
|
||||||
|
const uniqueItems = [];
|
||||||
|
|
||||||
|
for (const item of items) {
|
||||||
|
const projectName = normalizeProjectName(item);
|
||||||
|
|
||||||
|
if (!projectName) {
|
||||||
|
uniqueItems.push(item);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (seenProjectNames.has(projectName)) continue;
|
||||||
|
|
||||||
|
seenProjectNames.add(projectName);
|
||||||
|
uniqueItems.push(item);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (uniqueItems.length === 0) continue;
|
||||||
|
|
||||||
|
dedupedRows.push({
|
||||||
|
...row,
|
||||||
|
data: {
|
||||||
|
...(row.data || {}),
|
||||||
|
results: uniqueItems,
|
||||||
|
total: uniqueItems.length,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return dedupedRows;
|
||||||
|
}
|
||||||
|
|
||||||
|
function pickProjectLink(item) {
|
||||||
|
const candidates = [item?.detail_link, item?.target_link, item?.url, item?.href];
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
if (typeof candidate === 'string' && candidate.trim()) return candidate.trim();
|
||||||
|
}
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseProjectSection(item) {
|
||||||
|
if (typeof item?.section === 'string' && item.section.trim()) return item.section.trim();
|
||||||
|
if (typeof item?.type === 'string' && item.type.trim()) {
|
||||||
|
return item.type.split(/[-/]/)[0].trim();
|
||||||
|
}
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseProjectAmount(item) {
|
||||||
|
if (typeof item?.amount_yuan === 'number' && Number.isFinite(item.amount_yuan)) {
|
||||||
|
return item.amount_yuan;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (typeof item?.amount === 'number' && Number.isFinite(item.amount)) {
|
||||||
|
return item.amount;
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function projectSortValue(project) {
|
||||||
|
const dateValue = Date.parse(project.date || '');
|
||||||
|
const scrapedAtValue = Date.parse(project.scrapedAt || '');
|
||||||
|
|
||||||
|
return {
|
||||||
|
dateValue: Number.isFinite(dateValue) ? dateValue : 0,
|
||||||
|
scrapedAtValue: Number.isFinite(scrapedAtValue) ? scrapedAtValue : 0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function compareProjectsDesc(a, b) {
|
||||||
|
const aValue = projectSortValue(a);
|
||||||
|
const bValue = projectSortValue(b);
|
||||||
|
|
||||||
|
if (bValue.dateValue !== aValue.dateValue) return bValue.dateValue - aValue.dateValue;
|
||||||
|
if (bValue.scrapedAtValue !== aValue.scrapedAtValue) return bValue.scrapedAtValue - aValue.scrapedAtValue;
|
||||||
|
return (a.projectName || '').localeCompare(b.projectName || '', 'zh-CN');
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeSearchText(value) {
|
||||||
|
if (typeof value !== 'string') return '';
|
||||||
|
return value.trim().toLowerCase();
|
||||||
|
}
|
||||||
|
|
||||||
|
function includesSearchText(source, keyword) {
|
||||||
|
if (!keyword) return true;
|
||||||
|
return normalizeSearchText(source).includes(keyword);
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseNumberFilter(value) {
|
||||||
|
if (value === null || value === undefined || value === '') return null;
|
||||||
|
const parsed = Number(value);
|
||||||
|
return Number.isFinite(parsed) ? parsed : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseDateFilter(value) {
|
||||||
|
if (typeof value !== 'string' || !value.trim()) return null;
|
||||||
|
const timestamp = Date.parse(value);
|
||||||
|
return Number.isFinite(timestamp) ? timestamp : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function matchProjectFilters(project, filters = {}) {
|
||||||
|
const cityKeyword = normalizeSearchText(filters.city);
|
||||||
|
const sectionKeyword = normalizeSearchText(filters.section);
|
||||||
|
const projectNameKeyword = normalizeSearchText(filters.projectName);
|
||||||
|
const minAmount = parseNumberFilter(filters.minAmount);
|
||||||
|
const maxAmount = parseNumberFilter(filters.maxAmount);
|
||||||
|
const startDate = parseDateFilter(filters.startDate);
|
||||||
|
const endDate = parseDateFilter(filters.endDate);
|
||||||
|
const projectDate = parseDateFilter(project.date);
|
||||||
|
|
||||||
|
if (!includesSearchText(project.city, cityKeyword)) return false;
|
||||||
|
if (!includesSearchText(project.section || project.type, sectionKeyword)) return false;
|
||||||
|
if (!includesSearchText(project.projectName, projectNameKeyword)) return false;
|
||||||
|
if (minAmount !== null && (project.amountYuan === null || project.amountYuan < minAmount)) return false;
|
||||||
|
if (maxAmount !== null && (project.amountYuan === null || project.amountYuan > maxAmount)) return false;
|
||||||
|
if (startDate !== null && (projectDate === null || projectDate < startDate)) return false;
|
||||||
|
if (endDate !== null && (projectDate === null || projectDate > endDate)) return false;
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildProjectList(rows, {
|
||||||
|
dedupeByName = false,
|
||||||
|
city,
|
||||||
|
section,
|
||||||
|
projectNameKeyword,
|
||||||
|
minAmount,
|
||||||
|
maxAmount,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
} = {}) {
|
||||||
|
const seenProjectNames = new Set();
|
||||||
|
const projects = [];
|
||||||
|
|
||||||
|
for (const row of rows) {
|
||||||
|
const items = Array.isArray(row.data?.results) ? row.data.results : [];
|
||||||
|
|
||||||
|
for (let index = 0; index < items.length; index += 1) {
|
||||||
|
const item = items[index];
|
||||||
|
const projectName = normalizeProjectName(item);
|
||||||
|
if (!projectName) continue;
|
||||||
|
|
||||||
|
const projectSection = parseProjectSection(item);
|
||||||
|
if (city && row.city !== city) continue;
|
||||||
|
if (section && projectSection !== section) continue;
|
||||||
|
|
||||||
|
if (dedupeByName) {
|
||||||
|
if (seenProjectNames.has(projectName)) continue;
|
||||||
|
seenProjectNames.add(projectName);
|
||||||
|
}
|
||||||
|
|
||||||
|
projects.push({
|
||||||
|
id: `${row.id}:${index}`,
|
||||||
|
resultId: row.id,
|
||||||
|
taskId: row.taskId,
|
||||||
|
city: row.city || '',
|
||||||
|
section: projectSection,
|
||||||
|
type: typeof item?.type === 'string' ? item.type : '',
|
||||||
|
projectName,
|
||||||
|
amountYuan: parseProjectAmount(item),
|
||||||
|
date: typeof item?.date === 'string' ? item.date : '',
|
||||||
|
detailLink: pickProjectLink(item),
|
||||||
|
scrapedAt: row.scrapedAt,
|
||||||
|
raw: item,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return projects
|
||||||
|
.filter((project) => matchProjectFilters(project, {
|
||||||
|
city,
|
||||||
|
section,
|
||||||
|
projectName: projectNameKeyword,
|
||||||
|
minAmount,
|
||||||
|
maxAmount,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
}))
|
||||||
|
.sort(compareProjectsDesc);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function initResultsStore() {
|
||||||
|
if (initialized) return;
|
||||||
|
ensureSchema();
|
||||||
|
ensureTasksTableShape();
|
||||||
|
ensureJsonConfigExists();
|
||||||
|
initialized = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadConfig() {
|
||||||
|
initResultsStore();
|
||||||
|
const jsonConfig = normalizeJsonConfig(readJsonIfExists(CONFIG_PATH) || getDefaultJsonConfig());
|
||||||
|
return {
|
||||||
|
...clone(jsonConfig),
|
||||||
|
tasks: listTasks(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function saveConfig(config) {
|
||||||
|
initResultsStore();
|
||||||
|
const jsonConfig = stripTasksFromConfig(config);
|
||||||
|
writeFileSync(CONFIG_PATH, JSON.stringify(jsonConfig, null, 2), 'utf-8');
|
||||||
|
return {
|
||||||
|
...clone(jsonConfig),
|
||||||
|
tasks: listTasks(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function listTasks() {
|
||||||
|
initResultsStore();
|
||||||
|
return getDb()
|
||||||
|
.prepare(`
|
||||||
|
SELECT id, city, plate_name, prompt, enabled, mode
|
||||||
|
FROM tasks
|
||||||
|
ORDER BY rowid DESC
|
||||||
|
`)
|
||||||
|
.all()
|
||||||
|
.map(parseTaskRow);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getTaskById(id) {
|
||||||
|
initResultsStore();
|
||||||
|
const row = getDb()
|
||||||
|
.prepare(`
|
||||||
|
SELECT id, city, plate_name, prompt, enabled, mode
|
||||||
|
FROM tasks
|
||||||
|
WHERE id = ?
|
||||||
|
`)
|
||||||
|
.get(id);
|
||||||
|
|
||||||
|
return row ? parseTaskRow(row) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createTask(task) {
|
||||||
|
initResultsStore();
|
||||||
|
const record = buildTaskRecord(task);
|
||||||
|
getDb().prepare(`
|
||||||
|
INSERT INTO tasks (id, city, plate_name, prompt, enabled, mode, updated_at)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, datetime('now'))
|
||||||
|
`).run(
|
||||||
|
record.id,
|
||||||
|
record.city,
|
||||||
|
record.plateName,
|
||||||
|
record.prompt,
|
||||||
|
record.enabled ? 1 : 0,
|
||||||
|
record.mode,
|
||||||
|
);
|
||||||
|
return record;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function updateTask(id, patch) {
|
||||||
|
initResultsStore();
|
||||||
|
const current = getTaskById(id);
|
||||||
|
if (!current) return null;
|
||||||
|
|
||||||
|
const next = buildTaskRecord({ ...current, ...patch, id });
|
||||||
|
getDb().prepare(`
|
||||||
|
UPDATE tasks
|
||||||
|
SET city = ?,
|
||||||
|
plate_name = ?,
|
||||||
|
prompt = ?,
|
||||||
|
enabled = ?,
|
||||||
|
mode = ?,
|
||||||
|
updated_at = datetime('now')
|
||||||
|
WHERE id = ?
|
||||||
|
`).run(
|
||||||
|
next.city,
|
||||||
|
next.plateName,
|
||||||
|
next.prompt,
|
||||||
|
next.enabled ? 1 : 0,
|
||||||
|
next.mode,
|
||||||
|
id,
|
||||||
|
);
|
||||||
|
|
||||||
|
return next;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function deleteTaskById(id) {
|
||||||
|
initResultsStore();
|
||||||
|
const result = getDb().prepare('DELETE FROM tasks WHERE id = ?').run(id);
|
||||||
|
return result.changes > 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function appendResult(result) {
|
||||||
|
initResultsStore();
|
||||||
|
const record = buildResultRecord(result);
|
||||||
|
|
||||||
|
getDb().prepare(`
|
||||||
|
INSERT INTO results (id, task_id, city, scraped_at, error, data_json)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?)
|
||||||
|
`).run(
|
||||||
|
record.id,
|
||||||
|
record.taskId,
|
||||||
|
record.city,
|
||||||
|
record.scrapedAt,
|
||||||
|
record.error,
|
||||||
|
record.data === null ? null : JSON.stringify(record.data),
|
||||||
|
);
|
||||||
|
|
||||||
|
trimResults();
|
||||||
|
return record;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function listResults({ city, section, type, taskId, page = 1, pageSize = 20, projectMode = false } = {}) {
|
||||||
|
initResultsStore();
|
||||||
|
|
||||||
|
let results = queryBaseRows({ city, taskId });
|
||||||
|
if (section) results = results.filter((record) => matchSection(record, section));
|
||||||
|
if (type) results = results.filter((record) => matchType(record, type));
|
||||||
|
if (projectMode) results = dedupeRowsByProjectName(results);
|
||||||
|
|
||||||
|
const normalizedPage = Math.max(1, parseInt(page, 10) || 1);
|
||||||
|
const normalizedPageSize = Math.max(1, parseInt(pageSize, 10) || 20);
|
||||||
|
const start = (normalizedPage - 1) * normalizedPageSize;
|
||||||
|
|
||||||
|
return {
|
||||||
|
total: results.length,
|
||||||
|
page: normalizedPage,
|
||||||
|
pageSize: normalizedPageSize,
|
||||||
|
data: results.slice(start, start + normalizedPageSize),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function deleteResultById(id) {
|
||||||
|
initResultsStore();
|
||||||
|
const result = getDb().prepare('DELETE FROM results WHERE id = ?').run(id);
|
||||||
|
return result.changes > 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function clearResults() {
|
||||||
|
initResultsStore();
|
||||||
|
getDb().prepare('DELETE FROM results').run();
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getResultFilters({ projectMode = false } = {}) {
|
||||||
|
initResultsStore();
|
||||||
|
|
||||||
|
const rows = projectMode ? dedupeRowsByProjectName(queryBaseRows({})) : queryBaseRows({});
|
||||||
|
const cities = [...new Set(rows.map((row) => row.city).filter(Boolean))];
|
||||||
|
const sections = new Set();
|
||||||
|
const types = new Set();
|
||||||
|
|
||||||
|
for (const row of rows) {
|
||||||
|
if (row.section) sections.add(row.section);
|
||||||
|
if (row.subsection) sections.add(row.subsection);
|
||||||
|
if (row.type) types.add(row.type);
|
||||||
|
|
||||||
|
for (const item of row.data?.results || []) {
|
||||||
|
if (item.section) sections.add(item.section);
|
||||||
|
if (item.subsection) sections.add(item.subsection);
|
||||||
|
if (item.type) types.add(item.type);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
cities,
|
||||||
|
sections: [...sections],
|
||||||
|
types: [...types],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function listProjects({
|
||||||
|
city,
|
||||||
|
section,
|
||||||
|
projectName,
|
||||||
|
minAmount,
|
||||||
|
maxAmount,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
page = 1,
|
||||||
|
pageSize = 20,
|
||||||
|
dedupeByName = true,
|
||||||
|
} = {}) {
|
||||||
|
initResultsStore();
|
||||||
|
|
||||||
|
const rows = queryBaseRows({});
|
||||||
|
const projects = buildProjectList(rows, {
|
||||||
|
dedupeByName,
|
||||||
|
city,
|
||||||
|
section,
|
||||||
|
projectNameKeyword: projectName,
|
||||||
|
minAmount,
|
||||||
|
maxAmount,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
});
|
||||||
|
const normalizedPage = Math.max(1, parseInt(page, 10) || 1);
|
||||||
|
const normalizedPageSize = Math.max(1, parseInt(pageSize, 10) || 20);
|
||||||
|
const start = (normalizedPage - 1) * normalizedPageSize;
|
||||||
|
|
||||||
|
return {
|
||||||
|
total: projects.length,
|
||||||
|
page: normalizedPage,
|
||||||
|
pageSize: normalizedPageSize,
|
||||||
|
data: projects.slice(start, start + normalizedPageSize),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getProjectFilters({ dedupeByName = true } = {}) {
|
||||||
|
initResultsStore();
|
||||||
|
|
||||||
|
const projects = buildProjectList(queryBaseRows({}), { dedupeByName });
|
||||||
|
const cities = [...new Set(projects.map((project) => project.city).filter(Boolean))];
|
||||||
|
const sections = [...new Set(projects.map((project) => project.section).filter(Boolean))];
|
||||||
|
|
||||||
|
return {
|
||||||
|
cities,
|
||||||
|
sections,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getResultsDbPath() {
|
||||||
|
return DB_PATH;
|
||||||
|
}
|
||||||
274
src/scheduler.js
274
src/scheduler.js
@@ -1,261 +1,177 @@
|
|||||||
import 'dotenv/config';
|
import 'dotenv/config';
|
||||||
import cron from 'node-cron';
|
import cron from 'node-cron';
|
||||||
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
|
||||||
import { fileURLToPath } from 'url';
|
|
||||||
import { dirname, join } from 'path';
|
|
||||||
import Firecrawl from '@mendable/firecrawl-js';
|
|
||||||
import { z } from 'zod';
|
|
||||||
import { sendScraperResultsEmail } from './emailService.js';
|
import { sendScraperResultsEmail } from './emailService.js';
|
||||||
|
import { runAgentTask } from './agentService.js';
|
||||||
|
import { initResultsStore, loadConfig, appendResult } from './resultStore.js';
|
||||||
|
|
||||||
const __filename = fileURLToPath(import.meta.url);
|
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
|
||||||
const __dirname = dirname(__filename);
|
|
||||||
|
|
||||||
// 初始化 Firecrawl 客户端
|
let currentScheduledTask = null;
|
||||||
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
|
|
||||||
|
|
||||||
const RESULTS_PATH = join(__dirname, '..', 'results.json');
|
function normalizeTaskMode(value) {
|
||||||
|
if (typeof value === 'string' && value.trim()) return value.trim();
|
||||||
// 加载配置文件
|
return DEFAULT_TASK_MODE;
|
||||||
function loadConfig() {
|
|
||||||
try {
|
|
||||||
const configPath = join(__dirname, '..', 'config.json');
|
|
||||||
return JSON.parse(readFileSync(configPath, 'utf-8'));
|
|
||||||
} catch (error) {
|
|
||||||
console.error('加载配置文件失败:', error.message);
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ========== 结果存取（与 server.js 保持一致） ==========
|
async function runTask(task, agentCfg) {
|
||||||
|
const mode = normalizeTaskMode(task.mode);
|
||||||
|
|
||||||
function readResults() {
|
console.log(`[Scheduler][Agent] ${task.city}: start`);
|
||||||
if (!existsSync(RESULTS_PATH)) return [];
|
console.log(`[Scheduler][Agent] ${task.city}: mode=${mode}`);
|
||||||
try {
|
|
||||||
return JSON.parse(readFileSync(RESULTS_PATH, 'utf-8'));
|
|
||||||
} catch (e) {
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function saveResults(results) {
|
const { results } = await runAgentTask(task.prompt, {
|
||||||
writeFileSync(RESULTS_PATH, JSON.stringify(results, null, 2), 'utf-8');
|
baseUrl: agentCfg.baseUrl,
|
||||||
}
|
mode,
|
||||||
|
pollInterval: agentCfg.pollInterval,
|
||||||
function appendResult(result) {
|
timeout: agentCfg.timeout,
|
||||||
const results = readResults();
|
logPrefix: `[Scheduler][Agent][${task.city}]`,
|
||||||
results.unshift({ ...result, id: `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}` });
|
|
||||||
if (results.length > 500) results.splice(500);
|
|
||||||
saveResults(results);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ========== 统一的公告抓取 Schema ==========
|
|
||||||
|
|
||||||
// 公告抓取 Schema（result 包装数组）
|
|
||||||
const announcementSchema = z.object({
|
|
||||||
result: z.array(z.object({
|
|
||||||
title: z.string().describe('公告标题'),
|
|
||||||
amount: z.string().nullable().describe('项目金额（合同预估价/最高投标限价等），没有则为null'),
|
|
||||||
date: z.string().describe('发布日期，YYYY-MM-DD格式'),
|
|
||||||
url: z.string().describe('详情页完整URL，以https://开头'),
|
|
||||||
})).describe('页面上提取到的所有公告条目'),
|
|
||||||
});
|
|
||||||
|
|
||||||
/** 从 Firecrawl 返回结果中提取 result 数组 */
|
|
||||||
function extractItems(raw) {
|
|
||||||
if (!raw) return [];
|
|
||||||
const root = (raw.data && typeof raw.data === 'object') ? raw.data : raw;
|
|
||||||
if (Array.isArray(root.result)) return root.result;
|
|
||||||
if (root.result && typeof root.result === 'object') {
|
|
||||||
const keys = Object.keys(root.result).filter(k => !isNaN(parseInt(k)));
|
|
||||||
if (keys.length > 0) return keys.sort((a, b) => parseInt(a) - parseInt(b)).map(k => root.result[k]);
|
|
||||||
}
|
|
||||||
if (Array.isArray(root)) return root;
|
|
||||||
const numericKeys = Object.keys(root).filter(k => !isNaN(parseInt(k)));
|
|
||||||
if (numericKeys.length > 0) return numericKeys.sort((a, b) => parseInt(a) - parseInt(b)).map(k => root[k]);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
|
|
||||||
// ========== 抓取执行（复用 server.js 中 runScraper 的逻辑） ==========
|
|
||||||
|
|
||||||
async function runScraper(scraper) {
|
|
||||||
console.log(`[定时任务] ${scraper.city} - ${scraper.section} ${scraper.subsection} - ${scraper.type}：${scraper.url}`);
|
|
||||||
const fullPrompt = `访问这个URL: ${scraper.url}
|
|
||||||
【目标区域】：${scraper.section || ''} - ${scraper.subsection || ''}
|
|
||||||
【公告类型】：${scraper.type || ''}
|
|
||||||
|
|
||||||
${scraper.prompt || '提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL'}
|
|
||||||
|
|
||||||
请严格按照定义的 JSON 格式返回，每条公告包含 title、amount、date、url 四个字段。`;
|
|
||||||
|
|
||||||
const result = await firecrawl.agent({
|
|
||||||
prompt: fullPrompt,
|
|
||||||
schema: announcementSchema,
|
|
||||||
model: scraper.model || 'spark-1-mini',
|
|
||||||
});
|
});
|
||||||
|
|
||||||
console.log('[定时任务] 原始返回结果:', JSON.stringify(result).slice(0, 500));
|
console.log(`[Scheduler][Agent] ${task.city}: ${results.length} results`);
|
||||||
|
|
||||||
// 标准化结果
|
return appendResult({
|
||||||
const rawItems = extractItems(result);
|
taskId: task.id,
|
||||||
const items = rawItems.map(item => ({
|
city: task.city,
|
||||||
title: item.title || '',
|
|
||||||
amount: item.amount || null,
|
|
||||||
date: item.date || '',
|
|
||||||
url: item.url || '',
|
|
||||||
}));
|
|
||||||
|
|
||||||
console.log(`[定时任务] 提取到 ${items.length} 条公告`);
|
|
||||||
|
|
||||||
const record = {
|
|
||||||
scraperId: scraper.id,
|
|
||||||
city: scraper.city,
|
|
||||||
section: scraper.section,
|
|
||||||
subsection: scraper.subsection,
|
|
||||||
type: scraper.type,
|
|
||||||
url: scraper.url,
|
|
||||||
scrapedAt: new Date().toISOString(),
|
scrapedAt: new Date().toISOString(),
|
||||||
data: { result: items, total: items.length },
|
data: { results, total: results.length },
|
||||||
};
|
});
|
||||||
appendResult(record);
|
|
||||||
return record;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ========== 定时任务执行函数 ==========
|
|
||||||
|
|
||||||
async function executeScheduledTask(config) {
|
async function executeScheduledTask(config) {
|
||||||
try {
|
try {
|
||||||
console.log('========================================');
|
console.log('========================================');
|
||||||
console.log('定时任务开始执行');
|
console.log('Scheduler started');
|
||||||
console.log('执行时间:', new Date().toLocaleString('zh-CN'));
|
console.log('Time:', new Date().toLocaleString('zh-CN'));
|
||||||
console.log('========================================');
|
console.log('========================================');
|
||||||
|
|
||||||
// 获取所有已启用的抓取来源
|
const tasks = (config.tasks || []).filter((task) => task.enabled);
|
||||||
const scrapers = (config.scrapers || []).filter(s => s.enabled);
|
const agentCfg = config.agent || {};
|
||||||
|
|
||||||
if (scrapers.length === 0) {
|
if (tasks.length === 0) {
|
||||||
console.log('没有已启用的抓取来源，跳过');
|
console.log('No enabled tasks, skip');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
console.log(`共 ${scrapers.length} 个已启用的抓取来源`);
|
console.log(`Enabled tasks: ${tasks.length}`);
|
||||||
|
|
||||||
// 逐个运行抓取任务
|
|
||||||
const results = [];
|
const results = [];
|
||||||
for (const scraper of scrapers) {
|
for (const task of tasks) {
|
||||||
try {
|
try {
|
||||||
console.log(`\n---------- 抓取: ${scraper.city} - ${scraper.section} ${scraper.type} ----------`);
|
console.log(`\n---------- Task: ${task.city} ----------`);
|
||||||
const r = await runScraper(scraper);
|
const record = await runTask(task, agentCfg);
|
||||||
results.push(r);
|
results.push(record);
|
||||||
console.log(`✓ 抓取成功`);
|
console.log('Task completed');
|
||||||
} catch (err) {
|
} catch (error) {
|
||||||
console.error(`✗ 抓取失败: ${err.message}`);
|
console.error(`Task failed: ${error.message}`);
|
||||||
const errRecord = {
|
const errorRecord = appendResult({
|
||||||
scraperId: scraper.id,
|
taskId: task.id,
|
||||||
city: scraper.city,
|
city: task.city,
|
||||||
section: scraper.section,
|
|
||||||
subsection: scraper.subsection,
|
|
||||||
type: scraper.type,
|
|
||||||
url: scraper.url,
|
|
||||||
scrapedAt: new Date().toISOString(),
|
scrapedAt: new Date().toISOString(),
|
||||||
error: err.message,
|
error: error.message,
|
||||||
data: null,
|
data: null,
|
||||||
};
|
});
|
||||||
appendResult(errRecord);
|
results.push(errorRecord);
|
||||||
results.push(errRecord);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const successCount = results.filter(r => !r.error).length;
|
const successCount = results.filter((item) => !item.error).length;
|
||||||
const failCount = results.filter(r => r.error).length;
|
const failCount = results.filter((item) => item.error).length;
|
||||||
console.log(`\n========== 抓取完成 ==========`);
|
console.log('\n========== Scheduler finished ==========');
|
||||||
console.log(`成功: ${successCount} 条，失败: ${failCount} 条`);
|
console.log(`Success: ${successCount}, Failed: ${failCount}`);
|
||||||
|
|
||||||
// 检查是否需要发送邮件
|
|
||||||
if (successCount === 0) {
|
if (successCount === 0) {
|
||||||
console.log('没有成功的抓取结果，不发送邮件');
|
console.log('No successful results, skip email');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// 发送邮件报告
|
|
||||||
if (config.email?.smtpHost && config.email?.smtpUser) {
|
if (config.email?.smtpHost && config.email?.smtpUser) {
|
||||||
console.log('\n正在发送抓取结果邮件...');
|
console.log('\nSending email...');
|
||||||
try {
|
try {
|
||||||
const emailResult = await sendScraperResultsEmail(config.email, results);
|
const emailResult = await sendScraperResultsEmail(config.email, results);
|
||||||
console.log('邮件发送成功! MessageId:', emailResult.messageId);
|
console.log('Email sent:', emailResult.messageId);
|
||||||
} catch (emailErr) {
|
} catch (error) {
|
||||||
console.error('邮件发送失败:', emailErr.message);
|
console.error('Email failed:', error.message);
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
console.log('邮件配置不完整，跳过邮件发送');
|
console.log('Email config incomplete, skip email');
|
||||||
}
|
}
|
||||||
|
|
||||||
console.log('========================================');
|
console.log('========================================');
|
||||||
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('========================================');
|
console.error('========================================');
|
||||||
console.error('定时任务执行失败:', error.message);
|
console.error('Scheduler failed:', error.message);
|
||||||
console.error(error.stack);
|
console.error(error.stack);
|
||||||
console.error('========================================');
|
console.error('========================================');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 存储当前的定时任务
|
|
||||||
let currentScheduledTask = null;
|
|
||||||
|
|
||||||
export function initScheduler() {
|
export function initScheduler() {
|
||||||
|
initResultsStore();
|
||||||
|
|
||||||
const config = loadConfig();
|
const config = loadConfig();
|
||||||
if (!config) { console.error('无法启动定时任务: 配置文件加载失败'); return; }
|
if (!config.scheduler?.enabled) {
|
||||||
if (!config.scheduler?.enabled) { console.log('定时任务已禁用'); return; }
|
console.log('Scheduler disabled');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
const cronTime = config.scheduler.cronTime || '0 9 * * *';
|
const cronTime = config.scheduler.cronTime || '0 9 * * *';
|
||||||
const enabledCount = (config.scrapers || []).filter(s => s.enabled).length;
|
const enabledCount = (config.tasks || []).filter((task) => task.enabled).length;
|
||||||
|
|
||||||
console.log('========================================');
|
console.log('========================================');
|
||||||
console.log('定时任务已启动，执行计划:', cronTime);
|
console.log('Scheduler enabled:', cronTime);
|
||||||
console.log(`已启用的抓取来源: ${enabledCount} 个`);
|
console.log(`Enabled tasks: ${enabledCount}`);
|
||||||
if (config.email?.recipients) console.log('收件人:', config.email.recipients);
|
if (config.email?.recipients) {
|
||||||
|
console.log('Recipients:', config.email.recipients);
|
||||||
|
}
|
||||||
console.log('========================================');
|
console.log('========================================');
|
||||||
|
|
||||||
if (currentScheduledTask) { currentScheduledTask.stop(); }
|
if (currentScheduledTask) {
|
||||||
|
currentScheduledTask.stop();
|
||||||
|
}
|
||||||
|
|
||||||
currentScheduledTask = cron.schedule(cronTime, () => {
|
currentScheduledTask = cron.schedule(
|
||||||
// 每次执行时重新加载配置，确保使用最新的 scrapers
|
cronTime,
|
||||||
const latestConfig = loadConfig();
|
() => {
|
||||||
if (latestConfig) {
|
executeScheduledTask(loadConfig());
|
||||||
executeScheduledTask(latestConfig);
|
},
|
||||||
}
|
{ timezone: 'Asia/Shanghai' },
|
||||||
}, { timezone: 'Asia/Shanghai' });
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
export function reloadScheduler() {
|
export function reloadScheduler() {
|
||||||
console.log('重新加载定时任务配置...');
|
console.log('Reloading scheduler...');
|
||||||
if (currentScheduledTask) { currentScheduledTask.stop(); currentScheduledTask = null; }
|
if (currentScheduledTask) {
|
||||||
|
currentScheduledTask.stop();
|
||||||
|
currentScheduledTask = null;
|
||||||
|
}
|
||||||
initScheduler();
|
initScheduler();
|
||||||
}
|
}
|
||||||
|
|
||||||
export function stopScheduler() {
|
export function stopScheduler() {
|
||||||
if (currentScheduledTask) {
|
if (!currentScheduledTask) return false;
|
||||||
currentScheduledTask.stop(); currentScheduledTask = null;
|
|
||||||
console.log('定时任务已停止'); return true;
|
currentScheduledTask.stop();
|
||||||
}
|
currentScheduledTask = null;
|
||||||
return false;
|
console.log('Scheduler stopped');
|
||||||
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function getSchedulerStatus() {
|
export function getSchedulerStatus() {
|
||||||
const config = loadConfig();
|
const config = loadConfig();
|
||||||
const enabledScrapers = (config?.scrapers || []).filter(s => s.enabled).length;
|
const enabledTasks = (config.tasks || []).filter((task) => task.enabled).length;
|
||||||
|
|
||||||
return {
|
return {
|
||||||
isRunning: currentScheduledTask !== null,
|
isRunning: currentScheduledTask !== null,
|
||||||
enabledScrapers,
|
enabledTasks,
|
||||||
config: config ? {
|
config: {
|
||||||
enabled: config.scheduler?.enabled || false,
|
enabled: config.scheduler?.enabled || false,
|
||||||
cronTime: config.scheduler?.cronTime || '0 9 * * *',
|
cronTime: config.scheduler?.cronTime || '0 9 * * *',
|
||||||
description: config.scheduler?.description || '',
|
description: config.scheduler?.description || '',
|
||||||
} : null,
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function runTaskNow() {
|
export async function runTaskNow() {
|
||||||
const config = loadConfig();
|
initResultsStore();
|
||||||
if (!config) throw new Error('配置文件加载失败');
|
await executeScheduledTask(loadConfig());
|
||||||
await executeScheduledTask(config);
|
|
||||||
}
|
}
|
||||||
|
|||||||
692
src/server.js
692
src/server.js
@@ -1,378 +1,474 @@
|
|||||||
import 'dotenv/config';
|
import 'dotenv/config';
|
||||||
import express from 'express';
|
import express from 'express';
|
||||||
import cors from 'cors';
|
import cors from 'cors';
|
||||||
import Firecrawl from '@mendable/firecrawl-js';
|
import { existsSync } from 'node:fs';
|
||||||
import { z } from 'zod';
|
import { dirname, join } from 'node:path';
|
||||||
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { fileURLToPath } from 'url';
|
import {
|
||||||
import { dirname, join } from 'path';
|
initResultsStore,
|
||||||
import { sendCombinedReportEmail } from './emailService.js';
|
loadConfig,
|
||||||
|
saveConfig,
|
||||||
|
listTasks,
|
||||||
|
getTaskById,
|
||||||
|
createTask,
|
||||||
|
updateTask,
|
||||||
|
deleteTaskById,
|
||||||
|
listResults,
|
||||||
|
listProjects,
|
||||||
|
deleteResultById,
|
||||||
|
clearResults,
|
||||||
|
getResultFilters,
|
||||||
|
getProjectFilters,
|
||||||
|
appendResult,
|
||||||
|
} from './resultStore.js';
|
||||||
import { initScheduler, runTaskNow, reloadScheduler, getSchedulerStatus } from './scheduler.js';
|
import { initScheduler, runTaskNow, reloadScheduler, getSchedulerStatus } from './scheduler.js';
|
||||||
|
import { runAgentTask } from './agentService.js';
|
||||||
const app = express();
|
|
||||||
const PORT = process.env.PORT || 5000;
|
|
||||||
|
|
||||||
app.use(cors());
|
|
||||||
app.use(express.json());
|
|
||||||
app.use(express.static('public'));
|
|
||||||
|
|
||||||
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
|
|
||||||
|
|
||||||
const __filename = fileURLToPath(import.meta.url);
|
const __filename = fileURLToPath(import.meta.url);
|
||||||
const __dirname = dirname(__filename);
|
const __dirname = dirname(__filename);
|
||||||
const CONFIG_PATH = join(__dirname, '..', 'config.json');
|
const DIST_DIR = join(__dirname, '..', 'dist');
|
||||||
const RESULTS_PATH = join(__dirname, '..', 'results.json');
|
const DIST_INDEX = join(DIST_DIR, 'index.html');
|
||||||
|
const HAS_DIST = existsSync(DIST_INDEX);
|
||||||
|
|
||||||
function readConfig() {
|
const app = express();
|
||||||
return JSON.parse(readFileSync(CONFIG_PATH, 'utf-8'));
|
const PORT = process.env.PORT || 5000;
|
||||||
|
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
|
||||||
|
const MASKED_PASSWORD = '***已配置***';
|
||||||
|
|
||||||
|
app.use(cors());
|
||||||
|
app.use(express.json({ limit: '50mb' }));
|
||||||
|
app.use(express.urlencoded({ limit: '50mb', extended: true }));
|
||||||
|
|
||||||
|
if (HAS_DIST) {
|
||||||
|
app.use(express.static(DIST_DIR));
|
||||||
}
|
}
|
||||||
|
|
||||||
function saveConfig(cfg) {
|
function normalizeTaskMode(value) {
|
||||||
writeFileSync(CONFIG_PATH, JSON.stringify(cfg, null, 2), 'utf-8');
|
if (typeof value === 'string' && value.trim()) return value.trim();
|
||||||
|
return DEFAULT_TASK_MODE;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ========== 抓取结果存取 ==========
|
function buildTaskPayload(body = {}, { partial = false } = {}) {
|
||||||
|
const payload = {};
|
||||||
|
|
||||||
function readResults() {
|
if (!partial || Object.prototype.hasOwnProperty.call(body, 'city')) {
|
||||||
if (!existsSync(RESULTS_PATH)) return [];
|
payload.city = body.city || '';
|
||||||
try {
|
|
||||||
return JSON.parse(readFileSync(RESULTS_PATH, 'utf-8'));
|
|
||||||
} catch (e) {
|
|
||||||
return [];
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (!partial || Object.prototype.hasOwnProperty.call(body, 'plateName')) {
|
||||||
|
payload.plateName = body.plateName || '';
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!partial || Object.prototype.hasOwnProperty.call(body, 'prompt')) {
|
||||||
|
payload.prompt = body.prompt || '';
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!partial || Object.prototype.hasOwnProperty.call(body, 'enabled')) {
|
||||||
|
payload.enabled = body.enabled !== false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!partial || Object.prototype.hasOwnProperty.call(body, 'mode')) {
|
||||||
|
payload.mode = normalizeTaskMode(body.mode);
|
||||||
|
}
|
||||||
|
|
||||||
|
return payload;
|
||||||
}
|
}
|
||||||
|
|
||||||
function saveResults(results) {
|
function maskConfigSecrets(config) {
|
||||||
writeFileSync(RESULTS_PATH, JSON.stringify(results, null, 2), 'utf-8');
|
const next = { ...config };
|
||||||
}
|
|
||||||
|
|
||||||
function appendResult(result) {
|
if (config.email) {
|
||||||
const results = readResults();
|
next.email = {
|
||||||
results.unshift({ ...result, id: `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}` });
|
...config.email,
|
||||||
// 最多保留 500 条
|
smtpPass: config.email.smtpPass ? MASKED_PASSWORD : '',
|
||||||
if (results.length > 500) results.splice(500);
|
|
||||||
saveResults(results);
|
|
||||||
}
|
|
||||||
|
|
||||||
// 查询结果（支持分页与筛选）
|
|
||||||
app.get('/api/results', (req, res) => {
|
|
||||||
try {
|
|
||||||
const { city, type, section, page = 1, pageSize = 20, scraperId } = req.query;
|
|
||||||
let results = readResults();
|
|
||||||
if (city) results = results.filter(r => r.city === city);
|
|
||||||
if (type) results = results.filter(r => r.type === type);
|
|
||||||
if (section) results = results.filter(r => r.section === section);
|
|
||||||
if (scraperId) results = results.filter(r => r.scraperId === scraperId);
|
|
||||||
const total = results.length;
|
|
||||||
const start = (parseInt(page) - 1) * parseInt(pageSize);
|
|
||||||
const data = results.slice(start, start + parseInt(pageSize));
|
|
||||||
res.json({ success: true, total, page: parseInt(page), pageSize: parseInt(pageSize), data });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// 删除单条结果
|
|
||||||
app.delete('/api/results/:id', (req, res) => {
|
|
||||||
try {
|
|
||||||
const results = readResults();
|
|
||||||
const before = results.length;
|
|
||||||
const updated = results.filter(r => r.id !== req.params.id);
|
|
||||||
if (updated.length === before) return res.status(404).json({ success: false, error: '未找到' });
|
|
||||||
saveResults(updated);
|
|
||||||
res.json({ success: true });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// 清空所有结果
|
|
||||||
app.delete('/api/results', (req, res) => {
|
|
||||||
try {
|
|
||||||
saveResults([]);
|
|
||||||
res.json({ success: true, message: '已清空所有结果' });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// 获取结果的筛选选项（城市/板块/类型下拉枚举）
|
|
||||||
app.get('/api/results/filters', (req, res) => {
|
|
||||||
try {
|
|
||||||
const results = readResults();
|
|
||||||
const cities = [...new Set(results.map(r => r.city).filter(Boolean))];
|
|
||||||
const sections = [...new Set(results.map(r => r.section).filter(Boolean))];
|
|
||||||
const types = [...new Set(results.map(r => r.type).filter(Boolean))];
|
|
||||||
res.json({ success: true, data: { cities, sections, types } });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// ========== 抓取来源 CRUD ==========
|
|
||||||
|
|
||||||
app.get('/api/scrapers', (req, res) => {
|
|
||||||
try {
|
|
||||||
const cfg = readConfig();
|
|
||||||
res.json({ success: true, data: cfg.scrapers || [] });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
app.post('/api/scrapers', (req, res) => {
|
|
||||||
try {
|
|
||||||
const cfg = readConfig();
|
|
||||||
if (!cfg.scrapers) cfg.scrapers = [];
|
|
||||||
const item = {
|
|
||||||
id: `scraper-${Date.now()}`,
|
|
||||||
city: req.body.city || '',
|
|
||||||
url: req.body.url || '',
|
|
||||||
section: req.body.section || '',
|
|
||||||
subsection: req.body.subsection || '',
|
|
||||||
type: req.body.type || '招标公告',
|
|
||||||
prompt: req.body.prompt || '',
|
|
||||||
enabled: req.body.enabled !== false,
|
|
||||||
model: req.body.model || 'spark-1-mini',
|
|
||||||
};
|
};
|
||||||
cfg.scrapers.push(item);
|
|
||||||
saveConfig(cfg);
|
|
||||||
res.json({ success: true, data: item });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
}
|
||||||
});
|
|
||||||
|
|
||||||
app.put('/api/scrapers/:id', (req, res) => {
|
return next;
|
||||||
try {
|
|
||||||
const cfg = readConfig();
|
|
||||||
const idx = (cfg.scrapers || []).findIndex(s => s.id === req.params.id);
|
|
||||||
if (idx === -1) return res.status(404).json({ success: false, error: '未找到该配置' });
|
|
||||||
cfg.scrapers[idx] = { ...cfg.scrapers[idx], ...req.body, id: req.params.id };
|
|
||||||
saveConfig(cfg);
|
|
||||||
res.json({ success: true, data: cfg.scrapers[idx] });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
app.delete('/api/scrapers/:id', (req, res) => {
|
|
||||||
try {
|
|
||||||
const cfg = readConfig();
|
|
||||||
const before = (cfg.scrapers || []).length;
|
|
||||||
cfg.scrapers = (cfg.scrapers || []).filter(s => s.id !== req.params.id);
|
|
||||||
if (cfg.scrapers.length === before) return res.status(404).json({ success: false, error: '未找到' });
|
|
||||||
saveConfig(cfg);
|
|
||||||
res.json({ success: true });
|
|
||||||
} catch (e) {
|
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// ========== 统一抓取执行 ==========
|
|
||||||
|
|
||||||
// 公告抓取 Schema（result 包装数组）
|
|
||||||
const announcementSchema = z.object({
|
|
||||||
result: z.array(z.object({
|
|
||||||
title: z.string().describe('公告标题'),
|
|
||||||
amount: z.string().nullable().describe('项目金额（合同预估价/最高投标限价等），没有则为null'),
|
|
||||||
date: z.string().describe('发布日期，YYYY-MM-DD格式'),
|
|
||||||
url: z.string().describe('详情页完整URL，以https://开头'),
|
|
||||||
})).describe('页面上提取到的所有公告条目'),
|
|
||||||
});
|
|
||||||
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 从 Firecrawl agent 返回结果中提取 result 数组
|
|
||||||
* 优先取 root.result，再回退数字键处理
|
|
||||||
*/
|
|
||||||
function extractItems(raw) {
|
|
||||||
if (!raw) return [];
|
|
||||||
const root = (raw.data && typeof raw.data === 'object') ? raw.data : raw;
|
|
||||||
// 最优先：result 是真正数组
|
|
||||||
if (Array.isArray(root.result)) return root.result;
|
|
||||||
// result 是数字键对象
|
|
||||||
if (root.result && typeof root.result === 'object') {
|
|
||||||
const keys = Object.keys(root.result).filter(k => !isNaN(parseInt(k)));
|
|
||||||
if (keys.length > 0) return keys.sort((a, b) => parseInt(a) - parseInt(b)).map(k => root.result[k]);
|
|
||||||
}
|
|
||||||
// 如果 root 本身是数组
|
|
||||||
if (Array.isArray(root)) return root;
|
|
||||||
// 顶层数字键回退
|
|
||||||
const numericKeys = Object.keys(root).filter(k => !isNaN(parseInt(k)));
|
|
||||||
if (numericKeys.length > 0) return numericKeys.sort((a, b) => parseInt(a) - parseInt(b)).map(k => root[k]);
|
|
||||||
return [];
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// 执行单个抓取来源并保存结果
|
function mergeConfigWithExistingSecrets(incoming = {}) {
|
||||||
async function runScraper(scraper) {
|
const current = loadConfig();
|
||||||
console.log(`[Agent] ${scraper.city} - ${scraper.section} ${scraper.subsection} - ${scraper.type}：${scraper.url}`);
|
const next = {
|
||||||
const fullPrompt = `访问这个URL: ${scraper.url}
|
...current,
|
||||||
【目标区域】：${scraper.section || ''} - ${scraper.subsection || ''}
|
...incoming,
|
||||||
【公告类型】：${scraper.type || ''}
|
agent: { ...(current.agent || {}), ...(incoming.agent || {}) },
|
||||||
|
scheduler: { ...(current.scheduler || {}), ...(incoming.scheduler || {}) },
|
||||||
|
email: { ...(current.email || {}), ...(incoming.email || {}) },
|
||||||
|
};
|
||||||
|
|
||||||
${scraper.prompt || '提取页面上今日的招标公告信息，包括：标题、项目金额（可能为合同预估价/最高投标限价等等）、发布日期（YYYY-MM-DD格式）、详情页完整URL'}
|
if (next.email?.smtpPass === MASKED_PASSWORD) {
|
||||||
|
next.email.smtpPass = current.email?.smtpPass || '';
|
||||||
|
}
|
||||||
|
|
||||||
请严格按照定义的 JSON 格式返回，每条公告包含 title、amount、date、url 四个字段。`;
|
return next;
|
||||||
console.log(fullPrompt, 'fullPrompt=======');
|
}
|
||||||
|
|
||||||
const result = await firecrawl.agent({
|
let isRunning = false;
|
||||||
prompt: fullPrompt,
|
let runningStatus = null;
|
||||||
schema: announcementSchema,
|
|
||||||
model: scraper.model || 'spark-1-mini',
|
async function runTask(task) {
|
||||||
|
const config = loadConfig();
|
||||||
|
const agentCfg = config.agent || {};
|
||||||
|
const mode = normalizeTaskMode(task.mode);
|
||||||
|
|
||||||
|
console.log(`[Agent] ${task.city}: start`);
|
||||||
|
console.log(`[Agent] ${task.city}: mode=${mode}`);
|
||||||
|
|
||||||
|
const { results } = await runAgentTask(task.prompt, {
|
||||||
|
baseUrl: agentCfg.baseUrl,
|
||||||
|
mode,
|
||||||
|
pollInterval: agentCfg.pollInterval,
|
||||||
|
timeout: agentCfg.timeout,
|
||||||
|
logPrefix: `[Agent][${task.city}]`,
|
||||||
});
|
});
|
||||||
|
|
||||||
console.log('[Agent] 原始返回结果:', JSON.stringify(result).slice(0, 500));
|
return appendResult({
|
||||||
|
taskId: task.id,
|
||||||
const rawItems = extractItems(result);
|
city: task.city,
|
||||||
const items = rawItems.map(item => ({
|
|
||||||
title: item.title || '',
|
|
||||||
amount: item.amount || null,
|
|
||||||
date: item.date || '',
|
|
||||||
url: item.url || '',
|
|
||||||
}));
|
|
||||||
|
|
||||||
console.log(`[Agent] 提取到 ${items.length} 条公告`);
|
|
||||||
|
|
||||||
const record = {
|
|
||||||
scraperId: scraper.id,
|
|
||||||
city: scraper.city,
|
|
||||||
section: scraper.section,
|
|
||||||
subsection: scraper.subsection,
|
|
||||||
type: scraper.type,
|
|
||||||
url: scraper.url,
|
|
||||||
scrapedAt: new Date().toISOString(),
|
scrapedAt: new Date().toISOString(),
|
||||||
data: { result: items, total: items.length }, // 统一为 result 字段
|
data: { results, total: results.length },
|
||||||
};
|
});
|
||||||
appendResult(record);
|
|
||||||
return record;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// 运行指定 ID 的抓取来源（单条测试）
|
function runTaskInBackground(task) {
|
||||||
app.post('/api/scrapers/:id/run', async (req, res) => {
|
runningStatus = {
|
||||||
try {
|
taskId: task.id,
|
||||||
const cfg = readConfig();
|
city: task.city,
|
||||||
const scraper = (cfg.scrapers || []).find(s => s.id === req.params.id);
|
startTime: Date.now(),
|
||||||
if (!scraper) return res.status(404).json({ success: false, error: '未找到该配置' });
|
current: 0,
|
||||||
const result = await runScraper(scraper);
|
total: 1,
|
||||||
res.json({ success: true, data: result });
|
finished: false,
|
||||||
} catch (e) {
|
error: null,
|
||||||
console.error('测试抓取失败:', e.message);
|
};
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// 批量运行多个抓取来源
|
runTask(task)
|
||||||
// body: { ids: ['id1','id2',...] } 不传则运行所有已启用的
|
.then((record) => {
|
||||||
app.post('/api/scrape/run', async (req, res) => {
|
runningStatus = { ...runningStatus, finished: true, result: record, current: 1 };
|
||||||
try {
|
})
|
||||||
const cfg = readConfig();
|
.catch((error) => {
|
||||||
let scrapers = cfg.scrapers || [];
|
console.error('Task failed:', error.message);
|
||||||
|
runningStatus = { ...runningStatus, finished: true, error: error.message, current: 1 };
|
||||||
|
})
|
||||||
|
.finally(() => {
|
||||||
|
isRunning = false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
if (req.body.ids && req.body.ids.length > 0) {
|
function runTasksInBackground(tasks) {
|
||||||
scrapers = scrapers.filter(s => req.body.ids.includes(s.id));
|
runningStatus = {
|
||||||
} else {
|
taskId: null,
|
||||||
scrapers = scrapers.filter(s => s.enabled);
|
city: null,
|
||||||
}
|
startTime: Date.now(),
|
||||||
|
current: 0,
|
||||||
|
total: tasks.length,
|
||||||
|
finished: false,
|
||||||
|
error: null,
|
||||||
|
results: [],
|
||||||
|
};
|
||||||
|
|
||||||
if (scrapers.length === 0) {
|
(async () => {
|
||||||
return res.json({ success: true, data: [], message: '没有可运行的抓取来源' });
|
for (const task of tasks) {
|
||||||
}
|
runningStatus = { ...runningStatus, taskId: task.id, city: task.city };
|
||||||
|
|
||||||
const results = [];
|
|
||||||
for (const scraper of scrapers) {
|
|
||||||
try {
|
try {
|
||||||
const r = await runScraper(scraper);
|
const record = await runTask(task);
|
||||||
results.push(r);
|
runningStatus.results.push(record);
|
||||||
} catch (err) {
|
} catch (error) {
|
||||||
const errRecord = {
|
const errorRecord = appendResult({
|
||||||
scraperId: scraper.id,
|
taskId: task.id,
|
||||||
city: scraper.city,
|
city: task.city,
|
||||||
section: scraper.section,
|
|
||||||
subsection: scraper.subsection,
|
|
||||||
type: scraper.type,
|
|
||||||
url: scraper.url,
|
|
||||||
scrapedAt: new Date().toISOString(),
|
scrapedAt: new Date().toISOString(),
|
||||||
error: err.message,
|
error: error.message,
|
||||||
data: null,
|
data: null,
|
||||||
};
|
});
|
||||||
appendResult(errRecord);
|
runningStatus.results.push(errorRecord);
|
||||||
results.push(errRecord);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
runningStatus.current += 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
res.json({ success: true, data: results });
|
runningStatus.finished = true;
|
||||||
} catch (e) {
|
})()
|
||||||
res.status(500).json({ success: false, error: e.message });
|
.catch((error) => {
|
||||||
|
runningStatus = { ...runningStatus, finished: true, error: error.message };
|
||||||
|
})
|
||||||
|
.finally(() => {
|
||||||
|
isRunning = false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
app.get('/api/results', (req, res) => {
|
||||||
|
try {
|
||||||
|
const { city, section, type, page = 1, pageSize = 20, taskId, view } = req.query;
|
||||||
|
const projectMode = view === 'projects';
|
||||||
|
const result = listResults({ city, section, type, page, pageSize, taskId, projectMode });
|
||||||
|
res.json({ success: true, ...result });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// ========== 配置管理 ==========
|
app.delete('/api/results/:id', (req, res) => {
|
||||||
|
|
||||||
app.get('/api/config', (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
const cfg = readConfig();
|
const deleted = deleteResultById(req.params.id);
|
||||||
if (cfg.email?.smtpPass) cfg.email.smtpPass = '***已配置***';
|
if (!deleted) {
|
||||||
res.json({ success: true, data: cfg });
|
return res.status(404).json({ success: false, error: '未找到该结果' });
|
||||||
} catch (e) {
|
}
|
||||||
res.status(500).json({ success: false, error: e.message });
|
|
||||||
|
res.json({ success: true });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/api/results', (_req, res) => {
|
||||||
|
try {
|
||||||
|
clearResults();
|
||||||
|
res.json({ success: true, message: '已清空所有结果' });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/results/filters', (req, res) => {
|
||||||
|
try {
|
||||||
|
const projectMode = req.query.view === 'projects';
|
||||||
|
res.json({ success: true, data: getResultFilters({ projectMode }) });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/projects', (req, res) => {
|
||||||
|
try {
|
||||||
|
const {
|
||||||
|
city,
|
||||||
|
section,
|
||||||
|
projectName,
|
||||||
|
minAmount,
|
||||||
|
maxAmount,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
page = 1,
|
||||||
|
pageSize = 20,
|
||||||
|
} = req.query;
|
||||||
|
|
||||||
|
const result = listProjects({
|
||||||
|
city,
|
||||||
|
section,
|
||||||
|
projectName,
|
||||||
|
minAmount,
|
||||||
|
maxAmount,
|
||||||
|
startDate,
|
||||||
|
endDate,
|
||||||
|
page,
|
||||||
|
pageSize,
|
||||||
|
dedupeByName: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
res.json({ success: true, ...result });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/projects/filters', (_req, res) => {
|
||||||
|
try {
|
||||||
|
res.json({ success: true, data: getProjectFilters({ dedupeByName: true }) });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/tasks', (_req, res) => {
|
||||||
|
try {
|
||||||
|
res.json({ success: true, data: listTasks() });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/tasks', (req, res) => {
|
||||||
|
try {
|
||||||
|
const task = createTask(buildTaskPayload(req.body));
|
||||||
|
res.json({ success: true, data: task });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.put('/api/tasks/:id', (req, res) => {
|
||||||
|
try {
|
||||||
|
const task = updateTask(req.params.id, buildTaskPayload(req.body, { partial: true }));
|
||||||
|
if (!task) {
|
||||||
|
return res.status(404).json({ success: false, error: '未找到该任务' });
|
||||||
|
}
|
||||||
|
|
||||||
|
res.json({ success: true, data: task });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/api/tasks/:id', (req, res) => {
|
||||||
|
try {
|
||||||
|
const deleted = deleteTaskById(req.params.id);
|
||||||
|
if (!deleted) {
|
||||||
|
return res.status(404).json({ success: false, error: '未找到该任务' });
|
||||||
|
}
|
||||||
|
|
||||||
|
res.json({ success: true });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/tasks/status', (_req, res) => {
|
||||||
|
if (!runningStatus) {
|
||||||
|
return res.json({ success: true, data: { isRunning: false } });
|
||||||
|
}
|
||||||
|
|
||||||
|
const elapsed = Math.round((Date.now() - runningStatus.startTime) / 1000);
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
data: {
|
||||||
|
isRunning,
|
||||||
|
elapsed,
|
||||||
|
city: runningStatus.city,
|
||||||
|
current: runningStatus.current,
|
||||||
|
total: runningStatus.total,
|
||||||
|
finished: runningStatus.finished,
|
||||||
|
error: runningStatus.error,
|
||||||
|
results: runningStatus.finished
|
||||||
|
? (runningStatus.results || (runningStatus.result ? [runningStatus.result] : []))
|
||||||
|
: undefined,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/tasks/:id/run', (req, res) => {
|
||||||
|
if (isRunning) {
|
||||||
|
return res.status(409).json({ success: false, error: '当前已有任务在运行，请稍后再试' });
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const task = getTaskById(req.params.id);
|
||||||
|
if (!task) {
|
||||||
|
return res.status(404).json({ success: false, error: '未找到该任务' });
|
||||||
|
}
|
||||||
|
|
||||||
|
isRunning = true;
|
||||||
|
runTaskInBackground(task);
|
||||||
|
res.json({ success: true, message: `任务“${task.city}”已开始执行` });
|
||||||
|
} catch (error) {
|
||||||
|
isRunning = false;
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/tasks/run', (req, res) => {
|
||||||
|
if (isRunning) {
|
||||||
|
return res.status(409).json({ success: false, error: '当前已有任务在运行，请稍后再试' });
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
let tasks = listTasks();
|
||||||
|
|
||||||
|
if (Array.isArray(req.body?.ids) && req.body.ids.length > 0) {
|
||||||
|
const idSet = new Set(req.body.ids);
|
||||||
|
tasks = tasks.filter((task) => idSet.has(task.id));
|
||||||
|
} else {
|
||||||
|
tasks = tasks.filter((task) => task.enabled);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (tasks.length === 0) {
|
||||||
|
return res.json({ success: true, data: [], message: '没有可运行的任务' });
|
||||||
|
}
|
||||||
|
|
||||||
|
isRunning = true;
|
||||||
|
runTasksInBackground(tasks);
|
||||||
|
res.json({ success: true, message: `${tasks.length} 个任务已开始执行` });
|
||||||
|
} catch (error) {
|
||||||
|
isRunning = false;
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/api/config', (_req, res) => {
|
||||||
|
try {
|
||||||
|
res.json({ success: true, data: maskConfigSecrets(loadConfig()) });
|
||||||
|
} catch (error) {
|
||||||
|
res.status(500).json({ success: false, error: error.message });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
app.post('/api/config', (req, res) => {
|
app.post('/api/config', (req, res) => {
|
||||||
try {
|
try {
|
||||||
const newCfg = req.body;
|
saveConfig(mergeConfigWithExistingSecrets(req.body));
|
||||||
const oldCfg = readConfig();
|
|
||||||
if (newCfg.email?.smtpPass === '***已配置***') {
|
|
||||||
newCfg.email.smtpPass = oldCfg.email?.smtpPass || '';
|
|
||||||
}
|
|
||||||
saveConfig(newCfg);
|
|
||||||
reloadScheduler();
|
reloadScheduler();
|
||||||
res.json({ success: true, message: '配置已保存' });
|
res.json({ success: true, message: '配置已保存' });
|
||||||
} catch (e) {
|
} catch (error) {
|
||||||
res.status(500).json({ success: false, error: e.message });
|
res.status(500).json({ success: false, error: error.message });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// ========== 邮件 ==========
|
|
||||||
|
|
||||||
app.post('/api/send-email', async (req, res) => {
|
app.post('/api/send-email', async (req, res) => {
|
||||||
try {
|
try {
|
||||||
const { emailConfig, report } = req.body;
|
const { emailConfig, report } = req.body;
|
||||||
if (!emailConfig?.smtpHost || !emailConfig?.smtpUser || !emailConfig?.smtpPass)
|
|
||||||
|
if (!emailConfig?.smtpHost || !emailConfig?.smtpUser || !emailConfig?.smtpPass) {
|
||||||
return res.status(400).json({ success: false, error: '邮件配置不完整' });
|
return res.status(400).json({ success: false, error: '邮件配置不完整' });
|
||||||
if (!emailConfig.recipients?.trim())
|
}
|
||||||
|
|
||||||
|
if (!emailConfig.recipients?.trim()) {
|
||||||
return res.status(400).json({ success: false, error: '请指定收件人' });
|
return res.status(400).json({ success: false, error: '请指定收件人' });
|
||||||
if (!report)
|
}
|
||||||
|
|
||||||
|
if (!report) {
|
||||||
return res.status(400).json({ success: false, error: '没有报告数据' });
|
return res.status(400).json({ success: false, error: '没有报告数据' });
|
||||||
|
}
|
||||||
|
|
||||||
const { sendReportEmail } = await import('./emailService.js');
|
const { sendReportEmail } = await import('./emailService.js');
|
||||||
const result = await sendReportEmail(emailConfig, report);
|
const result = await sendReportEmail(emailConfig, report);
|
||||||
res.json({ success: true, message: '邮件发送成功', messageId: result.messageId });
|
res.json({ success: true, message: '邮件发送成功', messageId: result.messageId });
|
||||||
} catch (e) {
|
} catch (error) {
|
||||||
res.status(500).json({ success: false, error: e.message });
|
res.status(500).json({ success: false, error: error.message });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// ========== 定时任务 ==========
|
app.get('/api/scheduler/status', (_req, res) => {
|
||||||
|
|
||||||
app.get('/api/scheduler/status', (req, res) => {
|
|
||||||
try {
|
try {
|
||||||
res.json({ success: true, data: getSchedulerStatus() });
|
res.json({ success: true, data: getSchedulerStatus() });
|
||||||
} catch (e) {
|
} catch (error) {
|
||||||
res.status(500).json({ success: false, error: e.message });
|
res.status(500).json({ success: false, error: error.message });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
app.post('/api/run-scheduled-task', (req, res) => {
|
app.post('/api/run-scheduled-task', (_req, res) => {
|
||||||
try {
|
try {
|
||||||
runTaskNow().catch(err => console.error('定时任务执行失败:', err));
|
runTaskNow().catch((error) => console.error('Scheduled task failed:', error));
|
||||||
res.json({ success: true, message: '定时任务已在后台触发' });
|
res.json({ success: true, message: '定时任务已在后台触发' });
|
||||||
} catch (e) {
|
} catch (error) {
|
||||||
res.status(500).json({ success: false, error: e.message });
|
res.status(500).json({ success: false, error: error.message });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
app.get('/results.html', (_req, res) => {
|
||||||
|
res.redirect('/results');
|
||||||
|
});
|
||||||
|
|
||||||
|
app.use((req, res, next) => {
|
||||||
|
if (!HAS_DIST) return next();
|
||||||
|
if (req.method !== 'GET') return next();
|
||||||
|
if (req.path.startsWith('/api')) return next();
|
||||||
|
if (req.path.includes('.')) return next();
|
||||||
|
|
||||||
|
res.sendFile(DIST_INDEX);
|
||||||
|
});
|
||||||
|
|
||||||
|
initResultsStore();
|
||||||
|
|
||||||
app.listen(PORT, () => {
|
app.listen(PORT, () => {
|
||||||
console.log(`Server running at http://localhost:${PORT}`);
|
console.log(`Server running at http://localhost:${PORT}`);
|
||||||
initScheduler();
|
initScheduler();
|
||||||
|
|||||||
49
vite.config.js
Normal file
49
vite.config.js
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
import { defineConfig } from 'vite';
|
||||||
|
import vue from '@vitejs/plugin-vue';
|
||||||
|
import { fileURLToPath, URL } from 'node:url';
|
||||||
|
import AutoImport from 'unplugin-auto-import/vite';
|
||||||
|
import Components from 'unplugin-vue-components/vite';
|
||||||
|
import { ElementPlusResolver } from 'unplugin-vue-components/resolvers';
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
root: 'client',
|
||||||
|
plugins: [
|
||||||
|
vue(),
|
||||||
|
AutoImport({
|
||||||
|
imports: ['vue', 'vue-router'],
|
||||||
|
resolvers: [ElementPlusResolver()],
|
||||||
|
}),
|
||||||
|
Components({
|
||||||
|
resolvers: [ElementPlusResolver()],
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
resolve: {
|
||||||
|
alias: {
|
||||||
|
'@': fileURLToPath(new URL('./client/src', import.meta.url)),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
server: {
|
||||||
|
host: '0.0.0.0',
|
||||||
|
port: 5173,
|
||||||
|
proxy: {
|
||||||
|
'/api': {
|
||||||
|
target: 'http://localhost:5000',
|
||||||
|
changeOrigin: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
build: {
|
||||||
|
outDir: '../dist',
|
||||||
|
emptyOutDir: true,
|
||||||
|
rollupOptions: {
|
||||||
|
output: {
|
||||||
|
manualChunks(id) {
|
||||||
|
if (!id.includes('node_modules')) return;
|
||||||
|
if (id.includes('element-plus')) return 'element-plus';
|
||||||
|
if (id.includes('/vue/') || id.includes('@vue')) return 'vue-vendor';
|
||||||
|
if (id.includes('axios')) return 'http-vendor';
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user