Compare commits
6 Commits
vue
...
agent_new3
| Author | SHA1 | Date | |
|---|---|---|---|
| 052f3a137b | |||
| 0d74cfe754 | |||
| 89d0abd44c | |||
| f8374f5e0d | |||
| f8dfad26a4 | |||
| 2a5fd99319 |
617
README.md
617
README.md
@@ -1,191 +1,536 @@
|
||||
# 公告抓取与分析工具
|
||||
# 南京公共工程建设中心 - 公告采集工具
|
||||
|
||||
一个用于管理公告抓取任务、查看抓取结果、查询项目数据并维护系统配置的工具。
|
||||
一个用于采集南京公共工程建设中心公告信息的 Web 可视化工具,支持定时任务和邮件推送。
|
||||
|
||||
当前项目结构已经升级为:
|
||||
## 功能特性
|
||||
|
||||
- 前端:Vue 3 + JavaScript + Vite + Vue Router + Axios + Element Plus
|
||||
- 后端:Node.js + Express
|
||||
- 数据存储:better-sqlite3
|
||||
- 调度:node-cron
|
||||
- 邮件:nodemailer
|
||||
- ✅ 采集公告列表(支持分页)
|
||||
- ✅ 按时间范围智能采集
|
||||
- ✅ 采集公告详情内容
|
||||
- ✅ 智能提取预算金额
|
||||
- ✅ 生成统计报告
|
||||
- ✅ Web 可视化界面
|
||||
- ✅ 导出 Word/Markdown 报告
|
||||
- ✅ RESTful API 支持
|
||||
- ✅ **定时任务自动采集**
|
||||
- ✅ **邮件推送 HTML 报告**
|
||||
- ✅ **Web 配置界面**
|
||||
- ✅ **无需数据库,轻量部署**
|
||||
|
||||
## 功能概览
|
||||
## 快速开始
|
||||
|
||||
- 任务管理:新增、编辑、启用、禁用、删除任务
|
||||
- 手动执行:支持单任务运行和批量运行
|
||||
- 结果查看:按城市、板块、类型筛选抓取结果
|
||||
- 项目查询:按项目名称、金额、日期范围过滤项目
|
||||
- 系统设置:维护 Agent、定时任务、邮件配置
|
||||
- 定时调度:支持 cron 表达式配置
|
||||
- 数据持久化:任务与结果保存在 SQLite
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Node.js 20 及以上可运行当前项目
|
||||
- 建议使用 Node.js 22
|
||||
|
||||
说明:
|
||||
|
||||
- 当前依赖 `@mendable/firecrawl-js` 会提示要求 Node `>=22`
|
||||
- 目前在 Node `20.19.4` 下可以安装、构建和启动
|
||||
- 如果后续采集运行时出现环境兼容问题,优先升级到 Node 22
|
||||
|
||||
## 安装依赖
|
||||
### 1. 安装依赖
|
||||
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
## 配置文件
|
||||
### 2. 配置文件
|
||||
|
||||
项目根目录下使用 `config.json` 作为运行配置文件。
|
||||
|
||||
如果你还没有该文件,可以参考:
|
||||
首次使用需要创建配置文件:
|
||||
|
||||
```bash
|
||||
# 复制示例配置文件
|
||||
cp config.example.json config.json
|
||||
|
||||
# 编辑配置文件(或通过 Web 界面配置)
|
||||
# 填写邮件服务器信息和定时任务设置
|
||||
```
|
||||
|
||||
需要重点配置的内容包括:
|
||||
**配置文件说明:**
|
||||
|
||||
- `agent.baseUrl`
|
||||
- `scheduler.enabled`
|
||||
- `scheduler.cronTime`
|
||||
- `email.smtpHost`
|
||||
- `email.smtpUser`
|
||||
- `email.smtpPass`
|
||||
- `email.recipients`
|
||||
- `config.example.json` - 配置模板(不含敏感信息,可提交到 Git)
|
||||
- `config.json` - 实际配置(包含密码等敏感信息,已在 .gitignore 中忽略)
|
||||
|
||||
## 启动方式
|
||||
## 使用方法
|
||||
|
||||
### 开发模式
|
||||
|
||||
前后端同时启动:
|
||||
### 1. 启动服务器
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
启动后地址如下:
|
||||
|
||||
- 前端开发服务:`http://localhost:5173`
|
||||
- 后端 API 服务:`http://localhost:5000`
|
||||
|
||||
说明:
|
||||
|
||||
- Vite 前端会通过代理把 `/api` 请求转发到 `5000`
|
||||
- 开发时建议直接访问 `http://localhost:5173`
|
||||
|
||||
### 生产模式
|
||||
|
||||
先构建前端,再启动 Express:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
启动后访问:
|
||||
### 2. 访问界面
|
||||
|
||||
- `http://localhost:5000`
|
||||
打开浏览器访问: **http://localhost:5000** (或您配置的端口)
|
||||
|
||||
说明:
|
||||
### 3. 功能介绍
|
||||
|
||||
- `npm run build` 会把前端构建到 `dist/`
|
||||
- `npm start` 会由 Express 托管 `dist/` 静态资源和 `/api`
|
||||
**公告列表标签**
|
||||
|
||||
## 常用脚本
|
||||
- 快速查看所有公告
|
||||
- 支持分页浏览
|
||||
- 一键获取最新公告列表
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
npm run build
|
||||
npm run preview
|
||||
npm start
|
||||
**详情采集标签**
|
||||
|
||||
- 批量采集公告详情
|
||||
- 支持按时间范围采集
|
||||
- 自动提取预算金额
|
||||
- 可自定义采集数量
|
||||
|
||||
**生成报告标签**
|
||||
|
||||
- 支持按时间范围生成报告
|
||||
- 设置金额阈值筛选项目
|
||||
- 实时统计项目信息
|
||||
- 一键导出 Word/Markdown 报告
|
||||
|
||||
**定时任务标签** ⭐ 新增
|
||||
|
||||
- Web 界面配置定时任务
|
||||
- 支持 Cron 表达式自定义执行时间
|
||||
- 可选时间范围(今日/本周/本月)
|
||||
- 设置金额阈值自动筛选
|
||||
- 实时查看任务运行状态
|
||||
- 立即测试运行功能
|
||||
|
||||
**邮件配置标签** ⭐ 新增
|
||||
|
||||
- Web 界面配置 SMTP 邮件服务
|
||||
- 支持主流邮箱(QQ、163、Gmail 等)
|
||||
- 测试连接功能验证配置
|
||||
- 支持多个收件人
|
||||
- 自动发送精美 HTML 报告
|
||||
|
||||
## 报告示例
|
||||
|
||||
```markdown
|
||||
# 南京公共工程建设项目报告
|
||||
|
||||
**生成时间**: 2025/12/12 11:00:03
|
||||
|
||||
## 统计摘要
|
||||
|
||||
- 总项目数: 10
|
||||
- 超过 50 万元的项目: 3
|
||||
- 总金额: 5395.50 万元
|
||||
|
||||
## 项目列表
|
||||
|
||||
### 1. 项目名称
|
||||
|
||||
- **发布日期**: 2025-12-12
|
||||
- **发布时间**: 2025-12-12 10:35:00
|
||||
- **预算金额**: 5000 万元
|
||||
- **链接**: https://...
|
||||
```
|
||||
|
||||
脚本说明:
|
||||
## API 接口文档
|
||||
|
||||
- `npm run dev`:开发模式,前后端同时启动
|
||||
- `npm run build`:构建前端生产包
|
||||
- `npm run preview`:本地预览前端构建结果
|
||||
- `npm start`:启动后端服务,并托管前端构建产物
|
||||
服务器启动后提供以下 RESTful API 接口:
|
||||
|
||||
## 页面说明
|
||||
### 1. 获取公告列表
|
||||
|
||||
### 任务配置
|
||||
```
|
||||
GET /api/list?url=<列表页URL>&page=<页码>
|
||||
```
|
||||
|
||||
- 管理抓取任务
|
||||
- 支持单独运行和批量运行
|
||||
- 支持查看运行状态
|
||||
参数:
|
||||
|
||||
### 抓取结果
|
||||
- `url` (可选): 列表页 URL,默认为官网首页
|
||||
- `page` (可选): 页码,默认为 1
|
||||
|
||||
- 查看每次抓取生成的记录
|
||||
- 支持按条件筛选
|
||||
- 支持删除结果
|
||||
### 2. 按时间范围获取列表
|
||||
|
||||
### 项目管理
|
||||
```
|
||||
POST /api/list-daterange
|
||||
Content-Type: application/json
|
||||
|
||||
- 对抓取出的项目做去重查询
|
||||
- 支持金额和日期范围过滤
|
||||
{
|
||||
"startDate": "2025-11-01",
|
||||
"endDate": "2025-12-31",
|
||||
"maxPages": 23
|
||||
}
|
||||
```
|
||||
|
||||
### 系统设置
|
||||
### 3. 批量获取详情
|
||||
|
||||
- 配置 Agent 服务地址
|
||||
- 配置定时任务
|
||||
- 配置邮件发送参数
|
||||
```
|
||||
POST /api/details
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"items": [{ "title": "...", "href": "...", "date": "..." }],
|
||||
"limit": 10
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 生成报告
|
||||
|
||||
```
|
||||
POST /api/report
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"url": "https://gjzx.nanjing.gov.cn/gggs/",
|
||||
"limit": 15,
|
||||
"threshold": 50
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 按时间范围生成报告
|
||||
|
||||
```
|
||||
POST /api/report-daterange
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"startDate": "2025-11-01",
|
||||
"endDate": "2025-12-31",
|
||||
"threshold": 50,
|
||||
"maxPages": 23
|
||||
}
|
||||
```
|
||||
|
||||
### 6. 获取配置
|
||||
|
||||
```
|
||||
GET /api/config
|
||||
```
|
||||
|
||||
返回当前配置信息(密码会被隐藏)
|
||||
|
||||
### 7. 更新配置
|
||||
|
||||
```
|
||||
POST /api/config
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"scheduler": {
|
||||
"enabled": true,
|
||||
"cronTime": "0 9 * * *",
|
||||
"threshold": 100000,
|
||||
"timeRange": "today"
|
||||
},
|
||||
"email": {
|
||||
"smtpHost": "smtp.qq.com",
|
||||
"smtpPort": 587,
|
||||
"smtpUser": "your-email@qq.com",
|
||||
"smtpPass": "your-password",
|
||||
"recipients": "recipient@example.com"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 8. 获取定时任务状态
|
||||
|
||||
```
|
||||
GET /api/scheduler/status
|
||||
```
|
||||
|
||||
### 9. 手动触发定时任务
|
||||
|
||||
```
|
||||
POST /api/run-scheduled-task
|
||||
```
|
||||
|
||||
## 定时任务配置
|
||||
|
||||
### 通过 Web 界面配置(推荐)
|
||||
|
||||
1. 访问 `http://localhost:5000` (或您配置的端口)
|
||||
2. 切换到 **"定时任务"** 标签
|
||||
3. 配置以下选项:
|
||||
- **启用定时任务**:勾选启用
|
||||
- **执行时间**:选择预设时间或自定义 Cron 表达式
|
||||
- **时间范围**:今日/本周/本月
|
||||
- **金额阈值**:设置最低金额(单位:万元)
|
||||
4. 切换到 **"邮件配置"** 标签
|
||||
5. 配置 SMTP 邮件服务器信息
|
||||
6. 点击"测试连接"验证配置
|
||||
7. 点击"保存配置",定时任务自动生效
|
||||
|
||||
### 通过配置文件
|
||||
|
||||
编辑 `config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"scheduler": {
|
||||
"enabled": true, // 是否启用
|
||||
"cronTime": "0 9 * * *", // Cron表达式:每天9点
|
||||
"threshold": 100000, // 金额阈值(万元,10亿)
|
||||
"description": "每天9点采集大于10亿的项目",
|
||||
"timeRange": "today" // 采集范围: today/thisWeek/thisMonth
|
||||
},
|
||||
"email": {
|
||||
"smtpHost": "smtp.qq.com", // SMTP服务器
|
||||
"smtpPort": 587, // 端口
|
||||
"smtpUser": "your@qq.com", // 发件邮箱
|
||||
"smtpPass": "授权码", // QQ邮箱授权码
|
||||
"recipients": "to@qq.com" // 收件人(多个用逗号分隔)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Cron 表达式说明
|
||||
|
||||
格式:`分 时 日 月 周`
|
||||
|
||||
常用示例:
|
||||
- `0 9 * * *` - 每天 9:00
|
||||
- `0 9,18 * * *` - 每天 9:00 和 18:00
|
||||
- `0 9 * * 1` - 每周一 9:00
|
||||
- `0 9 1 * *` - 每月1号 9:00
|
||||
- `0 */6 * * *` - 每 6 小时
|
||||
|
||||
### 邮件服务配置参考
|
||||
|
||||
**QQ 邮箱:**
|
||||
- SMTP: `smtp.qq.com`
|
||||
- 端口: `587`
|
||||
- 密码: 使用授权码(在 QQ 邮箱设置中生成)
|
||||
|
||||
**163 邮箱:**
|
||||
- SMTP: `smtp.163.com`
|
||||
- 端口: `465`
|
||||
- 密码: 使用授权码
|
||||
|
||||
**Gmail:**
|
||||
- SMTP: `smtp.gmail.com`
|
||||
- 端口: `587`
|
||||
- 需要开启"不够安全的应用访问"
|
||||
|
||||
## 服务器部署
|
||||
|
||||
### 方法 1:使用 PM2(推荐)
|
||||
|
||||
```bash
|
||||
# 安装 PM2
|
||||
npm install -g pm2
|
||||
|
||||
# 启动服务
|
||||
pm2 start src/server.js --name gjzx-scraper
|
||||
|
||||
# 查看状态
|
||||
pm2 status
|
||||
|
||||
# 查看日志
|
||||
pm2 logs gjzx-scraper
|
||||
|
||||
# 设置开机自启
|
||||
pm2 startup
|
||||
pm2 save
|
||||
```
|
||||
|
||||
### 方法 2:使用 systemd
|
||||
|
||||
创建服务文件 `/etc/systemd/system/gjzx-scraper.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=GJZX Scraper Service
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=your-user
|
||||
WorkingDirectory=/path/to/tool-node
|
||||
ExecStart=/usr/bin/node src/server.js
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
启动服务:
|
||||
|
||||
```bash
|
||||
sudo systemctl enable gjzx-scraper
|
||||
sudo systemctl start gjzx-scraper
|
||||
sudo systemctl status gjzx-scraper
|
||||
```
|
||||
|
||||
### 注意事项
|
||||
|
||||
- ✅ **无需数据库**:项目采用无数据库架构,轻量部署
|
||||
- ✅ **配置持久化**:所有配置保存在 `config.json` 文件中
|
||||
- ✅ **进程保活**:使用 PM2 或 systemd 确保进程持续运行
|
||||
- ⚠️ **防火墙**:确保配置的端口可访问(默认 5000)
|
||||
- ⚠️ **配置安全**:不要将 `config.json` 提交到公开仓库
|
||||
|
||||
## 技术栈
|
||||
|
||||
- 前端:Vue 3、Vite、Vue Router、Axios、Element Plus
|
||||
- 后端:Express
|
||||
- 数据库:better-sqlite3
|
||||
- 调度:node-cron
|
||||
- 邮件:nodemailer
|
||||
- **后端**: Node.js + Express
|
||||
- **爬虫**: Axios + Cheerio
|
||||
- **定时任务**: node-cron
|
||||
- **邮件服务**: nodemailer
|
||||
- **前端**: 原生 HTML/CSS/JavaScript
|
||||
- **编码处理**: iconv-lite (支持 GBK/UTF-8)
|
||||
- **文档导出**: docx.js
|
||||
- **架构**: 无数据库设计
|
||||
|
||||
## 项目结构
|
||||
|
||||
```text
|
||||
```
|
||||
.
|
||||
├─ client/ # Vue 3 前端源码
|
||||
│ ├─ index.html
|
||||
│ └─ src/
|
||||
│ ├─ api/
|
||||
│ ├─ components/
|
||||
│ ├─ pages/
|
||||
│ ├─ router/
|
||||
│ ├─ App.vue
|
||||
│ ├─ main.js
|
||||
│ └─ styles.css
|
||||
├─ dist/ # Vite 构建产物
|
||||
├─ src/ # Express 服务端
|
||||
│ ├─ server.js
|
||||
│ ├─ scheduler.js
|
||||
│ ├─ resultStore.js
|
||||
│ ├─ agentService.js
|
||||
│ └─ emailService.js
|
||||
├─ data/ # SQLite 数据文件目录
|
||||
├─ config.json # 运行配置
|
||||
├─ config.example.json # 配置示例
|
||||
├─ package.json
|
||||
├─ vite.config.js
|
||||
└─ README.md
|
||||
├── src/
|
||||
│ ├── server.js # Web服务器及API
|
||||
│ ├── scheduler.js # 定时任务调度器
|
||||
│ └── emailService.js # 邮件发送服务
|
||||
├── public/
|
||||
│ ├── index.html # Web界面
|
||||
│ └── app.js # 前端逻辑
|
||||
├── config.json # 配置文件(不提交到Git)
|
||||
├── config.example.json # 配置示例
|
||||
├── package.json
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## 数据说明
|
||||
## 架构特点
|
||||
|
||||
- 任务数据和抓取结果保存在 SQLite
|
||||
- 默认数据库路径位于 `data/results.sqlite`
|
||||
- 配置数据保存在根目录 `config.json`
|
||||
### 无数据库设计
|
||||
|
||||
## 部署建议
|
||||
本项目采用**无数据库架构**,具有以下特点:
|
||||
|
||||
- 使用 PM2 或 systemd 保持进程常驻
|
||||
- 通过反向代理暴露 `5000` 端口
|
||||
- 不要把 `config.json`、`data/`、`.env` 提交到公开仓库
|
||||
- ✅ **轻量部署**:无需安装和配置数据库
|
||||
- ✅ **实时数据**:每次从源站实时抓取最新数据
|
||||
- ✅ **配置简单**:只需配置 config.json 文件
|
||||
- ✅ **邮件归档**:报告通过邮件发送,邮箱即为历史记录
|
||||
- ✅ **低资源消耗**:内存占用小,适合小型服务器
|
||||
|
||||
## 备注
|
||||
### 数据流程
|
||||
|
||||
- 旧版 `public/` 页面已不再作为当前主前端入口
|
||||
- 当前主前端以 `client/` 目录为准
|
||||
```
|
||||
定时触发 → 抓取网站数据 → 解析提取 → 筛选过滤 → 生成报告 → 发送邮件
|
||||
↑ ↓
|
||||
└──────────────────── 配置文件 ──────────────────────────┘
|
||||
```
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. **采集速度**:已限制为每条延迟 500ms-1s,避免请求过快
|
||||
2. **域名支持**:仅支持 gjzx.nanjing.gov.cn 域名的详情页解析
|
||||
3. **金额提取**:基于正则匹配,支持多种格式(预算金额、最高限价等)
|
||||
4. **端口配置**:Web 服务器默认端口 5000,支持通过环境变量 PORT 修改
|
||||
5. **智能停止**:按时间范围采集会在检测到所有公告早于起始日期时自动停止
|
||||
6. **编码处理**:自动识别,支持 GBK 和 UTF-8 网页
|
||||
7. **配置安全**:config.json 包含敏感信息,已加入 .gitignore,不要提交到公开仓库
|
||||
8. **进程保活**:部署时使用 PM2 或 systemd 确保定时任务持续运行
|
||||
|
||||
## 核心功能说明
|
||||
|
||||
### 时间范围采集逻辑
|
||||
|
||||
按时间范围采集时,程序会:
|
||||
|
||||
1. 从第一页开始顺序采集
|
||||
2. 检查每页公告的日期是否在指定范围内
|
||||
3. 如果某页所有公告都早于起始日期,自动停止采集
|
||||
4. 支持设置最大页数限制,避免过度采集
|
||||
|
||||
### 金额提取规则
|
||||
|
||||
支持识别以下格式:
|
||||
|
||||
- 预算金额: XX 万元
|
||||
- 最高限价: XX 万元
|
||||
- 预算: XX 万元
|
||||
- 金额: XX 万元
|
||||
- 直接数字: XX 万元
|
||||
|
||||
### 编码处理
|
||||
|
||||
自动识别网页编码:
|
||||
|
||||
- 优先读取 Content-Type 中的 charset
|
||||
- 自动处理 GBK、GB2312 编码
|
||||
- 默认使用 UTF-8
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q: 为什么采集速度比较慢?
|
||||
|
||||
A: 为了避免对服务器造成过大压力,程序限制了请求频率(每条延迟 500ms-1s)。这是一个负责任的爬虫设计。
|
||||
|
||||
### Q: 如何采集指定日期范围的公告?
|
||||
|
||||
A: 在 Web 界面的"详情采集"和"生成报告"标签中勾选"按时间范围采集",然后输入起始和结束日期即可。
|
||||
|
||||
### Q: 导出的报告在哪里?
|
||||
|
||||
A: 点击"导出 Word"或"导出 Markdown"按钮后会自动下载到浏览器的默认下载目录。
|
||||
|
||||
### Q: 可以采集其他网站吗?
|
||||
|
||||
A: 需要修改 server.js 中的 BASE_URL 和相应的解析函数,因为不同网站的 HTML 结构不同。
|
||||
|
||||
### Q: 需要安装数据库吗?
|
||||
|
||||
A: **不需要!** 本项目采用无数据库架构,只需安装 Node.js 依赖即可运行。所有配置保存在 config.json 文件中,报告通过邮件发送。
|
||||
|
||||
### Q: 定时任务配置后不生效怎么办?
|
||||
|
||||
A: 请检查以下几点:
|
||||
1. `config.json` 中 `scheduler.enabled` 是否为 `true`
|
||||
2. 邮件配置是否正确(SMTP 服务器、用户名、密码)
|
||||
3. Node.js 进程是否持续运行(使用 PM2 查看状态)
|
||||
4. 查看服务器日志是否有错误信息
|
||||
|
||||
### Q: 如何修改定时任务的执行时间?
|
||||
|
||||
A: 有两种方式:
|
||||
1. **Web 界面**(推荐):访问网页,切换到"定时任务"标签,选择预设时间或自定义 Cron 表达式,点击保存
|
||||
2. **配置文件**:编辑 `config.json` 中的 `scheduler.cronTime` 字段
|
||||
|
||||
### Q: 邮件发送失败怎么办?
|
||||
|
||||
A: 常见原因:
|
||||
1. **QQ 邮箱**:需要使用授权码,不是登录密码(在 QQ 邮箱设置 → 账户 → POP3/SMTP 服务中生成)
|
||||
2. **端口问题**:QQ 邮箱使用 587,某些邮箱可能需要 465(SSL)
|
||||
3. **防火墙**:确保服务器可以访问 SMTP 端口
|
||||
4. **测试连接**:在"邮件配置"标签中使用"测试连接"功能验证配置
|
||||
|
||||
### Q: 如何查看定时任务运行日志?
|
||||
|
||||
A: 如果使用 PM2 部署:
|
||||
```bash
|
||||
pm2 logs gjzx-scraper
|
||||
```
|
||||
|
||||
如果使用 systemd:
|
||||
```bash
|
||||
sudo journalctl -u gjzx-scraper -f
|
||||
```
|
||||
|
||||
### Q: 服务器重启后定时任务还会运行吗?
|
||||
|
||||
A: 需要设置开机自启:
|
||||
- **PM2 方式**:执行 `pm2 startup` 和 `pm2 save`
|
||||
- **systemd 方式**:执行 `sudo systemctl enable gjzx-scraper`
|
||||
|
||||
### Q: 可以设置多个定时任务吗?
|
||||
|
||||
A: 当前版本只支持一个定时任务。如需多个任务,可以:
|
||||
1. 使用 Cron 表达式设置多个时间点(如 `0 9,18 * * *` 表示每天 9:00 和 18:00)
|
||||
2. 部署多个实例,使用不同的配置文件和端口
|
||||
|
||||
## 更新日志
|
||||
|
||||
### v1.1.0 (2025-12-15)
|
||||
|
||||
- ✨ 新增定时任务功能(node-cron)
|
||||
- ✨ 新增邮件推送服务(nodemailer)
|
||||
- ✨ 新增 Web 配置界面(定时任务 + 邮件配置)
|
||||
- ✨ 支持多时间范围(今日/本周/本月)
|
||||
- ✨ 配置 API(读取/更新配置)
|
||||
- ✨ 定时任务状态查询
|
||||
- ✨ 手动触发任务功能
|
||||
- 📝 完善文档,新增部署指南
|
||||
|
||||
### v1.0.0 (2025-12-12)
|
||||
|
||||
- Web 可视化界面
|
||||
- 支持按时间范围采集
|
||||
- 支持分页浏览
|
||||
- 支持导出 Word/Markdown 报告
|
||||
- RESTful API 接口
|
||||
- 自动编码识别
|
||||
- 智能金额提取
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>公告抓取与分析工具</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="/src/main.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,10 +0,0 @@
|
||||
<script setup>
|
||||
import zhCn from 'element-plus/es/locale/lang/zh-cn';
|
||||
import AppShell from './components/AppShell.vue';
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-config-provider :locale="zhCn">
|
||||
<AppShell />
|
||||
</el-config-provider>
|
||||
</template>
|
||||
@@ -1,24 +0,0 @@
|
||||
import axios from 'axios';
|
||||
|
||||
const http = axios.create({
|
||||
baseURL: '/api',
|
||||
timeout: 30000,
|
||||
});
|
||||
|
||||
http.interceptors.response.use(
|
||||
(response) => {
|
||||
const payload = response.data;
|
||||
|
||||
if (payload && payload.success === false) {
|
||||
return Promise.reject(new Error(payload.error || '请求失败'));
|
||||
}
|
||||
|
||||
return payload;
|
||||
},
|
||||
(error) => {
|
||||
const message = error.response?.data?.error || error.message || '请求失败';
|
||||
return Promise.reject(new Error(message));
|
||||
},
|
||||
);
|
||||
|
||||
export default http;
|
||||
@@ -1,25 +0,0 @@
|
||||
import http from './http';
|
||||
|
||||
export function fetchResults(params = {}) {
|
||||
return http.get('/results', { params }).then((payload) => payload);
|
||||
}
|
||||
|
||||
export function fetchResultFilters(params = {}) {
|
||||
return http.get('/results/filters', { params }).then((payload) => payload.data || {});
|
||||
}
|
||||
|
||||
export function deleteResult(id) {
|
||||
return http.delete(`/results/${id}`);
|
||||
}
|
||||
|
||||
export function clearResults() {
|
||||
return http.delete('/results');
|
||||
}
|
||||
|
||||
export function fetchProjects(params = {}) {
|
||||
return http.get('/projects', { params }).then((payload) => payload);
|
||||
}
|
||||
|
||||
export function fetchProjectFilters() {
|
||||
return http.get('/projects/filters').then((payload) => payload.data || {});
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
import http from './http';
|
||||
|
||||
export function fetchConfig() {
|
||||
return http.get('/config').then((payload) => payload.data || {});
|
||||
}
|
||||
|
||||
export function saveConfig(payload) {
|
||||
return http.post('/config', payload);
|
||||
}
|
||||
|
||||
export function fetchSchedulerStatus() {
|
||||
return http.get('/scheduler/status').then((payload) => payload.data || {});
|
||||
}
|
||||
|
||||
export function triggerScheduledTask() {
|
||||
return http.post('/run-scheduled-task');
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
import http from './http';
|
||||
|
||||
export function fetchTasks() {
|
||||
return http.get('/tasks').then((payload) => payload.data || []);
|
||||
}
|
||||
|
||||
export function createTask(payload) {
|
||||
return http.post('/tasks', payload).then((response) => response.data);
|
||||
}
|
||||
|
||||
export function updateTask(id, payload) {
|
||||
return http.put(`/tasks/${id}`, payload).then((response) => response.data);
|
||||
}
|
||||
|
||||
export function deleteTask(id) {
|
||||
return http.delete(`/tasks/${id}`);
|
||||
}
|
||||
|
||||
export function runTask(id) {
|
||||
return http.post(`/tasks/${id}/run`);
|
||||
}
|
||||
|
||||
export function runAllTasks(ids = []) {
|
||||
const body = ids.length ? { ids } : {};
|
||||
return http.post('/tasks/run', body);
|
||||
}
|
||||
|
||||
export function toggleTask(id, enabled) {
|
||||
return http.put(`/tasks/${id}`, { enabled }).then((response) => response.data);
|
||||
}
|
||||
|
||||
export function fetchTaskStatus() {
|
||||
return http.get('/tasks/status').then((payload) => payload.data || { isRunning: false });
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
<script setup>
|
||||
import { computed } from 'vue';
|
||||
import { useRoute } from 'vue-router';
|
||||
|
||||
const route = useRoute();
|
||||
|
||||
const subtitle = computed(() => route.meta?.subtitle || '把抓取任务、分析结果和调度配置收在同一套前端工作台里。');
|
||||
|
||||
const menuItems = [
|
||||
{ index: '/tasks', label: '任务配置' },
|
||||
{ index: '/results', label: '抓取结果' },
|
||||
{ index: '/projects', label: '项目管理' },
|
||||
{ index: '/settings', label: '系统设置' },
|
||||
];
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="app-shell">
|
||||
<div class="app-frame">
|
||||
<header class="app-topbar">
|
||||
<div class="app-brand">
|
||||
<div class="app-brand__mark">公</div>
|
||||
<div class="app-brand__text">
|
||||
<h1>公告抓取与分析工具</h1>
|
||||
<p>{{ subtitle }}</p>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<el-menu :default-active="route.path" class="app-menu" mode="horizontal" router>
|
||||
<el-menu-item v-for="item in menuItems" :key="item.index" :index="item.index">
|
||||
{{ item.label }}
|
||||
</el-menu-item>
|
||||
</el-menu>
|
||||
|
||||
<main class="app-main">
|
||||
<RouterView />
|
||||
</main>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
@@ -1,55 +0,0 @@
|
||||
<script setup>
|
||||
import { computed } from 'vue';
|
||||
import { formatDateTime, formatYuan, pickResultLink, pickResultName } from '@/utils/format';
|
||||
|
||||
const props = defineProps({
|
||||
record: {
|
||||
type: Object,
|
||||
required: true,
|
||||
},
|
||||
});
|
||||
|
||||
const emit = defineEmits(['delete']);
|
||||
|
||||
const items = computed(() => props.record.data?.results || []);
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-card shadow="hover" class="result-card-plus">
|
||||
<template #header>
|
||||
<div class="result-card-plus__header">
|
||||
<div>
|
||||
<div class="result-card-plus__title">{{ record.city || '未命名城市' }}</div>
|
||||
<div class="result-card-plus__meta">
|
||||
<el-tag size="small" type="primary">{{ items.length }} 条结果</el-tag>
|
||||
<el-tag v-if="record.error" size="small" type="danger">执行失败</el-tag>
|
||||
</div>
|
||||
</div>
|
||||
<div class="result-card-plus__time">{{ formatDateTime(record.scrapedAt) }}</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<el-alert v-if="record.error" :title="`错误信息:${record.error}`" type="error" :closable="false" show-icon />
|
||||
|
||||
<div v-else-if="items.length" class="result-card-plus__list">
|
||||
<div v-for="(item, index) in items" :key="`${record.id}-${index}`" class="result-card-plus__item">
|
||||
<div class="result-card-plus__type">{{ item.type || '-' }}</div>
|
||||
<div class="result-card-plus__name">{{ pickResultName(item) }}</div>
|
||||
<div class="result-card-plus__amount">{{ formatYuan(item.amount_yuan) }}</div>
|
||||
<div class="result-card-plus__date">{{ item.date || '-' }}</div>
|
||||
<div class="result-card-plus__action">
|
||||
<el-link v-if="pickResultLink(item)" :href="pickResultLink(item)" target="_blank" type="primary">查看详情</el-link>
|
||||
<span v-else>-</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<el-empty v-else description="当前记录里没有可展示的数据" :image-size="88" />
|
||||
|
||||
<template #footer>
|
||||
<div class="result-card-plus__footer">
|
||||
<el-button type="danger" plain @click="emit('delete', record)">删除记录</el-button>
|
||||
</div>
|
||||
</template>
|
||||
</el-card>
|
||||
</template>
|
||||
@@ -1,105 +0,0 @@
|
||||
<script setup>
|
||||
import { computed, reactive, watch } from 'vue';
|
||||
import { TASK_MODE_OPTIONS } from '@/constants/taskModes';
|
||||
|
||||
const props = defineProps({
|
||||
modelValue: {
|
||||
type: Boolean,
|
||||
default: false,
|
||||
},
|
||||
task: {
|
||||
type: Object,
|
||||
default: null,
|
||||
},
|
||||
});
|
||||
|
||||
const emit = defineEmits(['update:modelValue', 'submit']);
|
||||
|
||||
const form = reactive({
|
||||
city: '',
|
||||
plateName: '',
|
||||
prompt: '',
|
||||
mode: TASK_MODE_OPTIONS[0],
|
||||
enabled: true,
|
||||
});
|
||||
|
||||
const dialogTitle = computed(() => (props.task?.id ? '编辑任务' : '新增任务'));
|
||||
const modeOptions = computed(() => {
|
||||
const set = new Set(TASK_MODE_OPTIONS);
|
||||
if (props.task?.mode) set.add(props.task.mode);
|
||||
return [...set];
|
||||
});
|
||||
|
||||
watch(
|
||||
() => [props.modelValue, props.task],
|
||||
() => {
|
||||
form.city = props.task?.city || '';
|
||||
form.plateName = props.task?.plateName || '';
|
||||
form.prompt = props.task?.prompt || '';
|
||||
form.mode = props.task?.mode || TASK_MODE_OPTIONS[0];
|
||||
form.enabled = props.task?.enabled ?? true;
|
||||
},
|
||||
{ immediate: true },
|
||||
);
|
||||
|
||||
function close() {
|
||||
emit('update:modelValue', false);
|
||||
}
|
||||
|
||||
function submit() {
|
||||
emit('submit', {
|
||||
city: form.city.trim(),
|
||||
plateName: form.plateName.trim(),
|
||||
prompt: form.prompt.trim(),
|
||||
mode: form.mode.trim() || TASK_MODE_OPTIONS[0],
|
||||
enabled: form.enabled,
|
||||
});
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<el-dialog :model-value="modelValue" :title="dialogTitle" width="760px" destroy-on-close @close="close">
|
||||
<el-form label-position="top">
|
||||
<el-form-item label="城市名称">
|
||||
<el-input v-model="form.city" placeholder="如:南京市" />
|
||||
</el-form-item>
|
||||
|
||||
<el-form-item label="板块名称">
|
||||
<el-input v-model="form.plateName" placeholder="如:工程" />
|
||||
</el-form-item>
|
||||
|
||||
<el-form-item label="提示词">
|
||||
<el-input
|
||||
v-model="form.prompt"
|
||||
type="textarea"
|
||||
:autosize="{ minRows: 6, maxRows: 12 }"
|
||||
placeholder="请访问目标网站,抓取今天的招标公告和中标公告信息……"
|
||||
/>
|
||||
</el-form-item>
|
||||
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="模型">
|
||||
<el-select v-model="form.mode" placeholder="选择模型" style="width: 100%;">
|
||||
<el-option v-for="option in modeOptions" :key="option" :label="option" :value="option" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="状态">
|
||||
<el-switch v-model="form.enabled" inline-prompt active-text="启用" inactive-text="禁用" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
|
||||
<el-alert title="建议把目标网址、筛选规则和字段要求都写进提示词里,后续维护会更稳定。" type="info" :closable="false" show-icon />
|
||||
|
||||
<template #footer>
|
||||
<div class="dialog-footer">
|
||||
<el-button @click="close">取消</el-button>
|
||||
<el-button type="primary" @click="submit">保存任务</el-button>
|
||||
</div>
|
||||
</template>
|
||||
</el-dialog>
|
||||
</template>
|
||||
@@ -1,10 +0,0 @@
|
||||
export const TASK_MODE_OPTIONS = [
|
||||
'qwen3.5-plus',
|
||||
'qwen3-max-2026-01-23',
|
||||
'qwen3-coder-next',
|
||||
'qwen3-coder-plus',
|
||||
'glm-5',
|
||||
'glm-4.7',
|
||||
'kimi-k2.5',
|
||||
'MiniMax-M2.5',
|
||||
];
|
||||
@@ -1,6 +0,0 @@
|
||||
import { createApp } from 'vue';
|
||||
import App from './App.vue';
|
||||
import router from './router';
|
||||
import './styles.css';
|
||||
|
||||
createApp(App).use(router).mount('#app');
|
||||
@@ -1,233 +0,0 @@
|
||||
<script setup>
|
||||
import { onMounted, reactive, ref } from 'vue';
|
||||
import { ElMessage } from 'element-plus';
|
||||
import { fetchProjectFilters, fetchProjects } from '@/api/results';
|
||||
import { formatYuan } from '@/utils/format';
|
||||
|
||||
const form = reactive({
|
||||
city: '',
|
||||
section: '',
|
||||
projectName: '',
|
||||
minAmount: '',
|
||||
maxAmount: '',
|
||||
startDate: '',
|
||||
endDate: '',
|
||||
});
|
||||
|
||||
const activeFilters = reactive({
|
||||
city: '',
|
||||
section: '',
|
||||
projectName: '',
|
||||
minAmount: '',
|
||||
maxAmount: '',
|
||||
startDate: '',
|
||||
endDate: '',
|
||||
});
|
||||
|
||||
const options = reactive({
|
||||
cities: [],
|
||||
sections: [],
|
||||
});
|
||||
|
||||
const state = reactive({
|
||||
records: [],
|
||||
page: 1,
|
||||
pageSize: 10,
|
||||
total: 0,
|
||||
loading: true,
|
||||
});
|
||||
|
||||
const summaryText = ref('加载中…');
|
||||
|
||||
onMounted(async () => {
|
||||
await Promise.all([loadFilters(), loadProjects()]);
|
||||
});
|
||||
|
||||
async function loadFilters() {
|
||||
try {
|
||||
const data = await fetchProjectFilters();
|
||||
options.cities = data.cities || [];
|
||||
options.sections = data.sections || [];
|
||||
} catch (error) {
|
||||
ElMessage.error(`项目筛选项加载失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function validateForm() {
|
||||
if (form.minAmount && form.maxAmount && Number(form.minAmount) > Number(form.maxAmount)) {
|
||||
throw new Error('最小金额不能大于最大金额');
|
||||
}
|
||||
|
||||
if (form.startDate && form.endDate && form.startDate > form.endDate) {
|
||||
throw new Error('开始日期不能晚于结束日期');
|
||||
}
|
||||
}
|
||||
|
||||
function syncActiveFilters() {
|
||||
Object.assign(activeFilters, form);
|
||||
}
|
||||
|
||||
async function loadProjects() {
|
||||
state.loading = true;
|
||||
|
||||
try {
|
||||
const payload = await fetchProjects({
|
||||
...activeFilters,
|
||||
page: state.page,
|
||||
pageSize: state.pageSize,
|
||||
});
|
||||
|
||||
state.records = payload.data || [];
|
||||
state.total = payload.total || 0;
|
||||
state.page = payload.page || 1;
|
||||
|
||||
if (!state.records.length) {
|
||||
summaryText.value = '当前筛选条件下没有项目';
|
||||
return;
|
||||
}
|
||||
|
||||
const startIndex = (state.page - 1) * state.pageSize + 1;
|
||||
const endIndex = Math.min(state.page * state.pageSize, state.total);
|
||||
summaryText.value = `显示第 ${startIndex} 至 ${endIndex} 条记录,共 ${state.total} 条`;
|
||||
} catch (error) {
|
||||
state.records = [];
|
||||
state.total = 0;
|
||||
summaryText.value = `项目加载失败:${error.message}`;
|
||||
ElMessage.error(`项目加载失败:${error.message}`);
|
||||
} finally {
|
||||
state.loading = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function search() {
|
||||
try {
|
||||
validateForm();
|
||||
syncActiveFilters();
|
||||
state.page = 1;
|
||||
await loadProjects();
|
||||
} catch (error) {
|
||||
ElMessage.error(error.message);
|
||||
}
|
||||
}
|
||||
|
||||
async function reset() {
|
||||
Object.keys(form).forEach((key) => {
|
||||
form[key] = '';
|
||||
activeFilters[key] = '';
|
||||
});
|
||||
state.page = 1;
|
||||
await loadProjects();
|
||||
}
|
||||
|
||||
async function changePage(page) {
|
||||
state.page = page;
|
||||
await loadProjects();
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="page-grid">
|
||||
<header class="page-head">
|
||||
<div>
|
||||
<h2>项目管理</h2>
|
||||
<p>用 Element Plus 表单和表格组织项目去重查询,适合继续扩展导出、排序和列配置。</p>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<el-card shadow="never">
|
||||
<el-form label-position="top">
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="城市">
|
||||
<el-select v-model="form.city" placeholder="全部城市" clearable style="width: 100%;">
|
||||
<el-option v-for="item in options.cities" :key="item" :label="item" :value="item" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="板块">
|
||||
<el-select v-model="form.section" placeholder="全部板块" clearable style="width: 100%;">
|
||||
<el-option v-for="item in options.sections" :key="item" :label="item" :value="item" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="项目名称">
|
||||
<el-input v-model="form.projectName" placeholder="输入项目名关键字" clearable />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="最小金额">
|
||||
<el-input-number v-model="form.minAmount" :min="0" :precision="2" :controls="false" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="最大金额">
|
||||
<el-input-number v-model="form.maxAmount" :min="0" :precision="2" :controls="false" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="开始日期">
|
||||
<el-date-picker v-model="form.startDate" type="date" value-format="YYYY-MM-DD" placeholder="选择日期" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-form-item label="结束日期">
|
||||
<el-date-picker v-model="form.endDate" type="date" value-format="YYYY-MM-DD" placeholder="选择日期" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
|
||||
<div class="toolbar">
|
||||
<div class="summary-text">按项目名称去重后展示,方便快速判断有哪些有效项目。</div>
|
||||
<div class="toolbar__group">
|
||||
<el-button @click="reset">重置</el-button>
|
||||
<el-button type="primary" @click="search">查询项目</el-button>
|
||||
</div>
|
||||
</div>
|
||||
</el-card>
|
||||
|
||||
<el-card shadow="never">
|
||||
<template #header>
|
||||
<div class="toolbar">
|
||||
<span>{{ summaryText }}</span>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<el-table v-loading="state.loading" :data="state.records" style="width: 100%;" empty-text="暂无项目数据">
|
||||
<el-table-column prop="city" label="城市" min-width="120" />
|
||||
<el-table-column label="板块" min-width="140">
|
||||
<template #default="{ row }">
|
||||
{{ row.section || row.type || '-' }}
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column prop="projectName" label="项目名称" min-width="280" show-overflow-tooltip />
|
||||
<el-table-column label="金额" min-width="150">
|
||||
<template #default="{ row }">
|
||||
{{ formatYuan(row.amountYuan) }}
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column prop="date" label="发布日期" min-width="140" />
|
||||
<el-table-column label="详情" min-width="120" fixed="right">
|
||||
<template #default="{ row }">
|
||||
<el-link v-if="row.detailLink" :href="row.detailLink" type="primary" target="_blank">查看详情</el-link>
|
||||
<span v-else>-</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
|
||||
<div class="pagination-wrap">
|
||||
<el-pagination
|
||||
v-if="state.total"
|
||||
background
|
||||
layout="total, prev, pager, next"
|
||||
:current-page="state.page"
|
||||
:page-size="state.pageSize"
|
||||
:total="state.total"
|
||||
@current-change="changePage"
|
||||
/>
|
||||
</div>
|
||||
</el-card>
|
||||
</section>
|
||||
</template>
|
||||
@@ -1,200 +0,0 @@
|
||||
<script setup>
|
||||
import { computed, onMounted, reactive } from 'vue';
|
||||
import { ElMessage, ElMessageBox } from 'element-plus';
|
||||
import ResultRecordCard from '@/components/ResultRecordCard.vue';
|
||||
import { deleteResult, fetchResultFilters, fetchResults } from '@/api/results';
|
||||
|
||||
const filters = reactive({
|
||||
city: '',
|
||||
section: '',
|
||||
type: '',
|
||||
});
|
||||
|
||||
const options = reactive({
|
||||
cities: [],
|
||||
sections: [],
|
||||
types: [],
|
||||
});
|
||||
|
||||
const stats = reactive({
|
||||
total: 0,
|
||||
success: 0,
|
||||
failed: 0,
|
||||
cities: 0,
|
||||
});
|
||||
|
||||
const state = reactive({
|
||||
records: [],
|
||||
page: 1,
|
||||
pageSize: 10,
|
||||
total: 0,
|
||||
loading: true,
|
||||
});
|
||||
|
||||
const hasRecords = computed(() => state.records.length > 0);
|
||||
|
||||
onMounted(async () => {
|
||||
await refreshAll();
|
||||
});
|
||||
|
||||
async function loadFilters() {
|
||||
const data = await fetchResultFilters();
|
||||
options.cities = data.cities || [];
|
||||
options.sections = data.sections || [];
|
||||
options.types = data.types || [];
|
||||
}
|
||||
|
||||
async function loadStats() {
|
||||
const payload = await fetchResults({ page: 1, pageSize: 500 });
|
||||
const items = payload.data || [];
|
||||
stats.total = payload.total || 0;
|
||||
stats.success = items.filter((item) => !item.error).length;
|
||||
stats.failed = items.filter((item) => item.error).length;
|
||||
stats.cities = new Set(items.map((item) => item.city).filter(Boolean)).size;
|
||||
}
|
||||
|
||||
async function loadRecords() {
|
||||
state.loading = true;
|
||||
|
||||
try {
|
||||
const payload = await fetchResults({
|
||||
page: state.page,
|
||||
pageSize: state.pageSize,
|
||||
city: filters.city || undefined,
|
||||
section: filters.section || undefined,
|
||||
type: filters.type || undefined,
|
||||
});
|
||||
|
||||
state.records = payload.data || [];
|
||||
state.total = payload.total || 0;
|
||||
state.page = payload.page || 1;
|
||||
} catch (error) {
|
||||
ElMessage.error(`结果加载失败:${error.message}`);
|
||||
state.records = [];
|
||||
state.total = 0;
|
||||
} finally {
|
||||
state.loading = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function refreshAll() {
|
||||
try {
|
||||
await Promise.all([loadFilters(), loadStats(), loadRecords()]);
|
||||
} catch (error) {
|
||||
ElMessage.error(`页面刷新失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function applyFilters() {
|
||||
state.page = 1;
|
||||
await loadRecords();
|
||||
}
|
||||
|
||||
async function changePage(page) {
|
||||
state.page = page;
|
||||
await loadRecords();
|
||||
}
|
||||
|
||||
async function handleDelete(record) {
|
||||
try {
|
||||
await ElMessageBox.confirm('确定要删除这条抓取记录吗?', '删除确认', {
|
||||
type: 'warning',
|
||||
});
|
||||
await deleteResult(record.id);
|
||||
ElMessage.success('抓取记录已删除');
|
||||
await refreshAll();
|
||||
} catch (error) {
|
||||
if (error !== 'cancel' && error !== 'close') {
|
||||
ElMessage.error(`删除失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function resetFilters() {
|
||||
filters.city = '';
|
||||
filters.section = '';
|
||||
filters.type = '';
|
||||
applyFilters();
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="page-grid">
|
||||
<header class="page-head">
|
||||
<div>
|
||||
<h2>抓取结果</h2>
|
||||
<p>结果页已经切到 Element Plus 组件风格,适合继续加更复杂的筛选、导出和批量操作。</p>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-statistic title="总记录数" :value="stats.total" />
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-statistic title="成功条数" :value="stats.success" />
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-statistic title="失败条数" :value="stats.failed" />
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12" :lg="6">
|
||||
<el-statistic title="来源城市" :value="stats.cities" />
|
||||
</el-col>
|
||||
</el-row>
|
||||
|
||||
<el-card shadow="never">
|
||||
<el-form label-position="top">
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="8">
|
||||
<el-form-item label="城市">
|
||||
<el-select v-model="filters.city" placeholder="全部城市" clearable style="width: 100%;" @change="applyFilters">
|
||||
<el-option v-for="item in options.cities" :key="item" :label="item" :value="item" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="8">
|
||||
<el-form-item label="板块">
|
||||
<el-select v-model="filters.section" placeholder="全部板块" clearable style="width: 100%;" @change="applyFilters">
|
||||
<el-option v-for="item in options.sections" :key="item" :label="item" :value="item" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="8">
|
||||
<el-form-item label="类型">
|
||||
<el-select v-model="filters.type" placeholder="全部类型" clearable style="width: 100%;" @change="applyFilters">
|
||||
<el-option v-for="item in options.types" :key="item" :label="item" :value="item" />
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
|
||||
<div class="toolbar">
|
||||
<div class="summary-text">当前页展示 {{ state.records.length }} 条记录。</div>
|
||||
<div class="toolbar__group">
|
||||
<el-button @click="resetFilters">重置筛选</el-button>
|
||||
<el-button type="primary" @click="refreshAll">刷新数据</el-button>
|
||||
</div>
|
||||
</div>
|
||||
</el-card>
|
||||
|
||||
<div v-loading="state.loading" class="page-grid">
|
||||
<template v-if="hasRecords">
|
||||
<ResultRecordCard v-for="item in state.records" :key="item.id" :record="item" @delete="handleDelete" />
|
||||
</template>
|
||||
<el-empty v-else description="暂无抓取结果" :image-size="96" />
|
||||
</div>
|
||||
|
||||
<div class="pagination-wrap">
|
||||
<el-pagination
|
||||
v-if="state.total"
|
||||
background
|
||||
layout="total, prev, pager, next"
|
||||
:current-page="state.page"
|
||||
:page-size="state.pageSize"
|
||||
:total="state.total"
|
||||
@current-change="changePage"
|
||||
/>
|
||||
</div>
|
||||
</section>
|
||||
</template>
|
||||
@@ -1,228 +0,0 @@
|
||||
<script setup>
|
||||
import { onMounted, reactive, ref } from 'vue';
|
||||
import { ElMessage, ElMessageBox } from 'element-plus';
|
||||
import { fetchConfig, fetchSchedulerStatus, saveConfig, triggerScheduledTask } from '@/api/settings';
|
||||
|
||||
const loading = ref(true);
|
||||
const saveStatus = ref('');
|
||||
const schedulerText = ref('未获取');
|
||||
|
||||
const form = reactive({
|
||||
agent: {
|
||||
baseUrl: '',
|
||||
pollInterval: 3000,
|
||||
timeout: 300000,
|
||||
},
|
||||
scheduler: {
|
||||
enabled: false,
|
||||
cronTime: '0 9 * * *',
|
||||
description: '',
|
||||
},
|
||||
email: {
|
||||
smtpHost: '',
|
||||
smtpPort: 587,
|
||||
smtpUser: '',
|
||||
smtpPass: '',
|
||||
recipients: '',
|
||||
},
|
||||
});
|
||||
|
||||
onMounted(async () => {
|
||||
await refresh();
|
||||
});
|
||||
|
||||
async function refresh() {
|
||||
loading.value = true;
|
||||
|
||||
try {
|
||||
const [config, schedulerStatus] = await Promise.all([fetchConfig(), fetchSchedulerStatus()]);
|
||||
|
||||
form.agent.baseUrl = config.agent?.baseUrl || '';
|
||||
form.agent.pollInterval = config.agent?.pollInterval || 3000;
|
||||
form.agent.timeout = config.agent?.timeout || 300000;
|
||||
|
||||
form.scheduler.enabled = config.scheduler?.enabled ?? false;
|
||||
form.scheduler.cronTime = config.scheduler?.cronTime || '0 9 * * *';
|
||||
form.scheduler.description = config.scheduler?.description || '';
|
||||
|
||||
form.email.smtpHost = config.email?.smtpHost || '';
|
||||
form.email.smtpPort = config.email?.smtpPort || 587;
|
||||
form.email.smtpUser = config.email?.smtpUser || '';
|
||||
form.email.smtpPass = config.email?.smtpPass || '';
|
||||
form.email.recipients = config.email?.recipients || '';
|
||||
|
||||
schedulerText.value = schedulerStatus.isRunning
|
||||
? `运行中(启用任务 ${schedulerStatus.enabledTasks || 0} 个)`
|
||||
: `未运行(启用任务 ${schedulerStatus.enabledTasks || 0} 个)`;
|
||||
} catch (error) {
|
||||
ElMessage.error(`设置加载失败:${error.message}`);
|
||||
} finally {
|
||||
loading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSave() {
|
||||
try {
|
||||
await saveConfig({
|
||||
agent: {
|
||||
baseUrl: form.agent.baseUrl.trim(),
|
||||
pollInterval: Number(form.agent.pollInterval) || 3000,
|
||||
timeout: Number(form.agent.timeout) || 300000,
|
||||
},
|
||||
scheduler: {
|
||||
enabled: Boolean(form.scheduler.enabled),
|
||||
cronTime: form.scheduler.cronTime.trim() || '0 9 * * *',
|
||||
description: form.scheduler.description.trim(),
|
||||
},
|
||||
email: {
|
||||
smtpHost: form.email.smtpHost.trim(),
|
||||
smtpPort: Number(form.email.smtpPort) || 587,
|
||||
smtpUser: form.email.smtpUser.trim(),
|
||||
smtpPass: form.email.smtpPass.trim(),
|
||||
recipients: form.email.recipients.trim(),
|
||||
},
|
||||
});
|
||||
|
||||
saveStatus.value = '设置已保存';
|
||||
ElMessage.success('系统设置已保存');
|
||||
await refresh();
|
||||
setTimeout(() => {
|
||||
saveStatus.value = '';
|
||||
}, 3000);
|
||||
} catch (error) {
|
||||
saveStatus.value = '';
|
||||
ElMessage.error(`保存失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleTriggerScheduler() {
|
||||
try {
|
||||
await ElMessageBox.confirm('确定要立即执行一次定时任务吗?', '执行确认', {
|
||||
type: 'warning',
|
||||
});
|
||||
const payload = await triggerScheduledTask();
|
||||
ElMessage.success(payload.message || '定时任务已在后台触发');
|
||||
await refresh();
|
||||
} catch (error) {
|
||||
if (error !== 'cancel' && error !== 'close') {
|
||||
ElMessage.error(`触发失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="page-grid">
|
||||
<header class="page-head">
|
||||
<div>
|
||||
<h2>系统设置</h2>
|
||||
<p>这一页已经改用 Element Plus 表单和卡片来管理 Agent、定时任务和邮件配置。</p>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<div v-loading="loading" class="page-grid">
|
||||
<el-card shadow="never">
|
||||
<template #header>
|
||||
<span>Agent 服务配置</span>
|
||||
</template>
|
||||
|
||||
<el-form label-position="top">
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="24">
|
||||
<el-form-item label="Agent 服务地址">
|
||||
<el-input v-model="form.agent.baseUrl" placeholder="http://127.0.0.1:18625" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="轮询间隔(毫秒)">
|
||||
<el-input-number v-model="form.agent.pollInterval" :min="1000" :step="100" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="超时时间(毫秒)">
|
||||
<el-input-number v-model="form.agent.timeout" :min="1000" :step="1000" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
</el-card>
|
||||
|
||||
<el-card shadow="never">
|
||||
<template #header>
|
||||
<div class="toolbar">
|
||||
<span>定时任务</span>
|
||||
<el-tag type="info">{{ schedulerText }}</el-tag>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<el-form label-position="top">
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="8">
|
||||
<el-form-item label="是否启用">
|
||||
<el-switch v-model="form.scheduler.enabled" inline-prompt active-text="启用" inactive-text="禁用" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="8">
|
||||
<el-form-item label="Cron 表达式">
|
||||
<el-input v-model="form.scheduler.cronTime" placeholder="0 9 * * *" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="8">
|
||||
<el-form-item label="描述">
|
||||
<el-input v-model="form.scheduler.description" placeholder="每天 9 点执行" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
|
||||
<el-button @click="handleTriggerScheduler">立即执行一次</el-button>
|
||||
</el-card>
|
||||
|
||||
<el-card shadow="never">
|
||||
<template #header>
|
||||
<span>邮件配置</span>
|
||||
</template>
|
||||
|
||||
<el-form label-position="top">
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="SMTP 服务地址">
|
||||
<el-input v-model="form.email.smtpHost" placeholder="smtp.qq.com" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="端口">
|
||||
<el-input-number v-model="form.email.smtpPort" :min="1" style="width: 100%;" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="用户名">
|
||||
<el-input v-model="form.email.smtpUser" placeholder="your-email@example.com" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="密码或授权码">
|
||||
<el-input v-model="form.email.smtpPass" type="password" show-password placeholder="留空会沿用已保存的掩码值" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="24">
|
||||
<el-form-item label="收件人">
|
||||
<el-input v-model="form.email.recipients" placeholder="a@example.com, b@example.com" />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
</el-card>
|
||||
|
||||
<el-card shadow="never">
|
||||
<div class="toolbar">
|
||||
<div class="summary-text">保存后会自动刷新定时任务配置。</div>
|
||||
<div class="toolbar__group">
|
||||
<span v-if="saveStatus" class="summary-text" style="color: var(--el-color-success);">{{ saveStatus }}</span>
|
||||
<el-button type="primary" @click="handleSave">保存全部设置</el-button>
|
||||
</div>
|
||||
</div>
|
||||
</el-card>
|
||||
</div>
|
||||
</section>
|
||||
</template>
|
||||
@@ -1,347 +0,0 @@
|
||||
<script setup>
|
||||
import { computed, onBeforeUnmount, onMounted, reactive, ref, watch } from 'vue';
|
||||
import { ElMessage, ElMessageBox } from 'element-plus';
|
||||
import TaskDialog from '@/components/TaskDialog.vue';
|
||||
import {
|
||||
createTask,
|
||||
deleteTask,
|
||||
fetchTasks,
|
||||
fetchTaskStatus,
|
||||
runAllTasks,
|
||||
runTask,
|
||||
toggleTask,
|
||||
updateTask,
|
||||
} from '@/api/tasks';
|
||||
import { formatElapsed } from '@/utils/format';
|
||||
|
||||
const tasks = ref([]);
|
||||
const isLoading = ref(true);
|
||||
const dialogOpen = ref(false);
|
||||
const editingTask = ref(null);
|
||||
|
||||
const filters = reactive({
|
||||
city: '',
|
||||
plate: '',
|
||||
});
|
||||
|
||||
const pagination = reactive({
|
||||
page: 1,
|
||||
pageSize: 10,
|
||||
});
|
||||
|
||||
const runner = reactive({
|
||||
timer: null,
|
||||
polling: false,
|
||||
});
|
||||
|
||||
const status = reactive({
|
||||
visible: false,
|
||||
type: 'info',
|
||||
message: '',
|
||||
});
|
||||
|
||||
const filteredTasks = computed(() => {
|
||||
const cityKeyword = filters.city.trim().toLowerCase();
|
||||
const plateKeyword = filters.plate.trim().toLowerCase();
|
||||
|
||||
return tasks.value.filter((task) => {
|
||||
const city = (task.city || '').toLowerCase();
|
||||
const plate = (task.plateName || '').toLowerCase();
|
||||
return (!cityKeyword || city.includes(cityKeyword)) && (!plateKeyword || plate.includes(plateKeyword));
|
||||
});
|
||||
});
|
||||
|
||||
const enabledCount = computed(() => tasks.value.filter((item) => item.enabled).length);
|
||||
const pagedTasks = computed(() => {
|
||||
const start = (pagination.page - 1) * pagination.pageSize;
|
||||
return filteredTasks.value.slice(start, start + pagination.pageSize);
|
||||
});
|
||||
|
||||
const summaryText = computed(
|
||||
() => `共 ${tasks.value.length} 个任务,已启用 ${enabledCount.value} 个,筛选后 ${filteredTasks.value.length} 个`,
|
||||
);
|
||||
|
||||
watch(
|
||||
() => [filters.city, filters.plate],
|
||||
() => {
|
||||
pagination.page = 1;
|
||||
},
|
||||
);
|
||||
|
||||
onMounted(async () => {
|
||||
await loadTasks();
|
||||
await restoreRunningStatus();
|
||||
});
|
||||
|
||||
onBeforeUnmount(() => {
|
||||
stopPolling();
|
||||
});
|
||||
|
||||
async function loadTasks() {
|
||||
isLoading.value = true;
|
||||
|
||||
try {
|
||||
tasks.value = await fetchTasks();
|
||||
} catch (error) {
|
||||
ElMessage.error(`任务加载失败:${error.message}`);
|
||||
} finally {
|
||||
isLoading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
function openCreateDialog() {
|
||||
editingTask.value = null;
|
||||
dialogOpen.value = true;
|
||||
}
|
||||
|
||||
function openEditDialog(task) {
|
||||
editingTask.value = task;
|
||||
dialogOpen.value = true;
|
||||
}
|
||||
|
||||
function updateStatus(type, message) {
|
||||
status.visible = true;
|
||||
status.type = type;
|
||||
status.message = message;
|
||||
}
|
||||
|
||||
function stopPolling() {
|
||||
if (runner.timer) {
|
||||
clearInterval(runner.timer);
|
||||
runner.timer = null;
|
||||
}
|
||||
runner.polling = false;
|
||||
}
|
||||
|
||||
function applyRunStatus(data) {
|
||||
if (!data?.isRunning && !data?.finished) {
|
||||
stopPolling();
|
||||
return;
|
||||
}
|
||||
|
||||
if (data.finished) {
|
||||
stopPolling();
|
||||
|
||||
if (data.error) {
|
||||
updateStatus('error', `运行失败:${data.error}`);
|
||||
ElMessage.error(`任务执行失败:${data.error}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const results = data.results || [];
|
||||
if (results.length <= 1) {
|
||||
const total = results[0]?.data?.total ?? 0;
|
||||
updateStatus('success', `运行完成,抓取到 ${total} 条结果。`);
|
||||
ElMessage.success(`任务完成,抓取到 ${total} 条结果`);
|
||||
return;
|
||||
}
|
||||
|
||||
const successCount = results.filter((item) => !item.error).length;
|
||||
const failCount = results.filter((item) => item.error).length;
|
||||
updateStatus('success', `批量运行完成,成功 ${successCount} 个,失败 ${failCount} 个。`);
|
||||
ElMessage.success(`批量运行完成,成功 ${successCount} 个`);
|
||||
return;
|
||||
}
|
||||
|
||||
updateStatus(
|
||||
'info',
|
||||
`正在执行:${data.city || '任务'}(${data.current || 0}/${data.total || 0}),已用时 ${formatElapsed(data.elapsed)}`,
|
||||
);
|
||||
}
|
||||
|
||||
async function pollStatus() {
|
||||
try {
|
||||
const data = await fetchTaskStatus();
|
||||
applyRunStatus(data);
|
||||
if (data.finished) {
|
||||
await loadTasks();
|
||||
}
|
||||
} catch (error) {
|
||||
stopPolling();
|
||||
updateStatus('error', `状态轮询失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function startPolling() {
|
||||
stopPolling();
|
||||
runner.polling = true;
|
||||
runner.timer = setInterval(pollStatus, 2000);
|
||||
}
|
||||
|
||||
async function restoreRunningStatus() {
|
||||
try {
|
||||
const data = await fetchTaskStatus();
|
||||
applyRunStatus(data);
|
||||
if (data.isRunning) {
|
||||
startPolling();
|
||||
}
|
||||
} catch (error) {
|
||||
ElMessage.error(`任务状态恢复失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function submitTask(payload) {
|
||||
try {
|
||||
if (editingTask.value?.id) {
|
||||
await updateTask(editingTask.value.id, payload);
|
||||
ElMessage.success('任务已更新');
|
||||
} else {
|
||||
await createTask(payload);
|
||||
ElMessage.success('任务已创建');
|
||||
}
|
||||
|
||||
dialogOpen.value = false;
|
||||
await loadTasks();
|
||||
} catch (error) {
|
||||
ElMessage.error(`保存失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDelete(task) {
|
||||
try {
|
||||
await ElMessageBox.confirm(`确定要删除“${task.city || '未命名任务'}”吗?`, '删除确认', {
|
||||
type: 'warning',
|
||||
});
|
||||
await deleteTask(task.id);
|
||||
ElMessage.success('任务已删除');
|
||||
await loadTasks();
|
||||
} catch (error) {
|
||||
if (error !== 'cancel' && error !== 'close') {
|
||||
ElMessage.error(`删除失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function handleToggle(task, enabled) {
|
||||
try {
|
||||
await toggleTask(task.id, enabled);
|
||||
ElMessage.success(enabled ? '任务已启用' : '任务已禁用');
|
||||
await loadTasks();
|
||||
} catch (error) {
|
||||
ElMessage.error(`状态更新失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleRun(task) {
|
||||
if (runner.polling) {
|
||||
ElMessage.info('当前已有任务在运行,请稍后再试');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await runTask(task.id);
|
||||
updateStatus('info', `已开始运行任务:${task.city}`);
|
||||
ElMessage.success(`已开始运行:${task.city}`);
|
||||
startPolling();
|
||||
} catch (error) {
|
||||
updateStatus('error', `启动失败:${error.message}`);
|
||||
ElMessage.error(`启动失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleRunAll() {
|
||||
if (!enabledCount.value) {
|
||||
ElMessage.info('没有已启用的任务可以运行');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await ElMessageBox.confirm(`确定要运行全部 ${enabledCount.value} 个已启用任务吗?`, '批量执行确认', {
|
||||
type: 'warning',
|
||||
});
|
||||
await runAllTasks();
|
||||
updateStatus('info', '已开始运行全部启用任务,请稍候…');
|
||||
ElMessage.success('已开始运行全部启用任务');
|
||||
startPolling();
|
||||
} catch (error) {
|
||||
if (error !== 'cancel' && error !== 'close') {
|
||||
updateStatus('error', `批量启动失败:${error.message}`);
|
||||
ElMessage.error(`批量启动失败:${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="page-grid">
|
||||
<header class="page-head">
|
||||
<div>
|
||||
<h2>任务配置</h2>
|
||||
|
||||
</div>
|
||||
</header>
|
||||
|
||||
|
||||
|
||||
<el-card shadow="never">
|
||||
<div class="toolbar">
|
||||
<div class="toolbar__group">
|
||||
<el-button type="primary" @click="openCreateDialog">新增任务</el-button>
|
||||
<el-button type="success" :disabled="runner.polling" @click="handleRunAll">运行全部启用</el-button>
|
||||
</div>
|
||||
<div class="summary-text">{{ summaryText }}</div>
|
||||
</div>
|
||||
|
||||
<el-form label-position="top" class="filter-form">
|
||||
<el-row :gutter="16">
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="搜索城市">
|
||||
<el-input v-model="filters.city" placeholder="输入城市关键字" clearable />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
<el-col :xs="24" :sm="12">
|
||||
<el-form-item label="搜索板块">
|
||||
<el-input v-model="filters.plate" placeholder="输入板块关键字" clearable />
|
||||
</el-form-item>
|
||||
</el-col>
|
||||
</el-row>
|
||||
</el-form>
|
||||
|
||||
<el-alert v-if="status.visible" :title="status.message" :type="status.type" :closable="false" show-icon />
|
||||
|
||||
<el-table v-loading="isLoading" :data="pagedTasks" style="width: 100%; margin-top: 18px;" empty-text="没有匹配的任务">
|
||||
<el-table-column prop="city" label="城市" min-width="120" />
|
||||
<el-table-column prop="plateName" label="板块" min-width="120" />
|
||||
<el-table-column label="提示词" min-width="280">
|
||||
<template #default="{ row }">
|
||||
<div class="table-prompt">{{ row.prompt || '-' }}</div>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="模型" min-width="160">
|
||||
<template #default="{ row }">
|
||||
<el-tag type="info">{{ row.mode || 'qwen3.5-plus' }}</el-tag>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="状态" width="100">
|
||||
<template #default="{ row }">
|
||||
<el-tag :type="row.enabled ? 'success' : 'danger'">{{ row.enabled ? '启用' : '禁用' }}</el-tag>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="操作" min-width="260" fixed="right">
|
||||
<template #default="{ row }">
|
||||
<div class="table-actions">
|
||||
<el-button size="small" @click="openEditDialog(row)">编辑</el-button>
|
||||
<el-button size="small" type="primary" :disabled="runner.polling" @click="handleRun(row)">运行</el-button>
|
||||
<el-button size="small" plain @click="handleToggle(row, !row.enabled)">{{ row.enabled ? '禁用' : '启用' }}</el-button>
|
||||
<el-button size="small" type="danger" plain @click="handleDelete(row)">删除</el-button>
|
||||
</div>
|
||||
</template>
|
||||
</el-table-column>
|
||||
</el-table>
|
||||
|
||||
<div class="pagination-wrap">
|
||||
<el-pagination
|
||||
v-if="filteredTasks.length"
|
||||
background
|
||||
layout="total, prev, pager, next"
|
||||
:current-page="pagination.page"
|
||||
:page-size="pagination.pageSize"
|
||||
:total="filteredTasks.length"
|
||||
@current-change="pagination.page = $event"
|
||||
/>
|
||||
</div>
|
||||
</el-card>
|
||||
|
||||
<TaskDialog v-model="dialogOpen" :task="editingTask" @submit="submitTask" />
|
||||
</section>
|
||||
</template>
|
||||
@@ -1,56 +0,0 @@
|
||||
import { createRouter, createWebHistory } from 'vue-router';
|
||||
|
||||
const routes = [
|
||||
{
|
||||
path: '/',
|
||||
redirect: '/tasks',
|
||||
},
|
||||
{
|
||||
path: '/tasks',
|
||||
name: 'tasks',
|
||||
component: () => import('@/pages/TasksPage.vue'),
|
||||
meta: {
|
||||
title: '任务配置',
|
||||
subtitle: '管理城市任务、切换模型,并手动触发抓取流程。',
|
||||
},
|
||||
},
|
||||
{
|
||||
path: '/results',
|
||||
name: 'results',
|
||||
component: () => import('@/pages/ResultsPage.vue'),
|
||||
meta: {
|
||||
title: '抓取结果',
|
||||
subtitle: '查看抓取记录、按条件筛选,并快速定位失败任务。',
|
||||
},
|
||||
},
|
||||
{
|
||||
path: '/projects',
|
||||
name: 'projects',
|
||||
component: () => import('@/pages/ProjectsPage.vue'),
|
||||
meta: {
|
||||
title: '项目管理',
|
||||
subtitle: '对抓取出的项目去重、筛选与金额范围查询。',
|
||||
},
|
||||
},
|
||||
{
|
||||
path: '/settings',
|
||||
name: 'settings',
|
||||
component: () => import('@/pages/SettingsPage.vue'),
|
||||
meta: {
|
||||
title: '系统设置',
|
||||
subtitle: '维护 Agent、定时任务与邮件推送配置。',
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
const router = createRouter({
|
||||
history: createWebHistory(),
|
||||
routes,
|
||||
});
|
||||
|
||||
router.afterEach((to) => {
|
||||
const title = to.meta?.title ? `${to.meta.title} | 公告抓取与分析工具` : '公告抓取与分析工具';
|
||||
document.title = title;
|
||||
});
|
||||
|
||||
export default router;
|
||||
@@ -1,383 +0,0 @@
|
||||
@import url('https://fonts.googleapis.com/css2?family=Plus+Jakarta+Sans:wght@400;500;600;700;800&family=Noto+Sans+SC:wght@400;500;700&family=IBM+Plex+Mono:wght@500&display=swap');
|
||||
|
||||
:root {
|
||||
--bg-top: #edf7ff;
|
||||
--bg-mid: #f8fcf8;
|
||||
--bg-bottom: #fff8ef;
|
||||
--line: rgba(15, 34, 52, 0.1);
|
||||
--text: #11263a;
|
||||
--muted: #60758a;
|
||||
--shadow-lg: 0 24px 54px rgba(11, 42, 68, 0.14);
|
||||
--shadow-md: 0 12px 28px rgba(11, 42, 68, 0.08);
|
||||
--radius-lg: 18px;
|
||||
}
|
||||
|
||||
html,
|
||||
body,
|
||||
#app {
|
||||
min-height: 100%;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: 'Plus Jakarta Sans', 'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||
color: var(--text);
|
||||
background:
|
||||
radial-gradient(880px 520px at -8% -12%, rgba(13, 108, 184, 0.2), transparent 58%),
|
||||
radial-gradient(760px 440px at 110% -10%, rgba(15, 157, 140, 0.18), transparent 52%),
|
||||
radial-gradient(900px 520px at 60% 118%, rgba(215, 139, 30, 0.12), transparent 56%),
|
||||
linear-gradient(160deg, var(--bg-top) 0%, var(--bg-mid) 52%, var(--bg-bottom) 100%);
|
||||
--el-color-primary: #0d6cb8;
|
||||
--el-border-radius-base: 12px;
|
||||
--el-font-family: 'Plus Jakarta Sans', 'Noto Sans SC', 'PingFang SC', 'Microsoft YaHei', sans-serif;
|
||||
}
|
||||
|
||||
a {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.app-shell {
|
||||
min-height: 100vh;
|
||||
padding: 24px;
|
||||
}
|
||||
|
||||
.app-frame {
|
||||
max-width: 1480px;
|
||||
margin: 0 auto;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 28px;
|
||||
overflow: hidden;
|
||||
background: rgba(255, 255, 255, 0.68);
|
||||
box-shadow: var(--shadow-lg);
|
||||
backdrop-filter: blur(22px);
|
||||
}
|
||||
|
||||
.app-topbar {
|
||||
padding: 24px 28px;
|
||||
color: #eff8ff;
|
||||
background:
|
||||
radial-gradient(circle at top right, rgba(255, 255, 255, 0.2), transparent 26%),
|
||||
linear-gradient(125deg, #0c639f 0%, #0f85a2 55%, #17a86b 100%);
|
||||
}
|
||||
|
||||
.app-brand {
|
||||
display: flex;
|
||||
gap: 16px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.app-brand__mark {
|
||||
width: 54px;
|
||||
height: 54px;
|
||||
border-radius: 16px;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
color: #0d4c6e;
|
||||
font-size: 24px;
|
||||
font-weight: 800;
|
||||
background: linear-gradient(145deg, rgba(255, 255, 255, 0.96), rgba(232, 248, 255, 0.84));
|
||||
}
|
||||
|
||||
.app-brand__text h1 {
|
||||
margin: 0;
|
||||
font-size: 26px;
|
||||
}
|
||||
|
||||
.app-brand__text p {
|
||||
margin: 6px 0 0;
|
||||
font-size: 13px;
|
||||
opacity: 0.92;
|
||||
}
|
||||
|
||||
.app-menu.el-menu {
|
||||
padding: 10px 10px 0;
|
||||
border-bottom: 1px solid var(--line);
|
||||
background: rgba(255, 255, 255, 0.26);
|
||||
}
|
||||
|
||||
.app-menu.el-menu--horizontal > .el-menu-item {
|
||||
height: 48px;
|
||||
line-height: 48px;
|
||||
border-bottom: 0;
|
||||
margin: 0 8px 10px 0;
|
||||
border-radius: 14px;
|
||||
color: var(--muted);
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.app-menu.el-menu--horizontal > .el-menu-item.is-active {
|
||||
color: var(--el-color-primary);
|
||||
background: rgba(255, 255, 255, 0.9);
|
||||
}
|
||||
|
||||
.app-main {
|
||||
padding: 28px;
|
||||
}
|
||||
|
||||
.page-grid {
|
||||
display: grid;
|
||||
gap: 18px;
|
||||
}
|
||||
|
||||
.page-head {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 20px;
|
||||
align-items: flex-end;
|
||||
}
|
||||
|
||||
.page-head h2 {
|
||||
margin: 0;
|
||||
font-size: 28px;
|
||||
}
|
||||
|
||||
.page-head p {
|
||||
margin: 8px 0 0;
|
||||
color: var(--muted);
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.toolbar {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.toolbar__group {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.summary-text {
|
||||
color: var(--muted);
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.hero-panel.el-card {
|
||||
border: 0;
|
||||
color: #f7fcff;
|
||||
background: linear-gradient(140deg, rgba(13, 108, 184, 0.95), rgba(15, 157, 140, 0.92));
|
||||
box-shadow: var(--shadow-md);
|
||||
}
|
||||
|
||||
.hero-panel .el-card__body {
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.hero-panel .el-card__body::after {
|
||||
content: '';
|
||||
position: absolute;
|
||||
right: -70px;
|
||||
top: -80px;
|
||||
width: 240px;
|
||||
height: 240px;
|
||||
border-radius: 50%;
|
||||
background: rgba(255, 255, 255, 0.14);
|
||||
}
|
||||
|
||||
.hero-panel h3,
|
||||
.hero-panel p {
|
||||
position: relative;
|
||||
z-index: 1;
|
||||
}
|
||||
|
||||
.hero-panel h3 {
|
||||
margin: 0;
|
||||
font-size: 18px;
|
||||
}
|
||||
|
||||
.hero-panel p {
|
||||
margin: 8px 0 0;
|
||||
max-width: 760px;
|
||||
opacity: 0.92;
|
||||
}
|
||||
|
||||
.filter-form {
|
||||
margin-top: 18px;
|
||||
}
|
||||
|
||||
.table-prompt {
|
||||
display: -webkit-box;
|
||||
overflow: hidden;
|
||||
-webkit-box-orient: vertical;
|
||||
-webkit-line-clamp: 2;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.table-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.pagination-wrap {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
margin-top: 18px;
|
||||
}
|
||||
|
||||
.dialog-footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.result-card-plus {
|
||||
border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-md);
|
||||
}
|
||||
|
||||
.result-card-plus__header {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.result-card-plus__title {
|
||||
font-size: 18px;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.result-card-plus__meta {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-top: 8px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.result-card-plus__time {
|
||||
color: var(--muted);
|
||||
font-size: 13px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.result-card-plus__list {
|
||||
display: grid;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.result-card-plus__item {
|
||||
display: grid;
|
||||
grid-template-columns: 110px 1.6fr 150px 120px auto;
|
||||
gap: 12px;
|
||||
align-items: center;
|
||||
padding-bottom: 12px;
|
||||
border-bottom: 1px solid rgba(15, 34, 52, 0.08);
|
||||
}
|
||||
|
||||
.result-card-plus__item:last-child {
|
||||
border-bottom: 0;
|
||||
padding-bottom: 0;
|
||||
}
|
||||
|
||||
.result-card-plus__type {
|
||||
color: var(--el-color-primary);
|
||||
font-size: 13px;
|
||||
font-weight: 800;
|
||||
}
|
||||
|
||||
.result-card-plus__name {
|
||||
min-width: 0;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.result-card-plus__amount {
|
||||
color: #99591a;
|
||||
font-weight: 700;
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
.result-card-plus__date {
|
||||
color: var(--muted);
|
||||
font-size: 13px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.result-card-plus__action {
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
.result-card-plus__footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
.el-card {
|
||||
--el-card-border-color: rgba(15, 34, 52, 0.08);
|
||||
border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-md);
|
||||
}
|
||||
|
||||
.el-statistic {
|
||||
padding: 18px;
|
||||
border-radius: var(--radius-lg);
|
||||
background: rgba(255, 255, 255, 0.76);
|
||||
border: 1px solid rgba(15, 34, 52, 0.08);
|
||||
box-shadow: var(--shadow-md);
|
||||
}
|
||||
|
||||
.el-statistic__head {
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
.el-table {
|
||||
margin-top: 8px;
|
||||
}
|
||||
|
||||
.el-table,
|
||||
.el-table tr,
|
||||
.el-table th.el-table__cell,
|
||||
.el-table td.el-table__cell {
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.el-empty {
|
||||
padding: 36px 0;
|
||||
}
|
||||
|
||||
@media (max-width: 1120px) {
|
||||
.result-card-plus__item {
|
||||
grid-template-columns: 90px 1fr;
|
||||
}
|
||||
|
||||
.result-card-plus__amount,
|
||||
.result-card-plus__date,
|
||||
.result-card-plus__action {
|
||||
text-align: left;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 840px) {
|
||||
.app-shell {
|
||||
padding: 12px;
|
||||
}
|
||||
|
||||
.app-main,
|
||||
.app-topbar {
|
||||
padding: 18px;
|
||||
}
|
||||
|
||||
.page-head {
|
||||
align-items: flex-start;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.app-menu.el-menu--horizontal {
|
||||
display: block;
|
||||
height: auto;
|
||||
}
|
||||
|
||||
.app-menu.el-menu--horizontal > .el-menu-item {
|
||||
display: block;
|
||||
}
|
||||
}
|
||||
@@ -1,26 +0,0 @@
|
||||
export function formatDateTime(value) {
|
||||
if (!value) return '-';
|
||||
const date = new Date(value);
|
||||
if (Number.isNaN(date.getTime())) return value;
|
||||
return date.toLocaleString('zh-CN');
|
||||
}
|
||||
|
||||
export function formatYuan(value) {
|
||||
if (typeof value !== 'number' || !Number.isFinite(value)) return '-';
|
||||
return `${value.toLocaleString('zh-CN', { maximumFractionDigits: 2 })} 元`;
|
||||
}
|
||||
|
||||
export function formatElapsed(totalSeconds) {
|
||||
const seconds = Number(totalSeconds) || 0;
|
||||
const minutes = Math.floor(seconds / 60);
|
||||
const remainder = seconds % 60;
|
||||
return minutes > 0 ? `${minutes} 分 ${remainder} 秒` : `${remainder} 秒`;
|
||||
}
|
||||
|
||||
export function pickResultName(item = {}) {
|
||||
return item.project_name || item.projectName || item.title || item.name || item.bidName || '-';
|
||||
}
|
||||
|
||||
export function pickResultLink(item = {}) {
|
||||
return item.detail_link || item.target_link || item.url || item.href || '';
|
||||
}
|
||||
@@ -6,7 +6,7 @@
|
||||
},
|
||||
"scheduler": {
|
||||
"enabled": true,
|
||||
"cronTime": "0 9 * * *",
|
||||
"cronTime": "40 08 * * *",
|
||||
"threshold": 100000,
|
||||
"description": "每天9点采集当日项目",
|
||||
"timeRange": "thisMonth"
|
||||
@@ -16,6 +16,6 @@
|
||||
"smtpPort": 587,
|
||||
"smtpUser": "1076597680@qq.com",
|
||||
"smtpPass": "nfrjdiraqddsjeeh",
|
||||
"recipients": "1650243281@qq.com"
|
||||
"recipients": "5482498@qq.com"
|
||||
}
|
||||
}
|
||||
13
disable-all-tasks.bat
Normal file
13
disable-all-tasks.bat
Normal file
@@ -0,0 +1,13 @@
|
||||
@echo off
|
||||
chcp 65001 >nul
|
||||
cd /d "%~dp0"
|
||||
|
||||
echo ========================================
|
||||
echo 批量禁用所有任务
|
||||
echo ========================================
|
||||
echo.
|
||||
|
||||
node disable-all-tasks.js
|
||||
|
||||
echo.
|
||||
pause
|
||||
65
disable-all-tasks.js
Normal file
65
disable-all-tasks.js
Normal file
@@ -0,0 +1,65 @@
|
||||
// 批量禁用所有任务脚本
|
||||
// 用法: node disable-all-tasks.js
|
||||
|
||||
import Database from 'better-sqlite3';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname, join } from 'path';
|
||||
import { existsSync, mkdirSync } from 'fs';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
const DEFAULT_DB_DIR = join(__dirname, 'data');
|
||||
const DEFAULT_DB_PATH = join(DEFAULT_DB_DIR, 'results.db');
|
||||
const LEGACY_DB_PATH = join(DEFAULT_DB_DIR, 'results.sqlite');
|
||||
|
||||
// 确定数据库路径
|
||||
let DB_PATH = process.env.APP_DB_PATH || process.env.RESULTS_DB_PATH;
|
||||
if (!DB_PATH) {
|
||||
if (existsSync(DEFAULT_DB_PATH)) {
|
||||
DB_PATH = DEFAULT_DB_PATH;
|
||||
} else if (existsSync(LEGACY_DB_PATH)) {
|
||||
DB_PATH = LEGACY_DB_PATH;
|
||||
} else {
|
||||
DB_PATH = DEFAULT_DB_PATH;
|
||||
}
|
||||
}
|
||||
|
||||
if (!existsSync(DB_PATH)) {
|
||||
console.error('数据库文件不存在:', DB_PATH);
|
||||
console.log('请确保项目已初始化并运行过至少一次');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
try {
|
||||
const db = new Database(DB_PATH);
|
||||
|
||||
// 先查询当前启用的任务数量
|
||||
const enabledCount = db.prepare('SELECT COUNT(*) as count FROM tasks WHERE enabled = 1').get();
|
||||
console.log(`当前启用的任务数量: ${enabledCount.count}`);
|
||||
|
||||
// 查询总任务数量
|
||||
const totalCount = db.prepare('SELECT COUNT(*) as count FROM tasks').get();
|
||||
console.log(`总任务数量: ${totalCount.count}`);
|
||||
|
||||
if (enabledCount.count === 0) {
|
||||
console.log('没有需要禁用的任务');
|
||||
db.close();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// 执行批量禁用
|
||||
const result = db.prepare('UPDATE tasks SET enabled = 0').run();
|
||||
console.log(`已禁用 ${result.changes} 个任务`);
|
||||
|
||||
// 验证结果
|
||||
const newEnabledCount = db.prepare('SELECT COUNT(*) as count FROM tasks WHERE enabled = 1').get();
|
||||
console.log(`禁用后启用的任务数量: ${newEnabledCount.count}`);
|
||||
|
||||
db.close();
|
||||
console.log('操作完成');
|
||||
|
||||
} catch (error) {
|
||||
console.error('操作失败:', error.message);
|
||||
process.exit(1);
|
||||
}
|
||||
1
logs/server.pid
Normal file
1
logs/server.pid
Normal file
@@ -0,0 +1 @@
|
||||
15556
|
||||
2391
package-lock.json
generated
2391
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
18
package.json
18
package.json
@@ -2,35 +2,19 @@
|
||||
"name": "njggzy-scraper",
|
||||
"version": "2.0.0",
|
||||
"type": "module",
|
||||
"description": "公告抓取与分析工具",
|
||||
"description": "南京公共资源交易平台 - 合同估算价采集工具",
|
||||
"main": "src/server.js",
|
||||
"scripts": {
|
||||
"dev": "concurrently \"npm:dev:server\" \"npm:dev:client\"",
|
||||
"dev:server": "node src/server.js",
|
||||
"dev:client": "vite --config vite.config.js",
|
||||
"build": "vite build --config vite.config.js",
|
||||
"preview": "vite preview --config vite.config.js",
|
||||
"start": "node src/server.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mendable/firecrawl-js": "latest",
|
||||
"axios": "^1.11.0",
|
||||
"better-sqlite3": "^12.8.0",
|
||||
"cors": "^2.8.5",
|
||||
"dotenv": "^17.2.3",
|
||||
"element-plus": "^2.11.4",
|
||||
"express": "^5.2.1",
|
||||
"node-cron": "^4.2.1",
|
||||
"nodemailer": "^7.0.11",
|
||||
"vue": "^3.5.18",
|
||||
"vue-router": "^4.5.1",
|
||||
"zod": "^3.24.2"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@vitejs/plugin-vue": "^6.0.1",
|
||||
"concurrently": "^9.2.1",
|
||||
"unplugin-auto-import": "^20.2.0",
|
||||
"unplugin-vue-components": "^29.0.0",
|
||||
"vite": "^7.1.3"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -622,6 +622,7 @@
|
||||
<div style="display:flex;gap:10px;align-items:center;">
|
||||
<button class="btn btn-primary btn-sm" onclick="searchProjectResults()">查询</button>
|
||||
<button class="btn btn-secondary btn-sm" onclick="resetProjectFilters()">重置</button>
|
||||
<button class="btn btn-success btn-sm" id="btnExportProjects" onclick="exportProjectResults(this)">导出</button>
|
||||
<!-- <button class="btn btn-danger btn-sm" onclick="clearResults()">清空全部</button> -->
|
||||
</div>
|
||||
</div>
|
||||
@@ -687,14 +688,51 @@
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Cron 表达式</label>
|
||||
<input type="text" id="cfgCronTime" placeholder="0 9 * * *">
|
||||
<label>执行方式</label>
|
||||
<select id="cfgScheduleMode" onchange="handleSchedulerModeChange()">
|
||||
<option value="daily">每天</option>
|
||||
<option value="weekly">每周</option>
|
||||
<option value="monthly">每月</option>
|
||||
<option value="custom">高级 Cron</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>描述</label>
|
||||
<input type="text" id="cfgDescription" placeholder="每天9点采集">
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-row">
|
||||
<div class="form-group">
|
||||
<label>执行时间</label>
|
||||
<input type="time" id="cfgScheduleTime" value="09:00" onchange="updateSchedulerCronPreview()">
|
||||
</div>
|
||||
<div class="form-group" id="cfgWeekdayGroup" style="display:none;">
|
||||
<label>星期</label>
|
||||
<select id="cfgWeekday" onchange="updateSchedulerCronPreview()">
|
||||
<option value="1">周一</option>
|
||||
<option value="2">周二</option>
|
||||
<option value="3">周三</option>
|
||||
<option value="4">周四</option>
|
||||
<option value="5">周五</option>
|
||||
<option value="6">周六</option>
|
||||
<option value="0">周日</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group" id="cfgMonthDayGroup" style="display:none;">
|
||||
<label>每月几号</label>
|
||||
<input type="number" id="cfgMonthDay" min="1" max="31" value="1" onchange="updateSchedulerCronPreview()">
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-row">
|
||||
<div class="form-group" id="cfgCustomCronGroup" style="display:none;">
|
||||
<label>高级 Cron 表达式</label>
|
||||
<input type="text" id="cfgCronTime" placeholder="0 9 * * *" oninput="updateSchedulerCronPreview()">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>计划预览</label>
|
||||
<input type="text" id="cfgCronPreview" readonly placeholder="每天 09:00">
|
||||
</div>
|
||||
</div>
|
||||
<button class="btn btn-secondary btn-sm" onclick="triggerScheduledTask()">立即执行定时任务</button>
|
||||
<span id="schedulerStatus" style="margin-left:12px;font-size:13px;color:#888;"></span>
|
||||
</div>
|
||||
@@ -1130,21 +1168,10 @@
|
||||
|
||||
async function loadProjectResults() {
|
||||
try {
|
||||
await loadProjectFilters();
|
||||
const params = new URLSearchParams({ page: currentProjectsPage, pageSize: 10 });
|
||||
const projectFilters = getProjectFilterValues();
|
||||
|
||||
if (projectFilters.minAmount && projectFilters.maxAmount && Number(projectFilters.minAmount) > Number(projectFilters.maxAmount)) {
|
||||
throw new Error('最小金额不能大于最大金额');
|
||||
}
|
||||
|
||||
if (projectFilters.startDate && projectFilters.endDate && projectFilters.startDate > projectFilters.endDate) {
|
||||
throw new Error('开始日期不能晚于结束日期');
|
||||
}
|
||||
|
||||
Object.entries(projectFilters).forEach(([key, value]) => {
|
||||
if (value !== '') params.set(key, value);
|
||||
});
|
||||
validateProjectFilters(projectFilters);
|
||||
await loadProjectFilters();
|
||||
const params = buildProjectQueryParams({ projectFilters, includePagination: true });
|
||||
|
||||
const res = await fetch('/api/projects?' + params);
|
||||
const json = await res.json();
|
||||
@@ -1198,6 +1225,31 @@
|
||||
};
|
||||
}
|
||||
|
||||
function validateProjectFilters(projectFilters) {
|
||||
if (projectFilters.minAmount && projectFilters.maxAmount && Number(projectFilters.minAmount) > Number(projectFilters.maxAmount)) {
|
||||
throw new Error('最小金额不能大于最大金额');
|
||||
}
|
||||
|
||||
if (projectFilters.startDate && projectFilters.endDate && projectFilters.startDate > projectFilters.endDate) {
|
||||
throw new Error('开始日期不能晚于结束日期');
|
||||
}
|
||||
}
|
||||
|
||||
function buildProjectQueryParams({ projectFilters = getProjectFilterValues(), includePagination = true } = {}) {
|
||||
const params = new URLSearchParams();
|
||||
|
||||
if (includePagination) {
|
||||
params.set('page', currentProjectsPage);
|
||||
params.set('pageSize', 10);
|
||||
}
|
||||
|
||||
Object.entries(projectFilters).forEach(([key, value]) => {
|
||||
if (value !== '') params.set(key, value);
|
||||
});
|
||||
|
||||
return params;
|
||||
}
|
||||
|
||||
function searchProjectResults() {
|
||||
currentProjectsPage = 1;
|
||||
loadProjectResults();
|
||||
@@ -1220,6 +1272,73 @@
|
||||
loadProjectResults();
|
||||
}
|
||||
|
||||
async function exportProjectResults(buttonEl) {
|
||||
const btn = buttonEl || document.getElementById('btnExportProjects');
|
||||
const originalText = btn ? btn.textContent : '导出';
|
||||
|
||||
try {
|
||||
const projectFilters = getProjectFilterValues();
|
||||
validateProjectFilters(projectFilters);
|
||||
|
||||
if (btn) {
|
||||
btn.disabled = true;
|
||||
btn.textContent = '导出中...';
|
||||
}
|
||||
|
||||
const res = await fetch('/api/projects/export?' + buildProjectQueryParams({
|
||||
projectFilters,
|
||||
includePagination: false,
|
||||
}));
|
||||
|
||||
if (!res.ok) {
|
||||
let message = '导出失败';
|
||||
try {
|
||||
const json = await res.json();
|
||||
message = json.error || message;
|
||||
} catch (_error) {
|
||||
// Ignore JSON parse failure and use the fallback message.
|
||||
}
|
||||
throw new Error(message);
|
||||
}
|
||||
|
||||
const blob = await res.blob();
|
||||
const downloadUrl = URL.createObjectURL(blob);
|
||||
const link = document.createElement('a');
|
||||
link.href = downloadUrl;
|
||||
link.download = getDownloadFilename(
|
||||
res.headers.get('content-disposition'),
|
||||
`projects-${new Date().toISOString().slice(0, 19).replace(/[-:]/g, '').replace('T', '-')}.csv`
|
||||
);
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
link.remove();
|
||||
URL.revokeObjectURL(downloadUrl);
|
||||
} catch (e) {
|
||||
alert('导出失败: ' + e.message);
|
||||
} finally {
|
||||
if (btn) {
|
||||
btn.disabled = false;
|
||||
btn.textContent = originalText;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function getDownloadFilename(contentDisposition, fallbackName) {
|
||||
if (!contentDisposition) return fallbackName;
|
||||
|
||||
const utf8Match = contentDisposition.match(/filename\*=UTF-8''([^;]+)/i);
|
||||
if (utf8Match && utf8Match[1]) {
|
||||
return decodeURIComponent(utf8Match[1]);
|
||||
}
|
||||
|
||||
const asciiMatch = contentDisposition.match(/filename="([^"]+)"/i) || contentDisposition.match(/filename=([^;]+)/i);
|
||||
if (asciiMatch && asciiMatch[1]) {
|
||||
return asciiMatch[1].trim();
|
||||
}
|
||||
|
||||
return fallbackName;
|
||||
}
|
||||
|
||||
function renderResults({ data, total, page, pageSize, listId, paginationId }) {
|
||||
const container = document.getElementById(listId);
|
||||
if (!data || data.length === 0) {
|
||||
@@ -1278,7 +1397,7 @@
|
||||
tbody.innerHTML = data.map(project => `
|
||||
<tr>
|
||||
<td>${escHtml(project.city || '-')}</td>
|
||||
<td>${escHtml(project.section || project.type || '-')}</td>
|
||||
<td>${escHtml( project.type ||project.section || '-')}</td>
|
||||
<td>${escHtml(project.projectName || '-')}</td>
|
||||
<td>${formatAmount(project.amountYuan)}</td>
|
||||
<td>${escHtml(project.date || '-')}</td>
|
||||
@@ -1351,6 +1470,123 @@
|
||||
.replace(/'/g, ''');
|
||||
}
|
||||
|
||||
function padSchedulerNumber(value) {
|
||||
return String(value).padStart(2, '0');
|
||||
}
|
||||
|
||||
function normalizeSchedulerTime(value) {
|
||||
const match = /^(\d{1,2}):(\d{1,2})$/.exec(String(value || '').trim());
|
||||
if (!match) return '09:00';
|
||||
|
||||
const hour = Math.min(23, Math.max(0, parseInt(match[1], 10)));
|
||||
const minute = Math.min(59, Math.max(0, parseInt(match[2], 10)));
|
||||
return `${padSchedulerNumber(hour)}:${padSchedulerNumber(minute)}`;
|
||||
}
|
||||
|
||||
function splitSchedulerTime(value) {
|
||||
const [hour, minute] = normalizeSchedulerTime(value).split(':');
|
||||
return { hour, minute };
|
||||
}
|
||||
|
||||
function describeScheduler(mode, time, weekday, monthDay, cronTime) {
|
||||
const normalizedTime = normalizeSchedulerTime(time);
|
||||
const weekdayMap = {
|
||||
'0': '周日',
|
||||
'1': '周一',
|
||||
'2': '周二',
|
||||
'3': '周三',
|
||||
'4': '周四',
|
||||
'5': '周五',
|
||||
'6': '周六',
|
||||
};
|
||||
|
||||
if (mode === 'daily') return `每天 ${normalizedTime}`;
|
||||
if (mode === 'weekly') return `每周${weekdayMap[String(weekday)] || '周一'} ${normalizedTime}`;
|
||||
if (mode === 'monthly') return `每月 ${monthDay} 号 ${normalizedTime}`;
|
||||
return cronTime || '0 9 * * *';
|
||||
}
|
||||
|
||||
function parseSchedulerCron(cronTime) {
|
||||
const normalizedCron = String(cronTime || '0 9 * * *').trim().replace(/\s+/g, ' ');
|
||||
const parts = normalizedCron.split(' ');
|
||||
|
||||
const fallback = {
|
||||
mode: 'custom',
|
||||
time: '09:00',
|
||||
weekday: '1',
|
||||
monthDay: '1',
|
||||
cronTime: normalizedCron || '0 9 * * *',
|
||||
};
|
||||
|
||||
if (parts.length !== 5) return fallback;
|
||||
|
||||
const [minute, hour, dayOfMonth, month, dayOfWeek] = parts;
|
||||
if (!/^\d+$/.test(minute) || !/^\d+$/.test(hour)) return fallback;
|
||||
|
||||
const time = `${padSchedulerNumber(parseInt(hour, 10))}:${padSchedulerNumber(parseInt(minute, 10))}`;
|
||||
|
||||
if (dayOfMonth === '*' && month === '*' && dayOfWeek === '*') {
|
||||
return { mode: 'daily', time, weekday: '1', monthDay: '1', cronTime: normalizedCron };
|
||||
}
|
||||
|
||||
if (dayOfMonth === '*' && month === '*' && /^(0|1|2|3|4|5|6|7)$/.test(dayOfWeek)) {
|
||||
return {
|
||||
mode: 'weekly',
|
||||
time,
|
||||
weekday: dayOfWeek === '7' ? '0' : dayOfWeek,
|
||||
monthDay: '1',
|
||||
cronTime: normalizedCron,
|
||||
};
|
||||
}
|
||||
|
||||
if (/^\d+$/.test(dayOfMonth) && month === '*' && dayOfWeek === '*') {
|
||||
const normalizedMonthDay = Math.min(31, Math.max(1, parseInt(dayOfMonth, 10)));
|
||||
return {
|
||||
mode: 'monthly',
|
||||
time,
|
||||
weekday: '1',
|
||||
monthDay: String(normalizedMonthDay),
|
||||
cronTime: normalizedCron,
|
||||
};
|
||||
}
|
||||
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function buildSchedulerCronFromInputs() {
|
||||
const mode = document.getElementById('cfgScheduleMode').value;
|
||||
const time = normalizeSchedulerTime(document.getElementById('cfgScheduleTime').value);
|
||||
const { hour, minute } = splitSchedulerTime(time);
|
||||
const weekday = document.getElementById('cfgWeekday').value || '1';
|
||||
const monthDayValue = parseInt(document.getElementById('cfgMonthDay').value, 10);
|
||||
const monthDay = Number.isFinite(monthDayValue) ? Math.min(31, Math.max(1, monthDayValue)) : 1;
|
||||
|
||||
if (mode === 'daily') return `${minute} ${hour} * * *`;
|
||||
if (mode === 'weekly') return `${minute} ${hour} * * ${weekday}`;
|
||||
if (mode === 'monthly') return `${minute} ${hour} ${monthDay} * *`;
|
||||
return document.getElementById('cfgCronTime').value.trim() || '0 9 * * *';
|
||||
}
|
||||
|
||||
function updateSchedulerCronPreview() {
|
||||
const cronTime = buildSchedulerCronFromInputs();
|
||||
document.getElementById('cfgCronTime').value = cronTime;
|
||||
document.getElementById('cfgCronPreview').value = describeScheduler(
|
||||
document.getElementById('cfgScheduleMode').value,
|
||||
document.getElementById('cfgScheduleTime').value,
|
||||
document.getElementById('cfgWeekday').value,
|
||||
document.getElementById('cfgMonthDay').value,
|
||||
cronTime
|
||||
);
|
||||
}
|
||||
|
||||
function handleSchedulerModeChange() {
|
||||
const mode = document.getElementById('cfgScheduleMode').value;
|
||||
document.getElementById('cfgWeekdayGroup').style.display = mode === 'weekly' ? '' : 'none';
|
||||
document.getElementById('cfgMonthDayGroup').style.display = mode === 'monthly' ? '' : 'none';
|
||||
document.getElementById('cfgCustomCronGroup').style.display = mode === 'custom' ? '' : 'none';
|
||||
updateSchedulerCronPreview();
|
||||
}
|
||||
|
||||
// ===== 设置 =====
|
||||
async function loadSettings() {
|
||||
try {
|
||||
@@ -1366,8 +1602,14 @@
|
||||
|
||||
// Scheduler
|
||||
document.getElementById('cfgSchedulerEnabled').value = String(cfg.scheduler?.enabled ?? false);
|
||||
document.getElementById('cfgCronTime').value = cfg.scheduler?.cronTime || '0 9 * * *';
|
||||
const schedulerUi = parseSchedulerCron(cfg.scheduler?.cronTime || '0 9 * * *');
|
||||
document.getElementById('cfgScheduleMode').value = schedulerUi.mode;
|
||||
document.getElementById('cfgScheduleTime').value = schedulerUi.time;
|
||||
document.getElementById('cfgWeekday').value = schedulerUi.weekday;
|
||||
document.getElementById('cfgMonthDay').value = schedulerUi.monthDay;
|
||||
document.getElementById('cfgCronTime').value = schedulerUi.cronTime;
|
||||
document.getElementById('cfgDescription').value = cfg.scheduler?.description || '';
|
||||
handleSchedulerModeChange();
|
||||
|
||||
// Email
|
||||
document.getElementById('cfgSmtpHost').value = cfg.email?.smtpHost || '';
|
||||
@@ -1391,6 +1633,14 @@
|
||||
|
||||
async function saveSettings() {
|
||||
try {
|
||||
const cronTime = buildSchedulerCronFromInputs();
|
||||
if (document.getElementById('cfgScheduleMode').value === 'custom') {
|
||||
const cronParts = cronTime.trim().split(/\s+/);
|
||||
if (cronParts.length !== 5) {
|
||||
throw new Error('Cron 表达式格式错误,请填写 5 段');
|
||||
}
|
||||
}
|
||||
|
||||
const cfg = {
|
||||
agent: {
|
||||
baseUrl: document.getElementById('cfgBaseUrl').value.trim(),
|
||||
@@ -1399,7 +1649,7 @@
|
||||
},
|
||||
scheduler: {
|
||||
enabled: document.getElementById('cfgSchedulerEnabled').value === 'true',
|
||||
cronTime: document.getElementById('cfgCronTime').value.trim() || '0 9 * * *',
|
||||
cronTime,
|
||||
description: document.getElementById('cfgDescription').value.trim(),
|
||||
},
|
||||
email: {
|
||||
@@ -1463,6 +1713,7 @@
|
||||
// ===== 初始化 =====
|
||||
initProjectSearchInputs();
|
||||
loadTasks();
|
||||
handleSchedulerModeChange();
|
||||
loadSettings();
|
||||
</script>
|
||||
</body>
|
||||
|
||||
100
restart.ps1
Normal file
100
restart.ps1
Normal file
@@ -0,0 +1,100 @@
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||
$LogDir = Join-Path $ScriptDir 'logs'
|
||||
$PidFile = Join-Path $LogDir 'server.pid'
|
||||
$LogFile = Join-Path $LogDir 'server.log'
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $LogDir | Out-Null
|
||||
|
||||
function Get-PortFromEnvFile {
|
||||
$envFile = Join-Path $ScriptDir '.env'
|
||||
if (-not (Test-Path $envFile)) {
|
||||
return $null
|
||||
}
|
||||
|
||||
$line = Get-Content $envFile | Where-Object { $_ -match '^PORT=' } | Select-Object -Last 1
|
||||
if (-not $line) {
|
||||
return $null
|
||||
}
|
||||
|
||||
return ($line -replace '^PORT=', '').Trim()
|
||||
}
|
||||
|
||||
function Get-ProjectServerProcesses {
|
||||
Get-CimInstance Win32_Process | Where-Object {
|
||||
$_.Name -eq 'node.exe' -and
|
||||
$_.CommandLine -like '*src/server.js*' -and
|
||||
$_.CommandLine -like "*$ScriptDir*"
|
||||
}
|
||||
}
|
||||
|
||||
function Stop-ExistingServer {
|
||||
$processes = @(Get-ProjectServerProcesses)
|
||||
|
||||
if ($processes.Count -eq 0) {
|
||||
Write-Host "No existing server process found for $ScriptDir"
|
||||
return
|
||||
}
|
||||
|
||||
$ids = $processes | ForEach-Object { $_.ProcessId }
|
||||
Write-Host ("Stopping existing server process(es): " + ($ids -join ', '))
|
||||
|
||||
foreach ($process in $processes) {
|
||||
Stop-Process -Id $process.ProcessId -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Start-Sleep -Seconds 2
|
||||
}
|
||||
|
||||
function Start-Server {
|
||||
$port = if ($env:PORT) { $env:PORT } else { Get-PortFromEnvFile }
|
||||
if (-not $port) { $port = '5000' }
|
||||
|
||||
Write-Host "Starting server from $ScriptDir on port $port"
|
||||
|
||||
$command = "Set-Location -LiteralPath '$ScriptDir'; node src/server.js *>> '$LogFile'"
|
||||
$process = Start-Process -FilePath 'powershell.exe' `
|
||||
-ArgumentList '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', $command `
|
||||
-WindowStyle Hidden `
|
||||
-PassThru
|
||||
|
||||
Set-Content -Path $PidFile -Value $process.Id
|
||||
Write-Host "Started PID: $($process.Id)"
|
||||
|
||||
return $port
|
||||
}
|
||||
|
||||
function Show-Status {
|
||||
param(
|
||||
[string]$Port
|
||||
)
|
||||
|
||||
Start-Sleep -Seconds 2
|
||||
|
||||
Write-Host ''
|
||||
Write-Host 'Active project server process(es):'
|
||||
$processes = @(Get-ProjectServerProcesses)
|
||||
if ($processes.Count -eq 0) {
|
||||
Write-Host 'None'
|
||||
} else {
|
||||
$processes | Select-Object ProcessId, Name, CommandLine | Format-Table -AutoSize
|
||||
}
|
||||
|
||||
Write-Host ''
|
||||
Write-Host 'Port check:'
|
||||
Get-NetTCPConnection -LocalPort ([int]$Port) -State Listen -ErrorAction SilentlyContinue |
|
||||
Select-Object LocalAddress, LocalPort, OwningProcess | Format-Table -AutoSize
|
||||
|
||||
Write-Host ''
|
||||
Write-Host 'Recent log output:'
|
||||
if (Test-Path $LogFile) {
|
||||
Get-Content $LogFile -Tail 30
|
||||
} else {
|
||||
Write-Host 'No log file yet.'
|
||||
}
|
||||
}
|
||||
|
||||
Stop-ExistingServer
|
||||
$port = Start-Server
|
||||
Show-Status -Port $port
|
||||
83
restart.sh
Normal file
83
restart.sh
Normal file
@@ -0,0 +1,83 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOG_DIR="$SCRIPT_DIR/logs"
|
||||
PID_FILE="$LOG_DIR/server.pid"
|
||||
LOG_FILE="$LOG_DIR/server.log"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
PORT="${PORT:-}"
|
||||
if [[ -z "$PORT" && -f "$SCRIPT_DIR/.env" ]]; then
|
||||
PORT="$(grep -E '^PORT=' "$SCRIPT_DIR/.env" | tail -n 1 | cut -d '=' -f 2- | tr -d '\r' || true)"
|
||||
fi
|
||||
PORT="${PORT:-5000}"
|
||||
|
||||
find_project_server_pids() {
|
||||
pgrep -f "node src/server.js" | while read -r pid; do
|
||||
[[ -n "$pid" ]] || continue
|
||||
[[ -d "/proc/$pid" ]] || continue
|
||||
|
||||
local cwd
|
||||
cwd="$(readlink -f "/proc/$pid/cwd" 2>/dev/null || true)"
|
||||
if [[ "$cwd" == "$SCRIPT_DIR" ]]; then
|
||||
echo "$pid"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
stop_existing_server() {
|
||||
local pids
|
||||
pids="$(find_project_server_pids || true)"
|
||||
|
||||
if [[ -z "$pids" ]]; then
|
||||
echo "No existing server process found for $SCRIPT_DIR"
|
||||
return
|
||||
fi
|
||||
|
||||
echo "Stopping existing server process(es): $pids"
|
||||
while read -r pid; do
|
||||
[[ -n "$pid" ]] || continue
|
||||
kill "$pid" 2>/dev/null || true
|
||||
done <<< "$pids"
|
||||
|
||||
sleep 2
|
||||
|
||||
local remaining
|
||||
remaining="$(find_project_server_pids || true)"
|
||||
if [[ -n "$remaining" ]]; then
|
||||
echo "Force killing remaining process(es): $remaining"
|
||||
while read -r pid; do
|
||||
[[ -n "$pid" ]] || continue
|
||||
kill -9 "$pid" 2>/dev/null || true
|
||||
done <<< "$remaining"
|
||||
fi
|
||||
}
|
||||
|
||||
start_server() {
|
||||
cd "$SCRIPT_DIR"
|
||||
echo "Starting server from $SCRIPT_DIR on port $PORT"
|
||||
nohup node src/server.js >> "$LOG_FILE" 2>&1 &
|
||||
local pid=$!
|
||||
echo "$pid" > "$PID_FILE"
|
||||
echo "Started PID: $pid"
|
||||
}
|
||||
|
||||
show_status() {
|
||||
sleep 2
|
||||
echo
|
||||
echo "Active project server process(es):"
|
||||
find_project_server_pids || true
|
||||
echo
|
||||
echo "Port check:"
|
||||
ss -lntp 2>/dev/null | grep ":$PORT" || true
|
||||
echo
|
||||
echo "Recent log output:"
|
||||
tail -n 30 "$LOG_FILE" 2>/dev/null || true
|
||||
}
|
||||
|
||||
stop_existing_server
|
||||
start_server
|
||||
show_status
|
||||
@@ -1,15 +1,23 @@
|
||||
import Database from 'better-sqlite3';
|
||||
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'fs';
|
||||
import {
|
||||
copyFileSync,
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
readFileSync,
|
||||
renameSync,
|
||||
unlinkSync,
|
||||
writeFileSync,
|
||||
} from 'fs';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname, join } from 'path';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
const DB_PATH =
|
||||
process.env.APP_DB_PATH ||
|
||||
process.env.RESULTS_DB_PATH ||
|
||||
join(__dirname, '..', 'data', 'results.sqlite');
|
||||
const DEFAULT_DB_DIR = join(__dirname, '..', 'data');
|
||||
const DEFAULT_DB_PATH = join(DEFAULT_DB_DIR, 'results.db');
|
||||
const LEGACY_DB_PATH = join(DEFAULT_DB_DIR, 'results.sqlite');
|
||||
const DB_PATH = resolveDbPath();
|
||||
const CONFIG_PATH = join(__dirname, '..', 'config.json');
|
||||
const MAX_RESULT_RECORDS = 500;
|
||||
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
|
||||
@@ -22,6 +30,55 @@ function clone(value) {
|
||||
return JSON.parse(JSON.stringify(value));
|
||||
}
|
||||
|
||||
function removeFileIfExists(filePath) {
|
||||
if (!existsSync(filePath)) return;
|
||||
unlinkSync(filePath);
|
||||
}
|
||||
|
||||
function migrateLegacyDbIfNeeded(nextPath, legacyPath) {
|
||||
if (existsSync(nextPath) || !existsSync(legacyPath)) return nextPath;
|
||||
|
||||
const legacyDb = new Database(legacyPath);
|
||||
|
||||
try {
|
||||
legacyDb.pragma('wal_checkpoint(TRUNCATE)');
|
||||
} catch (_error) {
|
||||
// Ignore checkpoint failures and still attempt to switch to single-file mode.
|
||||
}
|
||||
|
||||
legacyDb.pragma('journal_mode = DELETE');
|
||||
legacyDb.close();
|
||||
|
||||
try {
|
||||
renameSync(legacyPath, nextPath);
|
||||
} catch (_error) {
|
||||
copyFileSync(legacyPath, nextPath);
|
||||
}
|
||||
|
||||
removeFileIfExists(`${legacyPath}-shm`);
|
||||
removeFileIfExists(`${legacyPath}-wal`);
|
||||
|
||||
return nextPath;
|
||||
}
|
||||
|
||||
function resolveDbPath() {
|
||||
const explicitPath = process.env.APP_DB_PATH || process.env.RESULTS_DB_PATH;
|
||||
if (explicitPath) return explicitPath;
|
||||
|
||||
mkdirSync(DEFAULT_DB_DIR, { recursive: true });
|
||||
|
||||
try {
|
||||
return migrateLegacyDbIfNeeded(DEFAULT_DB_PATH, LEGACY_DB_PATH);
|
||||
} catch (error) {
|
||||
if (existsSync(LEGACY_DB_PATH)) {
|
||||
console.warn(`[resultStore] Legacy database migration skipped: ${error.message}`);
|
||||
return LEGACY_DB_PATH;
|
||||
}
|
||||
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function generateResultId() {
|
||||
return `result-${Date.now()}-${Math.random().toString(36).slice(2, 7)}`;
|
||||
}
|
||||
@@ -141,7 +198,13 @@ function getDb() {
|
||||
|
||||
mkdirSync(dirname(DB_PATH), { recursive: true });
|
||||
db = new Database(DB_PATH);
|
||||
db.pragma('journal_mode = WAL');
|
||||
|
||||
try {
|
||||
db.pragma('journal_mode = DELETE');
|
||||
} catch (error) {
|
||||
console.warn(`[resultStore] Database journal mode unchanged: ${error.message}`);
|
||||
}
|
||||
|
||||
return db;
|
||||
}
|
||||
|
||||
@@ -690,12 +753,11 @@ export function listProjects({
|
||||
} = {}) {
|
||||
initResultsStore();
|
||||
|
||||
const rows = queryBaseRows({});
|
||||
const projects = buildProjectList(rows, {
|
||||
const projects = getProjects({
|
||||
dedupeByName,
|
||||
city,
|
||||
section,
|
||||
projectNameKeyword: projectName,
|
||||
projectName,
|
||||
minAmount,
|
||||
maxAmount,
|
||||
startDate,
|
||||
@@ -713,10 +775,34 @@ export function listProjects({
|
||||
};
|
||||
}
|
||||
|
||||
export function getProjects({
|
||||
city,
|
||||
section,
|
||||
projectName,
|
||||
minAmount,
|
||||
maxAmount,
|
||||
startDate,
|
||||
endDate,
|
||||
dedupeByName = true,
|
||||
} = {}) {
|
||||
initResultsStore();
|
||||
|
||||
return buildProjectList(queryBaseRows({}), {
|
||||
dedupeByName,
|
||||
city,
|
||||
section,
|
||||
projectNameKeyword: projectName,
|
||||
minAmount,
|
||||
maxAmount,
|
||||
startDate,
|
||||
endDate,
|
||||
});
|
||||
}
|
||||
|
||||
export function getProjectFilters({ dedupeByName = true } = {}) {
|
||||
initResultsStore();
|
||||
|
||||
const projects = buildProjectList(queryBaseRows({}), { dedupeByName });
|
||||
const projects = getProjects({ dedupeByName });
|
||||
const cities = [...new Set(projects.map((project) => project.city).filter(Boolean))];
|
||||
const sections = [...new Set(projects.map((project) => project.section).filter(Boolean))];
|
||||
|
||||
|
||||
138
src/server.js
138
src/server.js
@@ -1,9 +1,6 @@
|
||||
import 'dotenv/config';
|
||||
import express from 'express';
|
||||
import cors from 'cors';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
initResultsStore,
|
||||
loadConfig,
|
||||
@@ -15,6 +12,7 @@ import {
|
||||
deleteTaskById,
|
||||
listResults,
|
||||
listProjects,
|
||||
getProjects,
|
||||
deleteResultById,
|
||||
clearResults,
|
||||
getResultFilters,
|
||||
@@ -24,12 +22,6 @@ import {
|
||||
import { initScheduler, runTaskNow, reloadScheduler, getSchedulerStatus } from './scheduler.js';
|
||||
import { runAgentTask } from './agentService.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
const DIST_DIR = join(__dirname, '..', 'dist');
|
||||
const DIST_INDEX = join(DIST_DIR, 'index.html');
|
||||
const HAS_DIST = existsSync(DIST_INDEX);
|
||||
|
||||
const app = express();
|
||||
const PORT = process.env.PORT || 5000;
|
||||
const DEFAULT_TASK_MODE = 'qwen3.5-plus';
|
||||
@@ -38,10 +30,7 @@ const MASKED_PASSWORD = '***已配置***';
|
||||
app.use(cors());
|
||||
app.use(express.json({ limit: '50mb' }));
|
||||
app.use(express.urlencoded({ limit: '50mb', extended: true }));
|
||||
|
||||
if (HAS_DIST) {
|
||||
app.use(express.static(DIST_DIR));
|
||||
}
|
||||
app.use(express.static('public'));
|
||||
|
||||
function normalizeTaskMode(value) {
|
||||
if (typeof value === 'string' && value.trim()) return value.trim();
|
||||
@@ -76,14 +65,12 @@ function buildTaskPayload(body = {}, { partial = false } = {}) {
|
||||
|
||||
function maskConfigSecrets(config) {
|
||||
const next = { ...config };
|
||||
|
||||
if (config.email) {
|
||||
next.email = {
|
||||
...config.email,
|
||||
smtpPass: config.email.smtpPass ? MASKED_PASSWORD : '',
|
||||
};
|
||||
}
|
||||
|
||||
return next;
|
||||
}
|
||||
|
||||
@@ -104,6 +91,58 @@ function mergeConfigWithExistingSecrets(incoming = {}) {
|
||||
return next;
|
||||
}
|
||||
|
||||
function getProjectQueryFilters(query = {}) {
|
||||
return {
|
||||
city: query.city,
|
||||
section: query.section,
|
||||
projectName: query.projectName,
|
||||
minAmount: query.minAmount,
|
||||
maxAmount: query.maxAmount,
|
||||
startDate: query.startDate,
|
||||
endDate: query.endDate,
|
||||
};
|
||||
}
|
||||
|
||||
function formatExportTimestamp(date = new Date()) {
|
||||
const pad = (value) => String(value).padStart(2, '0');
|
||||
return [
|
||||
date.getFullYear(),
|
||||
pad(date.getMonth() + 1),
|
||||
pad(date.getDate()),
|
||||
'-',
|
||||
pad(date.getHours()),
|
||||
pad(date.getMinutes()),
|
||||
pad(date.getSeconds()),
|
||||
].join('');
|
||||
}
|
||||
|
||||
function escapeCsvCell(value) {
|
||||
if (value === null || value === undefined) return '';
|
||||
const stringValue = String(value);
|
||||
if (/[",\r\n]/.test(stringValue)) {
|
||||
return `"${stringValue.replace(/"/g, '""')}"`;
|
||||
}
|
||||
return stringValue;
|
||||
}
|
||||
|
||||
function buildProjectsCsv(projects = []) {
|
||||
const columns = [
|
||||
['城市', (project) => project.city || ''],
|
||||
['板块', (project) => project.section || project.type || ''],
|
||||
['项目名称', (project) => project.projectName || ''],
|
||||
['金额(元)', (project) => Number.isFinite(project.amountYuan) ? project.amountYuan : ''],
|
||||
['发布日期', (project) => project.date || ''],
|
||||
['详情链接', (project) => project.detailLink || ''],
|
||||
];
|
||||
|
||||
const lines = [
|
||||
columns.map(([header]) => escapeCsvCell(header)).join(','),
|
||||
...projects.map((project) => columns.map(([, getter]) => escapeCsvCell(getter(project))).join(',')),
|
||||
];
|
||||
|
||||
return `\uFEFF${lines.join('\r\n')}`;
|
||||
}
|
||||
|
||||
let isRunning = false;
|
||||
let runningStatus = null;
|
||||
|
||||
@@ -147,7 +186,7 @@ function runTaskInBackground(task) {
|
||||
runningStatus = { ...runningStatus, finished: true, result: record, current: 1 };
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error('Task failed:', error.message);
|
||||
console.error('任务执行失败:', error.message);
|
||||
runningStatus = { ...runningStatus, finished: true, error: error.message, current: 1 };
|
||||
})
|
||||
.finally(() => {
|
||||
@@ -213,9 +252,8 @@ app.delete('/api/results/:id', (req, res) => {
|
||||
try {
|
||||
const deleted = deleteResultById(req.params.id);
|
||||
if (!deleted) {
|
||||
return res.status(404).json({ success: false, error: '未找到该结果' });
|
||||
return res.status(404).json({ success: false, error: '未找到结果' });
|
||||
}
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
@@ -231,9 +269,9 @@ app.delete('/api/results', (_req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
app.get('/api/results/filters', (req, res) => {
|
||||
app.get('/api/results/filters', (_req, res) => {
|
||||
try {
|
||||
const projectMode = req.query.view === 'projects';
|
||||
const projectMode = _req.query.view === 'projects';
|
||||
res.json({ success: true, data: getResultFilters({ projectMode }) });
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
@@ -242,37 +280,37 @@ app.get('/api/results/filters', (req, res) => {
|
||||
|
||||
app.get('/api/projects', (req, res) => {
|
||||
try {
|
||||
const {
|
||||
city,
|
||||
section,
|
||||
projectName,
|
||||
minAmount,
|
||||
maxAmount,
|
||||
startDate,
|
||||
endDate,
|
||||
page = 1,
|
||||
pageSize = 20,
|
||||
} = req.query;
|
||||
|
||||
const { page = 1, pageSize = 20 } = req.query;
|
||||
const filters = getProjectQueryFilters(req.query);
|
||||
const result = listProjects({
|
||||
city,
|
||||
section,
|
||||
projectName,
|
||||
minAmount,
|
||||
maxAmount,
|
||||
startDate,
|
||||
endDate,
|
||||
...filters,
|
||||
page,
|
||||
pageSize,
|
||||
dedupeByName: true,
|
||||
});
|
||||
|
||||
res.json({ success: true, ...result });
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
app.get('/api/projects/export', (req, res) => {
|
||||
try {
|
||||
const projects = getProjects({
|
||||
...getProjectQueryFilters(req.query),
|
||||
dedupeByName: true,
|
||||
});
|
||||
const csv = buildProjectsCsv(projects);
|
||||
const filename = `projects-${formatExportTimestamp()}.csv`;
|
||||
|
||||
res.setHeader('Content-Type', 'text/csv; charset=utf-8');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${filename}"; filename*=UTF-8''${encodeURIComponent(filename)}`);
|
||||
res.send(csv);
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
app.get('/api/projects/filters', (_req, res) => {
|
||||
try {
|
||||
res.json({ success: true, data: getProjectFilters({ dedupeByName: true }) });
|
||||
@@ -304,7 +342,6 @@ app.put('/api/tasks/:id', (req, res) => {
|
||||
if (!task) {
|
||||
return res.status(404).json({ success: false, error: '未找到该任务' });
|
||||
}
|
||||
|
||||
res.json({ success: true, data: task });
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
@@ -317,7 +354,6 @@ app.delete('/api/tasks/:id', (req, res) => {
|
||||
if (!deleted) {
|
||||
return res.status(404).json({ success: false, error: '未找到该任务' });
|
||||
}
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
@@ -416,15 +452,12 @@ app.post('/api/config', (req, res) => {
|
||||
app.post('/api/send-email', async (req, res) => {
|
||||
try {
|
||||
const { emailConfig, report } = req.body;
|
||||
|
||||
if (!emailConfig?.smtpHost || !emailConfig?.smtpUser || !emailConfig?.smtpPass) {
|
||||
return res.status(400).json({ success: false, error: '邮件配置不完整' });
|
||||
}
|
||||
|
||||
if (!emailConfig.recipients?.trim()) {
|
||||
return res.status(400).json({ success: false, error: '请指定收件人' });
|
||||
}
|
||||
|
||||
if (!report) {
|
||||
return res.status(400).json({ success: false, error: '没有报告数据' });
|
||||
}
|
||||
@@ -447,26 +480,13 @@ app.get('/api/scheduler/status', (_req, res) => {
|
||||
|
||||
app.post('/api/run-scheduled-task', (_req, res) => {
|
||||
try {
|
||||
runTaskNow().catch((error) => console.error('Scheduled task failed:', error));
|
||||
runTaskNow().catch((error) => console.error('定时任务执行失败:', error));
|
||||
res.json({ success: true, message: '定时任务已在后台触发' });
|
||||
} catch (error) {
|
||||
res.status(500).json({ success: false, error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
app.get('/results.html', (_req, res) => {
|
||||
res.redirect('/results');
|
||||
});
|
||||
|
||||
app.use((req, res, next) => {
|
||||
if (!HAS_DIST) return next();
|
||||
if (req.method !== 'GET') return next();
|
||||
if (req.path.startsWith('/api')) return next();
|
||||
if (req.path.includes('.')) return next();
|
||||
|
||||
res.sendFile(DIST_INDEX);
|
||||
});
|
||||
|
||||
initResultsStore();
|
||||
|
||||
app.listen(PORT, () => {
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
import { defineConfig } from 'vite';
|
||||
import vue from '@vitejs/plugin-vue';
|
||||
import { fileURLToPath, URL } from 'node:url';
|
||||
import AutoImport from 'unplugin-auto-import/vite';
|
||||
import Components from 'unplugin-vue-components/vite';
|
||||
import { ElementPlusResolver } from 'unplugin-vue-components/resolvers';
|
||||
|
||||
export default defineConfig({
|
||||
root: 'client',
|
||||
plugins: [
|
||||
vue(),
|
||||
AutoImport({
|
||||
imports: ['vue', 'vue-router'],
|
||||
resolvers: [ElementPlusResolver()],
|
||||
}),
|
||||
Components({
|
||||
resolvers: [ElementPlusResolver()],
|
||||
}),
|
||||
],
|
||||
resolve: {
|
||||
alias: {
|
||||
'@': fileURLToPath(new URL('./client/src', import.meta.url)),
|
||||
},
|
||||
},
|
||||
server: {
|
||||
host: '0.0.0.0',
|
||||
port: 5173,
|
||||
proxy: {
|
||||
'/api': {
|
||||
target: 'http://localhost:5000',
|
||||
changeOrigin: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
build: {
|
||||
outDir: '../dist',
|
||||
emptyOutDir: true,
|
||||
rollupOptions: {
|
||||
output: {
|
||||
manualChunks(id) {
|
||||
if (!id.includes('node_modules')) return;
|
||||
if (id.includes('element-plus')) return 'element-plus';
|
||||
if (id.includes('/vue/') || id.includes('@vue')) return 'vue-vendor';
|
||||
if (id.includes('axios')) return 'http-vendor';
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user