增加上传
This commit is contained in:
@ -10,3 +10,5 @@
|
|||||||
* [Ai服务接口功能](zh-cn/ai_function)
|
* [Ai服务接口功能](zh-cn/ai_function)
|
||||||
* [Ai服务接口列表](zh-cn/ai_service)
|
* [Ai服务接口列表](zh-cn/ai_service)
|
||||||
* **EasyApp接口服务**
|
* **EasyApp接口服务**
|
||||||
|
* **Excel服务模块**
|
||||||
|
* [导出/导入](zh-cn/excel_output_process)
|
30
zh-cn/excel_output_process.md
Normal file
30
zh-cn/excel_output_process.md
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
# 导出、导入任务
|
||||||
|
|
||||||
|
> 处理千万行数据必须分页、分块,避免内存溢出。
|
||||||
|
|
||||||
|
## 导出流程
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 1. 队列设计
|
||||||
|
系统整体导出分为以下流程:
|
||||||
|
|
||||||
|
- OpenSpout 流式写入(内存友好)
|
||||||
|
- 分页查询 + 生成器 Generator(避免全量加载)
|
||||||
|
- 进度推送(SSE)
|
||||||
|
- 协程上传 OSS
|
||||||
|
- GC 优化
|
||||||
|
- 清理临时文件
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[用户点击导出] --> B[生成任务, 发送 AMQP 消息]
|
||||||
|
B --> C[AMQP Consumer 消费消息]
|
||||||
|
C --> D[加锁, 启动协程]
|
||||||
|
D --> E[立即返回 ACK]
|
||||||
|
C --> F[RabbitMQ 确认完成]
|
||||||
|
D --> G[协程执行 runExport]
|
||||||
|
G --> H[分页流式写入 Excel]
|
||||||
|
H --> I[上传 OSS]
|
||||||
|
I --> J[记录任务, 推送完成]
|
||||||
|
```
|
Reference in New Issue
Block a user