| feat: optimize event log storage and retention (#2459)
1. Add direct stream reading from storage
- Add ReadFile interface to storage layer
- Implement ReadFile in S3Storage and LocalStorage
- Enable direct streaming from MinIO without local download
2. Refactor event log retrieval
- Modify GetMessages to read directly from storage
- Remove downloadWithRetry method (no longer needed)
- Add fallback to local file for backward compatibility
- Reduce disk usage and improve performance
3. Configure event log permanent retention
- Remove event log auto-cleanup rule from MinIO lifecycle
- Keep logs/eventlog/ files permanently
- Other cleanup policies remain unchanged
Changes:
- api/eventlog/db/eventFilePlugin.go: refactor to use direct streaming
- pkg/component/storage/storage.go: add ReadFile interface
- pkg/component/storage/s3_storage.go: implement ReadFile and remove log cleanup rule
- pkg/component/storage/local_storage.go: implement ReadFile
Co-authored-by: 张启航 <12344192+zhangsetsail@user.noreply.gitee.com> | 5 个月前 |