MinIO 可用性测试

1. 运行 minio 服务

以纠删码模式运行，默认可以丢失一半磁盘

./minio server /opt/minio/data/{1...8}

服务运行后，可以看到创建了 8 个文件夹

[root@HQVM-1-141 data]# pwd
/opt/minio/data
[root@HQVM-1-141 data]# ls
1  2  3  4  5  6  7  8

2. 上传文件

登录控制台，创建 bucket，然后上传几个文件。

3. 破坏性测试

测试 1

删除文件夹 1

[root@HQVM-1-141 data]# rm -rf 1

此时服务端打印日志，发现了故障并恢复了，文件可以正常下载。

Found unformatted drive /opt/minio/data/1, attempting to heal...
Found drives to heal 1, proceeding to heal content...
Healing disk '/opt/minio/data/1' on 1st zone
Healing disk '/opt/minio/data/1' on 1st zone complete

并且文件夹和文件都已经恢复

[root@HQVM-1-141 data]# ls
1  2  3  4  5  6  7  8

测试 2

删除文件夹 1、2、3、4

[root@HQVM-1-141 data]# rm -rf 1 2 3 4

此时服务端打印日志，发现了故障并恢复了，文件可以正常下载，但是无法上传文件。

Found unformatted drive /opt/minio/data/4, attempting to heal...
Found unformatted drive /opt/minio/data/1, attempting to heal...
Found unformatted drive /opt/minio/data/2, attempting to heal...
Found unformatted drive /opt/minio/data/3, attempting to heal...
Found drives to heal 4, proceeding to heal content...
Healing disk '/opt/minio/data/4' on 1st zone
Healing disk '/opt/minio/data/4' on 1st zone complete
Healing disk '/opt/minio/data/1' on 1st zone
Healing disk '/opt/minio/data/1' on 1st zone complete
Healing disk '/opt/minio/data/2' on 1st zone
Healing disk '/opt/minio/data/2' on 1st zone complete
Healing disk '/opt/minio/data/3' on 1st zone
Healing disk '/opt/minio/data/3' on 1st zone complete

并且文件夹和文件都已经恢复

[root@HQVM-1-141 data]# ls
1  2  3  4  5  6  7  8

测试 3

删除文件夹 1、2、3、4、5

[root@HQVM-1-141 data]# rm -rf 1 2 3 4 5

此时服务端打印日志，发现了故障，然后抛出异常，文件无法下载。

Found unformatted drive /opt/minio/data/5, attempting to heal...
Found unformatted drive /opt/minio/data/1, attempting to heal...
Found unformatted drive /opt/minio/data/2, attempting to heal...
Found unformatted drive /opt/minio/data/3, attempting to heal...
Found unformatted drive /opt/minio/data/4, attempting to heal...
 
API: SYSTEM()
Time: 15:47:35 CST 11/02/2020
DeploymentID: aac210fa-ac5d-4579-a4bc-ef37e7743567
Error: Read failed. Insufficient number of disks online
       1: cmd/data-crawler.go:110:cmd.runDataCrawler()
 
API: SYSTEM()
Time: 15:47:35 CST 11/02/2020
DeploymentID: aac210fa-ac5d-4579-a4bc-ef37e7743567
Error: Read failed. Insufficient number of disks online
       3: cmd/erasure-zones.go:1687:cmd.(*erasureZones).HealFormat()
       2: cmd/background-heal-ops.go:123:cmd.healDiskFormat()
       1: cmd/background-heal-ops.go:93:cmd.(*healRoutine).run()
Found drives to heal 5, proceeding to heal content...
 
API: SYSTEM()
Time: 15:47:35 CST 11/02/2020
DeploymentID: aac210fa-ac5d-4579-a4bc-ef37e7743567
Error: Disk: /opt/minio/data/1 returned unformatted disk found
       endpoint=/opt/minio/data/1
       2: cmd/prepare-storage.go:52:cmd.glob..func5.1()
       1: cmd/background-newdisks-heal-ops.go:132:cmd.monitorLocalDisksAndHeal()

只是恢复了文件夹并没有恢复文件

[root@HQVM-1-141 data]# du -sh *
4.0K    1
4.0K    2
4.0K    3
4.0K    4
4.0K    5
24M     6
24M     7
24M     8

控制台已经崩溃，无法查看和上传下载文件。

总结

丢失的盘不能超过一半，否则服务无法工作，丢失一半或一半以内可以自动恢复文件，不影响服务运行。