一、更换磁盘
1、? 确认需要更换的磁盘对应的盘符、OSD ID；
root@Ceph-B**A**-04-S2-60:~# df -h |grep 17
/dev/sdp3?????? 1.1T? 566G? 551G? 51% /var/lib/ceph/osd/ceph-17
*sdp为服务器上对应的盘符；
*ceph-17 17为OSD对应的ID；
?
2、? 确认磁盘在服务器上对应的槽位；
root@Ceph-B**A**-04-S2-60:~# ll /dev/disk/by-path/ |grep sdp
lrwxrwxrwx 1 root root??? 9 Aug 17 19:17 pci-0000:02:00.0-scsi-0:0:18:0 -> ../../sdp
lrwxrwxrwx 1 root root?? 10 Aug 17 19:17 pci-0000:02:00.0-scsi-0:0:18:0-part1 -> ../../sdp1
lrwxrwxrwx 1 root root?? 10 Aug 17 19:17 pci-0000:02:00.0-scsi-0:0:18:0-part2 -> ../../sdp2
lrwxrwxrwx 1 root root?? 10 Aug 17 19:17 pci-0000:02:00.0-scsi-0:0:18:0-part3 -> ../../sdp3
*根据盘符找到服务器上对应的磁盘槽位，scsi-0:0:18:0标识第18号槽位；
?
3、? 确认主机的SN；
root@Ceph-B**A**-04-S2-60:~# dmidecode -s system-serial-number
HVNHB**
?
4、? 在Crushmap设置故障OSD OUT标志；
ceph osd out 17
*设置OUT标识，告诉集群不要再往这个OSD磁盘上读写数据；
?
5、? 查看集群状态；
ceph –s
*确认OSD设置OUT标志后，集群有没有出现异常；
?
6、? 关闭OSD对应的实例进程；
stop cep-osd id=17
?
7、? 从Crushmap中移除OSD；
ceph osd crush remove osd.17
?
8、? 删除集群中OSD认证信息；
ceph auth del osd.17
?*清除旧的认证key信息，新的磁盘加入集群后会重新生产认证密钥；
?
9、? 从集群中移除OSD；
ceph osd rm 17
?
10、?????卸载OSD磁盘挂载点；
umount /var/lib/ceph/osd/ceph-17
?
11、?????IDC维保服务商更换新的磁盘；
?
*根据确认的服务器SN和对应的磁盘槽位，将旧的磁盘取出，更换新的磁盘；
*登入IDRAC将磁盘调整为非RAID模式；
?
12、?????初始化新的磁盘；
ceph-deploy disk zap Ceph-BXXAXX-04-S2-60.domain.tld:sdp
?*更换新的磁盘后，有可能磁盘的盘符会变掉，初始化新的磁盘前，最好确认清楚（建议lsblk确认）；
?
13、?????OSD进程初始化；
ceph-deploy osd create Ceph-BXXAXX-04-S2-60.domain.tld:sdp
?*如果更换的是旧盘，这一步操作有可能失败报错，此时最好用dd（或者分区工具）摧毁磁盘上残留的Raid或分区表信息，然后重新初始化尝试；
?
14、?????调整Crushmap；
ceph osd crush add osd.17 1.17 root=sata pod=sata-pod-01 chassis=sata-chassis-01 host=Ceph-B**A**-04-S2-60-sata
*新的OSD权重和位置信息，可以使用ceph osd tree确认服务器上其他OSD的位置和权重做参考；
?
15、?????优化挂载参数；
mount -o remount,nobarrier /var/lib/ceph/osd/ceph-17
?*根据需要做优化调整，主要是优化filestore做后端存储引擎时，文件系统层面的提升；
*ceph –s关注集群在数据回填（backfill）中，集群读写IO；
?
16、?????完成。
?
?
二、 关机、重启
1、??查看确认Ceph集群状态；
ceph –s
*主要是看Ceph集群当前的状态，有没有pg在做scrub，有没有OSD down、out，数据有没有在做大量的backfill，remap等；
?
2、??设置OSD NOOUT标志；
ceph osd set nooout
?
3、??关闭本机上所有的OSD进程服务；
stop ceph-osd-all
*等待所有的OSD进程关闭，使用ps aux |grep ceph确认；
?
4、??关机或重启；
sync；sync；shutdown –h now
or
sync;sync;reboot
*如果更换、扩容内存，关机后可以操作；
*如果迁移服务器，关机之后可以迁移（在服务器开启前，网络要保证和迁移前相同）；
?
5、??开机或重启后，启动OSD进程服务；
start ceph-osd-all-starter
?
6、??取消OSD NOOUT标志；
ceph osd unset noout
?
7、??查看集群状态；
ceph –s
*确认集群恢复到正常状态；
?
8、??完成。
?
?
三、PG异常状态修复
常见状态说明
|
状态（state） |
说明(description) |
|
Active |
Ceph可正常处理此pg请求 |
|
Clean |
PG内所有的对象都被正确的复制了对应的份数 |
|
Down |
一个包含必备数据的副本离线，所以PG也离线了 |
|
Degraded |
PG中的一些对象还没有被复制到规定的份数 |
|
Inconsistent |
Ceph检测到PG中对象的一份或多份数据不一致 |
|
Peering |
PG正在互联过程中 |
|
Recovering |
Ceph正在迁移/同步对象和其副本 |
|
Incomplete |
Ceph 探测到某一PG可能丢失了写入信息，或者没有健康的副本 |
|
Stale |
PG状态未知，从PG mapping更新后Monitor一直没有收到更新 |
?
1、 查找Inconsistent pg并修复
????? ceph pg dump | grep insconsistent
????? or
????? ceph health detail
???? ceph pg repair {pgnum}
?
*如果修复不好，可以尝试ceph pg scrub {pgnum}或ceph pg deep-scrub {pgnum}
?
2、 查找stale pg 并修复
?????ceph pg dump |grep stale
?????
????? ceph pg map {pgname}
????? ceph pg force_create_pg {pgnum}
?
?
四、对象存储帐户管理
1、 创建帐户；
root@node1:~# radosgw-admin user create --display-name="Johnny wick" --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 0,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "S54GYGAB21NYEF765DSV",
??????????? "secret_key": "i6AswOcSF5xaeLm4sahRDpYpClpVTYe6FURWuDrU"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
??????? "enabled": false,
??????? "max_size_kb": -1,
??????? "max_objects": -1
??? },
??? "temp_url_keys": []
}
?
2、 停用帐户；
root@node1:~# radosgw-admin user suspend --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 1,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "S54GYGAB21NYEF765DSV",
??????????? "secret_key": "i6AswOcSF5xaeLm4sahRDpYpClpVTYe6FURWuDrU"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
??????? "enabled": false,
??????? "max_size_kb": -1,
??????? "max_objects": -1
??? },
??? "temp_url_keys": []
}
?
3、 启用帐户；
root@node1:~# radosgw-admin user enable --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 0,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "S54GYGAB21NYEF765DSV",
??????????? "secret_key": "i6AswOcSF5xaeLm4sahRDpYpClpVTYe6FURWuDrU"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
??????? "enabled": false,
??????? "max_size_kb": -1,
??????? "max_objects": -1
??? },
???"temp_url_keys": []
}
?
4、 删除帐户；
root@node1:~# radosgw-admin user rm --uid=Johnny
root@node1:~# radosgw-admin user info --uid=Johnny
could not fetch user info: no user info saved
?
5、 查看帐户；
root@node1:~# radosgw-admin user info --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 0,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "B6WXEJ94UG9V68VFD3UK",
??????????? "secret_key": "zaSwdDGSIWtWKYRp5dXQJymGXuzQrYa9pnaK4QxM"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
??????? "enabled": false,
??????? "max_size_kb": -1,
??????? "max_objects": -1
??? },
??? "temp_url_keys": []
}
6、 设置用户配额；
root@node1:~# radosgw-admin quota set --quota-scope=user --uid=Johnny --max-objects=2048 --max-size=204800
root@node1:~# radosgw-admin user info --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 0,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "B6WXEJ94UG9V68VFD3UK",
??????????? "secret_key": "zaSwdDGSIWtWKYRp5dXQJymGXuzQrYa9pnaK4QxM"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
???????"enabled": false,
???????"max_size_kb": 200,
???????"max_objects": 2048
???},
??? "temp_url_keys": []
}
?
root@node1:~# radosgw-admin quota enable --quota-scope=user --uid=Johnny
root@node1:~# radosgw-admin user info --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 0,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "B6WXEJ94UG9V68VFD3UK",
??????????? "secret_key": "zaSwdDGSIWtWKYRp5dXQJymGXuzQrYa9pnaK4QxM"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
??????? "enabled": true,
??????? "max_size_kb": 200,
??????? "max_objects": 2048
??? },
??? "temp_url_keys": []
}
?
7、 设置bucket配额；
root@node1:~# radosgw-admin quota set quota-scope=bucket --uid=Johnny --bucket=bucket-test01 --max-objects=512 --max-size=51200
root@node1:~# radosgw-admin quota set quota-scope=bucket --uid=Johnny --bucket=bucket-test02 --max-objects=1024 --max-size=102400
root@node1:~# radosgw-admin quota enable --quota-scope=bucket --uid=Johnny --bucket=bucket-test01
root@node1:~# radosgw-admin quota enable --quota-scope=bucket --uid=Johnny --bucket=bucket-test02
root@node1:~# radosgw-admin bucket stats --bucket=bucket-test02
{
??? "bucket": "bucket-test02",
??? "pool": ".rgw.buckets",
??? "index_pool": ".rgw.buckets.index",
??? "id": "default.124124.2",
??? "marker": "default.124124.2",
??? "owner": "Johnny",
??? "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1,11#1,12#1,13#1,14#1,15#1,16#1,17#1,18#1,19#1,20#1,21#1,22#1,23#1,24#1,25#1,26#1,27#1,28#1,29#1,30#1,31#1,32#1,33#1,34#1,35#1,36#1,37#1,38#1,39#1,40#1,41#1,42#1,43#1,44#1,45#1,46#1,47#1,48#1,49#1,50#1,51#1,52#1,53#1,54#1,55#1,56#1,57#1,58#1,59#1,60#1,61#1,62#1,63#1,64#1,65#1,66#1,67#1,68#1,69#1,70#1,71#1,72#1,73#1,74#1,75#1,76#1,77#1,78#1,79#1,80#1,81#1,82#1,83#1,84#1,85#1,86#1,87#1,88#1,89#1,90#1,91#1,92#1,93#1,94#1,95#1,96#1,97#1,98#1,99#1,100#1,101#1,102#1,103#1,104#1,105#1,106#1,107#1,108#1,109#1,110#1,111#1,112#1,113#1,114#1,115#1,116#1,117#1,118#1,119#1,120#1,121#1,122#1,123#1,124#1,125#1,126#1,127#1",
??? "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0,11#0,12#0,13#0,14#0,15#0,16#0,17#0,18#0,19#0,20#0,21#0,22#0,23#0,24#0,25#0,26#0,27#0,28#0,29#0,30#0,31#0,32#0,33#0,34#0,35#0,36#0,37#0,38#0,39#0,40#0,41#0,42#0,43#0,44#0,45#0,46#0,47#0,48#0,49#0,50#0,51#0,52#0,53#0,54#0,55#0,56#0,57#0,58#0,59#0,60#0,61#0,62#0,63#0,64#0,65#0,66#0,67#0,68#0,69#0,70#0,71#0,72#0,73#0,74#0,75#0,76#0,77#0,78#0,79#0,80#0,81#0,82#0,83#0,84#0,85#0,86#0,87#0,88#0,89#0,90#0,91#0,92#0,93#0,94#0,95#0,96#0,97#0,98#0,99#0,100#0,101#0,102#0,103#0,104#0,105#0,106#0,107#0,108#0,109#0,110#0,111#0,112#0,113#0,114#0,115#0,116#0,117#0,118#0,119#0,120#0,121#0,122#0,123#0,124#0,125#0,126#0,127#0",
??? "mtime": "2021-04-14 10:33:46.000000",
??? "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#,11#,12#,13#,14#,15#,16#,17#,18#,19#,20#,21#,22#,23#,24#,25#,26#,27#,28#,29#,30#,31#,32#,33#,34#,35#,36#,37#,38#,39#,40#,41#,42#,43#,44#,45#,46#,47#,48#,49#,50#,51#,52#,53#,54#,55#,56#,57#,58#,59#,60#,61#,62#,63#,64#,65#,66#,67#,68#,69#,70#,71#,72#,73#,74#,75#,76#,77#,78#,79#,80#,81#,82#,83#,84#,85#,86#,87#,88#,89#,90#,91#,92#,93#,94#,95#,96#,97#,98#,99#,100#,101#,102#,103#,104#,105#,106#,107#,108#,109#,110#,111#,112#,113#,114#,115#,116#,117#,118#,119#,120#,121#,122#,123#,124#,125#,126#,127#",
??? "usage": {},
??? "bucket_quota": {
???????"enabled": true,
???????"max_size_kb": 100,
???????"max_objects": 1024
???}
}
root@node1:~# radosgw-admin bucket stats --bucket=bucket-test01
{
??? "bucket": "bucket-test01",
??? "pool": ".rgw.buckets",
??? "index_pool": ".rgw.buckets.index",
??? "id": "default.124118.1",
??? "marker": "default.124118.1",
??? "owner": "Johnny",
??? "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1,11#1,12#1,13#1,14#1,15#1,16#1,17#1,18#1,19#1,20#1,21#1,22#1,23#1,24#1,25#1,26#1,27#1,28#1,29#1,30#1,31#1,32#1,33#1,34#1,35#1,36#1,37#1,38#1,39#1,40#1,41#1,42#1,43#1,44#1,45#1,46#1,47#1,48#1,49#1,50#1,51#1,52#1,53#1,54#1,55#1,56#1,57#1,58#1,59#1,60#1,61#1,62#1,63#1,64#1,65#1,66#1,67#1,68#1,69#1,70#1,71#1,72#1,73#1,74#1,75#1,76#1,77#1,78#1,79#1,80#1,81#1,82#1,83#1,84#1,85#1,86#1,87#1,88#1,89#1,90#1,91#1,92#1,93#1,94#1,95#1,96#1,97#1,98#1,99#1,100#1,101#1,102#1,103#1,104#1,105#1,106#1,107#1,108#1,109#1,110#1,111#1,112#1,113#1,114#1,115#1,116#1,117#1,118#1,119#1,120#1,121#1,122#1,123#1,124#1,125#1,126#1,127#1",
??? "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0,11#0,12#0,13#0,14#0,15#0,16#0,17#0,18#0,19#0,20#0,21#0,22#0,23#0,24#0,25#0,26#0,27#0,28#0,29#0,30#0,31#0,32#0,33#0,34#0,35#0,36#0,37#0,38#0,39#0,40#0,41#0,42#0,43#0,44#0,45#0,46#0,47#0,48#0,49#0,50#0,51#0,52#0,53#0,54#0,55#0,56#0,57#0,58#0,59#0,60#0,61#0,62#0,63#0,64#0,65#0,66#0,67#0,68#0,69#0,70#0,71#0,72#0,73#0,74#0,75#0,76#0,77#0,78#0,79#0,80#0,81#0,82#0,83#0,84#0,85#0,86#0,87#0,88#0,89#0,90#0,91#0,92#0,93#0,94#0,95#0,96#0,97#0,98#0,99#0,100#0,101#0,102#0,103#0,104#0,105#0,106#0,107#0,108#0,109#0,110#0,111#0,112#0,113#0,114#0,115#0,116#0,117#0,118#0,119#0,120#0,121#0,122#0,123#0,124#0,125#0,126#0,127#0",
??? "mtime": "2021-04-14 10:33:45.000000",
??? "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#,11#,12#,13#,14#,15#,16#,17#,18#,19#,20#,21#,22#,23#,24#,25#,26#,27#,28#,29#,30#,31#,32#,33#,34#,35#,36#,37#,38#,39#,40#,41#,42#,43#,44#,45#,46#,47#,48#,49#,50#,51#,52#,53#,54#,55#,56#,57#,58#,59#,60#,61#,62#,63#,64#,65#,66#,67#,68#,69#,70#,71#,72#,73#,74#,75#,76#,77#,78#,79#,80#,81#,82#,83#,84#,85#,86#,87#,88#,89#,90#,91#,92#,93#,94#,95#,96#,97#,98#,99#,100#,101#,102#,103#,104#,105#,106#,107#,108#,109#,110#,111#,112#,113#,114#,115#,116#,117#,118#,119#,120#,121#,122#,123#,124#,125#,126#,127#",
??? "usage": {},
??? "bucket_quota": {
???????"enabled": true,
???????"max_size_kb": 50,
???????"max_objects": 512
???}
}
8、 禁用配额
root@node1:~# radosgw-admin quota disable --quota-scope=user --uid=Johnny
root@node1:~# radosgw-admin user info --uid=Johnny
{
??? "user_id": "Johnny",
??? "display_name": "Johnny wick",
??? "email": "",
??? "suspended": 0,
??? "max_buckets": 1000,
??? "auid": 0,
??? "subusers": [],
??? "keys": [
??????? {
??????????? "user": "Johnny",
??????????? "access_key": "B6WXEJ94UG9V68VFD3UK",
??????????? "secret_key": "zaSwdDGSIWtWKYRp5dXQJymGXuzQrYa9pnaK4QxM"
??????? }
??? ],
??? "swift_keys": [],
??? "caps": [],
??? "op_mask": "read, write, delete",
??? "default_placement": "",
??? "placement_tags": [],
??? "bucket_quota": {
??????? "enabled": true,
??????? "max_size_kb": -1,
??????? "max_objects": 100
??? },
??? "user_quota": {
??????? "enabled": false,
??????? "max_size_kb": 200,
??????? "max_objects": 2048
??? },
??? "temp_url_keys": []
}
?
?
root@node1:~# radosgw-admin quota disable --quota-scope=bucket --uid=Johnny --bucket=bucket-test02
root@node1:~# radosgw-admin bucket stats --bucket=bucket-test02
{
??? "bucket": "bucket-test02",
??? "pool": ".rgw.buckets",
??? "index_pool": ".rgw.buckets.index",
??? "id": "default.124124.2",
??? "marker": "default.124124.2",
??? "owner": "Johnny",
??? "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1,11#1,12#1,13#1,14#1,15#1,16#1,17#1,18#1,19#1,20#1,21#1,22#1,23#1,24#1,25#1,26#1,27#1,28#1,29#1,30#1,31#1,32#1,33#1,34#1,35#1,36#1,37#1,38#1,39#1,40#1,41#1,42#1,43#1,44#1,45#1,46#1,47#1,48#1,49#1,50#1,51#1,52#1,53#1,54#1,55#1,56#1,57#1,58#1,59#1,60#1,61#1,62#1,63#1,64#1,65#1,66#1,67#1,68#1,69#1,70#1,71#1,72#1,73#1,74#1,75#1,76#1,77#1,78#1,79#1,80#1,81#1,82#1,83#1,84#1,85#1,86#1,87#1,88#1,89#1,90#1,91#1,92#1,93#1,94#1,95#1,96#1,97#1,98#1,99#1,100#1,101#1,102#1,103#1,104#1,105#1,106#1,107#1,108#1,109#1,110#1,111#1,112#1,113#1,114#1,115#1,116#1,117#1,118#1,119#1,120#1,121#1,122#1,123#1,124#1,125#1,126#1,127#1",
??? "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0,11#0,12#0,13#0,14#0,15#0,16#0,17#0,18#0,19#0,20#0,21#0,22#0,23#0,24#0,25#0,26#0,27#0,28#0,29#0,30#0,31#0,32#0,33#0,34#0,35#0,36#0,37#0,38#0,39#0,40#0,41#0,42#0,43#0,44#0,45#0,46#0,47#0,48#0,49#0,50#0,51#0,52#0,53#0,54#0,55#0,56#0,57#0,58#0,59#0,60#0,61#0,62#0,63#0,64#0,65#0,66#0,67#0,68#0,69#0,70#0,71#0,72#0,73#0,74#0,75#0,76#0,77#0,78#0,79#0,80#0,81#0,82#0,83#0,84#0,85#0,86#0,87#0,88#0,89#0,90#0,91#0,92#0,93#0,94#0,95#0,96#0,97#0,98#0,99#0,100#0,101#0,102#0,103#0,104#0,105#0,106#0,107#0,108#0,109#0,110#0,111#0,112#0,113#0,114#0,115#0,116#0,117#0,118#0,119#0,120#0,121#0,122#0,123#0,124#0,125#0,126#0,127#0",
??? "mtime": "2021-04-14 10:42:23.000000",
??? "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#,11#,12#,13#,14#,15#,16#,17#,18#,19#,20#,21#,22#,23#,24#,25#,26#,27#,28#,29#,30#,31#,32#,33#,34#,35#,36#,37#,38#,39#,40#,41#,42#,43#,44#,45#,46#,47#,48#,49#,50#,51#,52#,53#,54#,55#,56#,57#,58#,59#,60#,61#,62#,63#,64#,65#,66#,67#,68#,69#,70#,71#,72#,73#,74#,75#,76#,77#,78#,79#,80#,81#,82#,83#,84#,85#,86#,87#,88#,89#,90#,91#,92#,93#,94#,95#,96#,97#,98#,99#,100#,101#,102#,103#,104#,105#,106#,107#,108#,109#,110#,111#,112#,113#,114#,115#,116#,117#,118#,119#,120#,121#,122#,123#,124#,125#,126#,127#",
??? "usage": {},
??? "bucket_quota": {
??????? "enabled": false,
??????? "max_size_kb": 100,
??????? "max_objects": 1024
??? }
}
?
?
五、对象存储bucket管理
1、 创建bucket；
root@node1:~# s3cmd ls
root@node1:~# s3cmd mb s3://bkt-a01
Bucket ‘s3://bkt-a01/‘ created
root@node1:~# s3cmd ls
2021-04-14 03:01?s3://bkt-a01
?
2、 删除bucket；
root@node1:~# s3cmd ls
2021-04-14 03:01? s3://bkt-a01
root@node1:~# s3cmd rb s3://bkt-a01
Bucket ‘s3://bkt-a01/‘ removed
root@node1:~# s3cmd ls
?
3、 上传文件到bucket；
root@node1:~# s3cmd put ceph_0.94.10-1trusty_amd64.deb s3://bkt-a02/
upload: ‘ceph_0.94.10-1trusty_amd64.deb‘ -> ‘s3://bkt-a02/ceph_0.94.10-1trusty_amd64.deb‘?[1 of 1]
?10931302 of 10931302?? 100% in???0s??? 30.35 MB/s? done
root@node1:~# s3cmd ls s3://bkt-a02/
2021-04-14 03:14 10.4249019623M?d43b24ab71fb36c6626b2d243e5a03bd?s3://bkt-a02/ceph_0.94.10-1trusty_amd64.deb
?
4、 下载bucket里面文件；
root@node1:/tmp# s3cmd get s3://bkt-a02/ceph_0.94.10-1trusty_amd64.deb /tmp/
download: ‘s3://bkt-a02/ceph_0.94.10-1trusty_amd64.deb‘ -> ‘/tmp/ceph_0.94.10-1trusty_amd64.deb‘? [1 of 1]
?10931302 of 10931302?? 100% in??14s?? 712.42 kB/s? done
root@node1:/tmp# ls /tmp/
ceph_0.94.10-1trusty_amd64.deb
?
5、 删除bucket里面文件；
root@node1:/tmp# s3cmd rm s3://bkt-a02/ceph_0.94.10-1trusty_amd64.deb
delete: ‘s3://bkt-a02/ceph_0.94.10-1trusty_amd64.deb‘
root@node1:/tmp# s3cmd ls s3://bkt-a02/
?
6、 查看bucket里面文件；
root@node1:~/my-cluster# s3cmd info s3://bkt-a02/ceph-deploy_1.5.35_all.deb
s3://bkt-a02/ceph-deploy_1.5.35_all.deb (object):
?? File size: 96446
?? Last mod:?Wed, 14 Apr 2021 03:23:22 GMT
?? MIME type: application/x-debian-package
?? Storage:??STANDARD
?? MD5 sum:??05c4b861386a5b18ad008731ba7bf3d7
?? SSE:??????none
?? Policy:???<?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>bkt-a02</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>ceph-deploy_1.5.35_all.deb</Key><LastModified>2021-04-14T03:23:22.000Z</LastModified><ETag>"05c4b861386a5b18ad008731ba7bf3d7"</ETag><Size>96446</Size><StorageClass>STANDARD</StorageClass><Owner><ID>Johnny</ID><DisplayName>Johnny wick</DisplayName></Owner></Contents></ListBucketResult>
?? CORS:?????none
?? ACL:??????Johnny wick: FULL_CONTROL
?? x-amz-meta-s3cmd-attrs: atime:1560760630/ctime:1560498948/gid:0/gname:root/md5:05c4b861386a5b18ad008731ba7bf3d7/mode:33188/mtime:1560498933/uid:0/uname:root
?
7、拷贝bucket里面文件；
root@node1:~/my-cluster# s3cmd ls s3://bkt-a02
2021-04-14 03:23 94.185546875k?05c4b861386a5b18ad008731ba7bf3d7?s3://bkt-a02/ceph-deploy_1.5.35_all.deb
root@node1:~/my-cluster# s3cmd ls s3://bkt-a03
root@node1:~/my-cluster# s3cmd cp s3://bkt-a02/ceph-deploy_1.5.35_all.deb s3://bkt-a03
remote copy: ‘s3://bkt-a02/ceph-deploy_1.5.35_all.deb‘ -> ‘s3://bkt-a03/ceph-deploy_1.5.35_all.deb‘
root@node1:~/my-cluster# s3cmd ls s3://bkt-a03
2021-04-14 05:28 94.185546875k?05c4b861386a5b18ad008731ba7bf3d7?s3://bkt-a03/ceph-deploy_1.5.35_all.deb
?
8、移动bucket里面文件；
root@node1:~/my-cluster# s3cmd mb s3://bkt-a04
Bucket ‘s3://bkt-a04/‘ created
root@node1:~/my-cluster# s3cmd mv s3://bkt-a02/ceph-deploy_1.5.35_all.deb s3://bkt-a04/
move: ‘s3://bkt-a02/ceph-deploy_1.5.35_all.deb‘ -> ‘s3://bkt-a04/ceph-deploy_1.5.35_all.deb‘
root@node1:~/my-cluster# s3cmd ls s3://bkt-a02/
root@node1:~/my-cluster# s3cmd ls s3://bkt-a04
2021-04-14 05:34 94.185546875k?05c4b861386a5b18ad008731ba7bf3d7?s3://bkt-a04/ceph-deploy_1.5.35_all.deb
root@node1:~/my-cluster#
?
9、 查看bucket使用容量；
root@node1:~/my-cluster# s3cmd du s3://bkt-a04
94.185546875k 1 objects s3://bkt-a04/
?
10、???? 设置bucket文件权限
root@node1:~/my-cluster# s3cmd modify s3://bkt-a04/ --acl-public --recursive
modify: ‘s3://bkt-a04/ceph-deploy_1.5.35_all.deb‘
?