部署ceph

ussuri版本已经不支持通过kolla部署ceph，需要使用外部ceph，参看
https://blog.51cto.com/penguintux/2536500
先部署一套ceph
参看
https://docs.openstack.org/kolla-ansible/ussuri/reference/storage/external-ceph-guide.html
配置外部ceph

准备ceph

• 创建pool

ceph osd pool create volumes
ceph osd pool create images
ceph osd pool create backups
ceph osd pool create vms

rbd pool init volumes
rbd pool init images
rbd pool init backups
rbd pool init vms

• 创建keyring

  ceph auth get-or-create client.glance mon 'profile rbd' osd 'profile rbd pool=images' mgr 'profile rbd pool=images' -o /etc/ceph/ceph.client.glance.keyring
  ceph auth get-or-create client.cinder mon 'profile rbd' osd 'profile rbd pool=volumes, profile rbd pool=vms, profile rbd-read-only pool=images' mgr 'profile rbd pool=volumes, profile rbd pool=vms' -o /etc/ceph/ceph.client.cinder.keyring
  ceph auth get-or-create client.cinder-backup mon 'profile rbd' osd 'profile rbd pool=backups' mgr 'profile rbd pool=backups' -o /etc/ceph/ceph.client.cinder-backup.keyring

kolla-ansible的组件版本

cat kolla-ansible/ansible/roles/prechecks/vars/main.yml
---
docker_version_min: '1.10.0'
docker_py_version_min: '2.0.0'
ansible_version_min: '2.8'
ansible_version_max: '2.9'

准备CentOS 8环境

• 准备virtualven环境

yum install python3
pip3 install docker

python3 -m venv /opt/ven_op
source /opt/ven_op/bin/activate

• 使用国内pip源

cat .pip/pip.conf
[global]
index-url = https://mirrors.aliyun.com/pypi/simple/

[install]
trusted-host=mirrors.aliyun.com

• 更新pip

  source /opt/ven_op/bin/activate
  pip install pip -U

• 安装Ansible
  Kolla Ansible 需要至少 Ansible 2.8 并支持2.9

  source /opt/ven_op/bin/activate
  pip install 'ansible<2.10'

• 安装docker

cat <<EOF>> /etc/yum.repos.d/docker-ce.repo   
[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/8/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
EOF

yum install docker-ce
systemctl start docker
systemctl enable docker

• 启用嵌套虚拟化

  cat <<EOF>>/etc/modprobe.d/kvm.conf
  options kvm_intel nested=1
  EOF

准备kolla-ansible

• 下载kolla-ansible

  git clone https://github.com/openstack/kolla-ansible

• 安装kolla-ansible

cd kolla-ansible
git checkout stable/ussuri
pip install .

mkdir -p /etc/kolla
chown $USER:$USER /etc/kolla

cp -r etc/kolla/* /etc/kolla/  # 实际只是复制了globals.yml、passwords.yml

配置ansible

cat <<EOF>>/etc/ansible/ansible.cfg
[defaults]
host_key_checking=False
pipelining=True
forks=100
EOF

配置kolla-ansible

• 生成密码

  kolla-genpwd

• 配置/etc/kolla/globals.yml

kolla_base_distro: "centos"
kolla_install_type: "source"
node_custom_config: "/etc/kolla/config"

# network_interface默认承载各api、vxlan、存储流量
network_interface: "eth1"

# 管理IP
# 需要与network_interface在同一网段
# 如果启用了haproxy，使用一个未被使用的IP
# 如果没启用haproxy，使用eth1的IP
kolla_internal_vip_address: "192.168.254.207"

# pulibc 网络，用于浮动IP
neutron_external_interface: "eth2"

neutron_plugin_agent: "openvswitch"

# 配置虚拟化类型
nova_compute_virt_type: "qemu"

# 要安装的组件
nable_openstack_core: "yes"

enable_haproxy: "no"
enable_cinder: "yes"

# 以下是配置使用外部ceph
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"
# gnocchi_backend_storage: "ceph"
# enable_manila_backend_cephfs_native: "yes"

ceph_glance_keyring: "ceph.client.glance.keyring"
ceph_glance_user: "glance"
ceph_glance_pool_name: "images"

ceph_cinder_keyring: "ceph.client.cinder.keyring"
ceph_cinder_user: "cinder"
ceph_cinder_pool_name: "volumes"
ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring"
ceph_cinder_backup_user: "cinder-backup"
ceph_cinder_backup_pool_name: "backups"

ceph_nova_keyring: "{{ ceph_cinder_keyring }}"
ceph_nova_user: "cinder"
ceph_nova_pool_name: "vms"

• 配置glance使用ceph

cat <<EOF>>/etc/kolla/config/glance/ceph.conf
[global]
fsid = 0e7fe350-fbbf-11ea-a78a-52540079f755
# mon_initial_members = ceph-0
mon_host = 192.168.254.200,192.168.254.201,192.168.254.202
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
EOF

cp ceph.client.glance.keyring /etc/kolla/config/glance/

• 配置cinder使用ceph

cp /etc/kolla/config/glance/ceph.conf  /etc/kolla/config/cinder

cp ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-volume/
cp ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-backup/
cp ceph.client.cinder-backup.keyring /etc/kolla/config/cinder/cinder-backup/

• 配置nova使用ceph

cp /etc/kolla/config/glance/ceph.conf /etc/kolla/config/nova
cp ceph.client.cinder.keyring /etc/kolla/config/nova/

• 配置nova
  umap可以启用trim
  writeback设置虚拟机的disk cache

  cat <<EOF>>/etc/kolla/config/nova/nova.conf 
  [libvirt]
  hw_disk_discard = unmap
  disk_cachemodes="network=writeback"
  cpu_mode=host-passthrough

  部署openstack

• 安装依赖

  kolla-ansible -i kolla-ansible/ansible/inventory/all-in-one bootstrap-servers

• 检查

  kolla-ansible -i kolla-ansible/ansible/inventory/all-in-one  prechecks

• 拉取镜像

  kolla-ansible -i kolla-ansible/ansible/inventory/all-in-one  pull

• 部署

  kolla-ansible -i kolla-ansible/ansible/inventory/all-in-one  deploy

• 完成部署

  kolla-ansible post-deploy

使用openstack

pip install python-openstackclient
. /etc/kolla/admin-openrc.sha