Floating IP in OpenStack Neutron

共计 8653 个字符，预计需要花费 22 分钟才能阅读完成。

前言

Floating IP 是相对于 Fixed IP 而言的，它一般是在 VM 创建后分配给 VM 的，可以达到的目的就是，外界可以访问通过这个 Floating Ip 访问这个 VM，VM 也可以通过这个 IP 访问外界。

在 OpenStack 中，这个 Floating IP 使用了 namespace 内的 iptables 建立 NAT 转发机制来达到 VM 与外界的通讯的。这片文章主要讲述如何使用 OpenStack 搭建和使用 Floating IP.

Environment Setup

Ubuntu 14.04 LTS
2 个网卡，分别是 eth0（192.168.1.46）和 eth1（192.168.2.46，这个 interface 在 switch 使用 vlanid=100）
如果你的机器是 VM，请参照 http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004099，去开启 eth0 和 eth1 的 promisic 模式，这一步很关键，如果没有启用，外界是无法 ping 通分配给 VM 的 Floating IP 的

Devstack Configuration

启用 Nova, Neutron, Cinder, Keystone, Glance
安装 OpenvSwitch，然后做如下配置

# Add eth1 into br-eth1 and set eth0 and eth1 to promisc mode
sudo ovs-vsctl br-exists br-eth1 || sudo ovs-vsctl add-br br-eth1
sudo ovs-vsctl --may-exist add-port br-eth1 eth1
sudo ip link set dev eth1 promisc on
sudo ip link set dev eth0 promisc on
sudo ip addr flush eth1
sudo ip link set dev eth1 up

Devstack local.conf

#  Note: please change XX to your devstack node IP,  change VLAN_START and VLAN_END to your reserved vlan range Here
# 我的 IP 是 192.168.1.46，所在网络是 192.168.1.0/24
[[local|localrc]]
# Set API endpoint host using HOST_IP
HOST_IP=XX
# Use to specify the endpoint
SERVICE_HOST=XX

ADMIN_PASSWORD=welcome
MYSQL_PASSWORD=welcome
RABBIT_PASSWORD=welcome
SERVICE_PASSWORD=welcome
SERVICE_TOKEN=welcome
disable_service h-eng
disable_service h-api
disable_service h-api-cfn
disable_service h-api-cw
disable_service tempest
#enable_service tempest
disable_service dstat
# Enable Neturon
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron

# stack.sh will freshen each repo on each run if RECLONE
# is set to yes
RECLONE=True

# Setting OFFLINE=True to enable stack.sh to run multiple
# times without an Internet connection
OFFLINE=False

# Set FLAT_INTERFACE to the Ethernet interface that connects
# the host to your local network
FLAT_INTERFACE=eth1
#FLOATING_RANGE=192.168.1.0/24
#Q_FLOATING_ALLOCATION_POOL=start=192.168.1.100,end=192.168.1.119
#PUBLIC_NETWORK_GATEWAY=192.168.1.30
PUBLIC_INTERFACE=eth0
PHYSICAL_NETWORK=public_eth1
OVS_PHYSICAL_BRIDGE=br-ex
#PUBLIC_BRIDGE=br-ex

FIXED_RANGE=192.168.100.0/24
#FIXED_NETWORK_SIZE=32
NETWORK_GATEWAY=192.168.100.1


# IMAGE_URLS accepts a comma separated list of images to pre-load into OpenStack
IMAGE_URLS=http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
[[post-config|$NOVA_CONF]]
[DEFAULT]
rpc_response_timeout=300
service_down_time=300
libvirt_iscsi_use_multipath = True
#force_config_drive = False

[libvirt]
iscsi_use_multipath = True

[database]
max_pool_size=40
max_overflow=60

[[post-config|/$Q_PLUGIN_CONF_FILE]]
[ml2]
tenant_network_types = vlan,flat
[ml2_type_flat]
flat_networks = public_eth0
[ml2_type_vlan]
network_vlan_ranges = public_eth1:VLAN_START:VLAN_END

[ovs]
bridge_mappings = public_eth0:br-ex,public_eth1:br-eth1
enable_tunneling = False

Setup flat and vlan network

接下来，我要做两件事，一个是创建一个 VLAN ID 为 100 的 private network，以后的虚拟机默认是从这个网络分配 IP 的。
然后是一个 flat 的 public network，这个网络上是用来分配 floating IP

创建 private network，VLAN ID 为 100

stack@openstack-wangp11-01:~/devstack$ neutron net-create --provider:network_type vlan  --provider:physical_network public_eth1 --provider:segmentation_id 100 vlan_100
stack@openstack-wangp11-01:~/devstack$ neutron net-show vlan_100
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | 2426ff0d-953f-467f-a564-c4f63d926836 |
| mtu                       | 0                                    |
| name                      | vlan_100                             |
| port_security_enabled     | True                                 |
| provider:network_type     | vlan                                 |
| provider:physical_network | public_eth1                          |
| provider:segmentation_id  | 100                                  |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f |
| tenant_id                 | 8cb8c084ffb84914b41d5044ecbcad4e     |
+---------------------------+--------------------------------------+
stack@openstack-wangp11-01:~/devstack$ neutron subnet-create --enable-dhcp  --ip-version 4 --name vlan_100_subnet01 2426ff0d-953f-467f-a564-c4f63d926836  192.168.46.0/24
stack@openstack-wangp11-01:~/devstack$ neutron subnet-show fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
+-------------------+----------------------------------------------------+
| Field             | Value                                              |
+-------------------+----------------------------------------------------+
| allocation_pools  | {"start": "192.168.46.2", "end": "192.168.46.254"} |
| cidr              | 192.168.46.0/24                                    |
| dns_nameservers   |                                                    |
| enable_dhcp       | True                                               |
| gateway_ip        | 192.168.46.1                                       |
| host_routes       |                                                    |
| id                | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f               |
| ip_version        | 4                                                  |
| ipv6_address_mode |                                                    |
| ipv6_ra_mode      |                                                    |
| name              | vlan_100_subnet01                                  |
| network_id        | 2426ff0d-953f-467f-a564-c4f63d926836               |
| subnetpool_id     |                                                    |
| tenant_id         | 8cb8c084ffb84914b41d5044ecbcad4e                   |
+-------------------+----------------------------------------------------+

接下来，创建 Floating IP 所在的 public network, 网络类型是 flat

注意下面：192.168.1.100-192.168.1.119 是网络管理员分配给我的 IP pool，192.168.1.30 是我们网络的 router ip

# Step 1: create external flat network
$ neutron net-create --router:external  --provider:network_type flat  --provider:physical_network public_eth0  public_net
# Step 2: create subnet based on your reservation Here
$ neutron subnet-create  --name public_eth0_subnet01 --allocation-pool start=192.168.1.100,end=192.168.1.119 --gateway 192.168.1.30 --disable-dhcp --ip-version 4  22d01ce7-b4c4-4af2-bc3c-9c3991903b4c 192.168.1.0/24
# Step 3: create a external router 
$ neutron router-create  router1
# Step 4: connect router1 with the private subnet via 'neutron router-interface-add <router-id> <private_subnet_id>'
$ neutron router-interface-add 2da62299-6e41-4b24-a1c7-a3d6cc4db1c8 fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
# Step 5: Set the external public network  as the router gateway 'neutron router-gateway-set <router-id> <public_subnet_id>'
$ neutron router-gateway-set   73a38db4-20f9-49ba-b855-472be1e2fd45  e0368da6-197e-4c46-bfd1-4897d61b519a
# Step 6: Create floating IP from public network 'neutron floatingip-create  <public network id>'
$ neutron floatingip-create  1452712a-acb3-4341-a5ca-1d838eb2feb9
# Step 7: assicate the VM instance's port in private network to the newly created floating ip port
## get the port id of floating IP (in bold)
$ stack@openstack-wangp11-01:~$ neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id                                   | fixed_ip_address | floating_ip_address | port_id                              |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| a2316e0b-6d72-420f-8a23-f8421160d3d4 |                | 192.168.1.102       | 75353e16-5a16-452e-b420-0ada719d625c |
+--------------------------------------+------------------+---------------------+--------------------------------------+
## get the port id of private interface attached to VM instance
$ stack@openstack-wangp11-01:~$ neutron port-list
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                            |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 16a3e39c-48ef-4d4d-bc8a-96b27da7ff3c |      | fa:16:3e:c1:24:51 | {"subnet_id": "e0368da6-197e-4c46-bfd1-4897d61b519a", "ip_address": "192.168.1.100"} |
| 616d03cd-df6a-465f-a23a-b6bf46e55f7a |      | fa:16:3e:48:2e:7f | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.4"}  |
| 713e4541-7d22-4403-b828-94d714d94ec0 |      | fa:16:3e:af:43:7b | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.2"}  |
| 75353e16-5a16-452e-b420-0ada719d625c |      | fa:16:3e:23:11:db | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.3"} 
# 上面红色的 port 是 VM 在 private network 的 interface
$ stack@openstack-wangp11-01:~$ neutron floatingip-associate  a2316e0b-6d72-420f-8a23-f8421160d3d4 75353e16-5a16-452e-b420-0ada719d625c
# Now you can use '192.168.1.102' to login to the VM instance from 192.168.1.XX network

我在最前面提到，这个 Floating IP 是通过 namespace 内的 iptables 规则实现的，如何查看呢？下面是在 neutron node 上的一个实例，192.168.1.102 是 Floating IP，192.168.46.3 是 private IP

ip netns exec qrouter-<private-network-id> iptables -t nat -S

-A quantum-l3-agent-OUTPUT -d 192.168.1.102/32 -j DNAT --to-destination 192.168.46.3
-A quantum-l3-agent-PREROUTING -d 192.168.1.102/32 -j DNAT --to-destination 192.168.46.3 
-A quantum-l3-agent-float-snat -s 192.168.46.3/32 -j SNAT --to-source 192.168.1.102

启用 nameserver

现在，VM 跟外界可以通过 192.168.1.102 进行通信了，在外界看来，VM 的 Ip 就是 192.168.1.102 了，但是有一个问题，无法访问网站，也无法 sudo apt-get update

必须要启用 nameserver 才可以访问外面的网址

# add following to /etc/resolv.conf of the VM instance to enable name service
 
nameserver <your name sever ip>
nameserver <your name server ip 2>

#and run following to enable
sudo resolvconf -u