Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with Google Sign In with OpenID Sign In with Twitter

Tagged

Quantum basic setup
  • rszabo50rszabo50 +1 -1
    Posts: 24

    Hi all,

    I understand that quantum is new and that due to nic freeze up its important to get things right before you document it. but at the very least we should have a known working setup that clearly shows both a physical and logical network setup (including ip addressing to avoid confusion) and mimics the flat setup that nova-networking used to provide.

    There is so much conflicting documentation out there, and no one seems to have a fully functional setup and is willing to talk about it or post the full layout and changes for /etc/sysconfig/networking-scripts and quantum commands to invoke. Configuration items like CONFIG_QUANTUM_OVS_BRIDGE_MAPPINGS are still not ducmented on the rdo site which is very underwhelming.

    I was really disappointed last week to re install from the grizzly-2 rpm that nova-networking was removed in favor of quantum networking and it did not work out of the box any longer.

    So I guess, my question is, how long till the documentation catches up with the code?

  • britthouserbritthouser +1 -1 (+3 / -0 )
    Posts: 8

    I feel your pain! I can't answer the "how long" question, but I can share what it took for me to get an --allinone installation working. It was slightly more work than with nova, but not bad. I haven't yet gotten a multi-node setup working yet, so this is all I have to share right now.

    First do basic install as before: yum install -y http://rdo.fedorapeople.org/openstack/openstack-grizzly/rdo-release-grizzly-3.noarch.rpm yum update -y && reboot yum install -y openstack-packstack packstack --allinone

    From a different post I found I needed to edit l3-agent, dhcp-agent, and lb-agent in /etc/quantum and uncomment 'ovs_use_veth = True'. Then restart these three services.

    Next you'll need to decide which interface you want to be the "outside" of your cloud. This is the interface which connects to the network from which we will assign floating IPs. Add this interface to OVS br-ex bridge. In my case, this was interface bond0.844:

    ovs-vsctl add-port br-ex bond0.844

    From there you can do pretty much everything from the Horizon GUI. You'll need to create a virtual network with subnet that matches the real network the interface you put br-ex into is in. Second, add an internal network/subnet (this can be anything as it never leaves the box). Then create a virtual router whose gateway is set to your "outside" network, and also has a port on the inside network. Horizon makes this pretty painless, but I did it from the CLI and it looked like this:

    quantum net-create net844 --router:external=True

    quantum subnet-create --allocation-pool start=172.29.84.135,end=172.29.84.158 net844 172.29.84.128/27

    quantum net-create net10

    quantum subnet-create net10 10.10.10.0/24 --dns_nameservers list=true 8.8.8.8

    quantum router-create router1

    The hex at the end is the ID of the net10-subnet

    quantum router-interface-add router1 5e1abd39-9db6-4ecb-b2a2-eb2bd8cc7529

    The hex at the end is the ID of the net844 network

    quantum router-gateway-set router1 532a0efe-d7f7-447e-83de-f1bfd3527781

    From there, I think you can follow the "Running an Instance" directions.

  • binarysolobinarysolo +1 -1 (+1 / -0 )
    Posts: 15

    thanks a lot britthouser! this is what I was looking for....I will try this out today.

    In words to the documentation.... the quick start guide should at least be updated, since it is no longer relevant...

  • binarysolobinarysolo +1 -1 (+1 / -0 )
    Posts: 15

    @britthouser, is your setup a 2 NIC setup? In your case, is the "bond0.844" interface your primary NIC?

    I have a RHEL6.4 system with 2 NICs. 1 configured with a static IP, the other not configured/running yet.

    Appreciate the help,

    thanks.

  • vsthvsth +1 -1 (+3 / -0 )
    Posts: 2

    I have the same 2 nic setup. After adding "ovs-svctl add-port br-ex eth0", my eth0 is not reachable anymore. The normal ifcfg scripts for bridges are not working here. I found this solution:

    /etc/sysconfig/network-scripts/ifcfg-eth0::

    DEVICE=eth0 ONBOOT=yes HWADDR=AA:BB:CC:11:22:33 NM_CONTROLLED=no IPV6INIT=no TYPE="OVSPort" DEVICETYPE="ovs" OVS_BRIDGE=br-ex

    /etc/sysconfig/network-scripts/ifcfg-br-ex::

    DEVICE="br-ex" BOOTPROTO="none" BROADCAST="192.168.1.255" GATEWAY="192.168.1.1" IPADDR="192.168.1.2" NETMASK="255.255.255.0" NM_CONTROLLED="no" DEFROUTE="yes" IPV4_FAILURE_FATAL="yes" IPV6INIT=no ONBOOT="yes" TYPE="OVSBridge" DEVICETYPE="ovs"

    With this ifcfg scripts a can connect to br-ex from external.

    My next problem is now. If i create a new instance, this instance get a ipv6 address from the dhcp server, put i have a ip4v network. No idea why?

  • My setup for the --allinone had a single NIC, but with an extra VLAN being trunked down to the server. So my management IP address was configured on bond0 (the native vlan), and my external cloud interface was bond0.844. If you want to use the same interface for both, then I think what vsth has posted is the right way.

  • vsthvsth +1 -1
    Posts: 2

    My setup looks this way:

    • eth0 -> br-ex(192.168.1.0/24-gw192.168.1.1-ip192.168.1.2) reachable from external.
    • eth1 -> (192.168.2.0/24)
    • br-int

    In quantum i have created this networks and router:

    • net-1(192.168.1.0/24)
    • net-10(10.10.10.0/24)
    • router1(gw192.168.1.1) ->net-1
    • router1(10.10.10.1)->net-10

    One problem, from dhcp i get ipv6 adresse with the instance eth0 config:

    DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=on

    If i setup the network manuel like:

    ifconfig eth0 0 ifconfig eth0 10.10.10.3 netmask 255.255.255.0 route add default gw 10.10.10.1

    I am able to ping from instance in net-10:

    • gw 10.10.10.1 ok
    • other vm 10.10.10.4 ok
    • gw from net-1 192.168.1.1 ok (But 192.168.1.1 is not the same gw that i have on br-ex.)

    The internal quantum network are working. But i can not connect from outside or from the vm's to outside. Maybe the problem is that net-1 and the external Network on eth0 are the same subnet.

    Very interesting!

  • rszabo50rszabo50 +1 -1
    Posts: 24

    Thanks britthouser, I will check this out in the morning.

  • rszabo50rszabo50 +1 -1
    Posts: 24

    I tried this today. My network is not the same as yours.

    I have a 10.10.5 subnet that is my local LAN, and a 192.168.32 subnet that is an internal connect subnet

    I then set up my interfaces for persistence following reboot as:

    /etc/sysconfig/network-scripts/ifcfg-br-ex

    DEVICE="br-ex"
    BOOTPROTO="none"
    IPADDR="10.10.5.113"
    NETMASK="255.255.255.0"
    DNS1="10.10.9.6"
    DNS2="10.10.9.7"
    BROADCAST="10.10.5.255"
    GATEWAY="10.10.5.1"
    NM_CONTROLLED="no"
    DEFROUTE="yes"
    IPV4_FAILURE_FATAL="yes"
    IPV6INIT=no
    ONBOOT="yes"
    TYPE="OVSBridge"
    DEVICETYPE="ovs"
    

    /etc/sysconfig/network-scripts/ifcfg-br-int

    DEVICE="br-int"
    BOOTPROTO="none"
    IPADDR="192.168.32.113"
    NETMASK="255.255.255.0"
    BROADCAST="192.168.32.255"
    NM_CONTROLLED="no"
    IPV4_FAILURE_FATAL="yes"
    IPV6INIT=no
    ONBOOT="yes"
    TYPE="OVSBridge"
    DEVICETYPE="ovs"
    

    I then had to alter the original files for eth0 and eth2: /etc/sysconfig/network-scripts/ifcfg-eth0

    DEVICE="eth0"
    ONBOOT="yes"
    HWADDR="1C:C1:DE:76:19:70"
    TYPE="OVSPort"
    DEVICETYPE="ovs"
    OVS_BRIDGE=br-ex
    NM_CONTROLLED=no
    IPV6INIT=no
    

    /etc/sysconfig/network-scripts/ifcfg-eth2

    DEVICE="eth2"
    ONBOOT="yes"
    HWADDR="1C:C1:DE:76:19:71"
    IPV6INIT="no"
    TYPE="OVSPort"
    DEVICETYPE="ovs"
    OVS_BRIDGE=br-int
    NM_CONTROLLED=no
    IPV6INIT=no
    

    I also modified my quantum files for the ovs_use_veth flag you mentioned above and rebooted the boxes.

    So far so good.

    I then defined my quantum network topology as:

    # get tenant ids
    ADMIN_TENANT_ID=`keystone tenant-list | grep admin | awk '{ print $2}'`
    SERVICE_TENANT_ID=`keystone tenant-list | grep service | awk '{ print $2}'`
    # create internal subnet
    quantum net-create --name InternalNetwork --tenant-id $ADMIN_TENANT_ID InternalNetwork
    quantum subnet-create --name InternalSubnet --tenant-id $ADMIN_TENANT_ID  --allocation-pool start=192.168.32.193,end=192.168.32.254 InternalNetwork 192.168.32.0/24
    INT_SUBNET_ID=`quantum subnet-list | grep InternalSubnet | awk '{ print $2}'`
    # configure dns nameservers and gateway for dns pool
     quantum subnet-update $INT_SUBNET_ID --dns_nameservers list=true 10.10.9.6 10.10.9.7
     
    quantum subnet-update $INT_SUBNET_ID --gateway_ip 10.10.9.1
    # create external subnet
    quantum net-create --name ExternalNetwork --tenant-id $SERVICE_TENANT_ID ExternalNetwork --router:external=True
    quantum subnet-create --name ExternalSubnet --tenant-id $SERVICE_TENANT_ID --allocation-pool start=10.10.5.193,end=10.10.5.254 ExternalNetwork 10.10.5.0/24 --enable_dhcp=False
     
    # create a router
    quantum router-create --name router1 --tenant-id $SERVICE_TENANT_ID router1
    ROUTER_ID=`quantum router-list | grep router1 | awk '{ print $2}'`
     
    EXTERNAL_NETWORK_ID=`quantum net-list | grep ExternalNetwork | awk '{ print $2 }'`
    EXTERNAL_SUBNET_ID=`quantum subnet-list | grep ExternalSubnet | awk '{ print $2 }'`
    # configure the router
    quantum router-interface-add router1 $INTERNAL_SUBNET_ID
    quantum router-gateway-set router1 $EXTERNAL_NETWORK_ID
    quantum subnet-show $EXTERNAL_SUBNET_ID
    

    Now I have a clean topology that shows two networks with a router in the middle. Again so far so good.

    However ....

    I also have no connectivity with my guests, and the guests do not seem to be able to obtain meta-data with the 169.254.196.254 link local address.

    I am going to keep digging but if someone has some light to shed, that would be awesome.

    Cheers for the help so far...

  • tekkatekka +1 -1
    Posts: 19

    I suggest reading this: https://wiki.openstack.org/wiki/ConfigureOpenvswitch

    In particular this regarding br-int bridge that should not be bound to an physical interface: Agent Integration Bridge Configuration

    A well-known OVS integration bridge connects entities such Nova instance vNICs and the Quantum DHCP and L3 agents with virtual networks. The name of this bridge can be configured using the OVS.integration_bridge variable, but overriding the default value of "br-int" is not recommended as all entities need to agree on the bridge name.

    The integration bridge must be administratively created before first running the quantum agent: sudo ovs-vsctl add-br br-int

  • rszabo50rszabo50 +1 -1
    Posts: 24

    taking a look at the docs now.

    the br-int and br-ex interfaces automatically appear following kernel reboot, and the sysconfig entries handle the physical nic and bridge creation/startup on boot, which is a generally accepted approach to persistent network configuration.

    Still unsure about the rest of that document as it though

  • tekkatekka +1 -1
    Posts: 19

    The problem is your /etc/sysconfig/network-scripts/ifcfg-eth2 file

    DEVICE="eth2" ONBOOT="yes" HWADDR="1C:C1:DE:76:19:71" IPV6INIT="no" TYPE="OVSPort" DEVICETYPE="ovs" OVS_BRIDGE=br-int NM_CONTROLLED=no IPV6INIT=no

    eth2 should not be put in br-int bridge, unless you redefined the openstack default br-int" to another name and then used br-int name for this particular custom bridge. But it is not reccomended.

  • rszabo50rszabo50 +1 -1 (+1 / -0 )
    Posts: 24

    Cool, I removed the config for br-int and all is good, I can now access the node once I associate it with a floating IP.

    Here is my revised approach

    I have a 10.10.5 subnet that is my local LAN, and a 192.168.32 subnet that is an internal connect subnet

    I then set up my interfaces for persistence following reboot as:

    /etc/sysconfig/network-scripts/ifcfg-br-ex

    DEVICE="br-ex"
    BOOTPROTO="none"
    IPADDR="10.10.5.113"
    NETMASK="255.255.255.0"
    DNS1="10.10.9.6"
    DNS2="10.10.9.7"
    BROADCAST="10.10.5.255"
    GATEWAY="10.10.5.1"
    NM_CONTROLLED="no"
    DEFROUTE="yes"
    IPV4_FAILURE_FATAL="yes"
    IPV6INIT=no
    ONBOOT="yes"
    TYPE="OVSBridge"
    DEVICETYPE="ovs"
    

    I then had to alter the original files for eth0: /etc/sysconfig/network-scripts/ifcfg-eth0

    DEVICE="eth0"
    ONBOOT="yes"
    HWADDR="1C:C1:DE:76:19:70"
    TYPE="OVSPort"
    DEVICETYPE="ovs"
    OVS_BRIDGE=br-ex
    NM_CONTROLLED=no
    IPV6INIT=no
    

    Modify quantum configs:

    for i in /etc/quantum/*.ini
    do 
        sed -i "s/^[# ]*ovs_use_veth.*$/ovs_use_veth = True/g" $i
    done
    
    sed -i \
        -e "s/^[# ]*enable_isolated_metadata.*$/enable_isolated_metadata = True/g" \
        -e "s/^[# ]*enable_metadata_network.*$/enable_metadata_network = True/g"  \
        /etc/quantum/dhcp_agent.ini
    
    

    Once the modifications were complete and I configured the network interfaces I rebooted the box

    So far so good.

    I then defined my quantum network topology as:

    # get tenant ids
    ADMIN_TENANT_ID=`keystone tenant-list | grep admin | awk '{ print $2}'`
    SERVICE_TENANT_ID=`keystone tenant-list | grep service | awk '{ print $2}'`
    # create internal subnet
    quantum net-create --name InternalNetwork --tenant-id $ADMIN_TENANT_ID InternalNetwork
    quantum subnet-create --name InternalSubnet --tenant-id $ADMIN_TENANT_ID  --allocation-pool start=192.168.32.193,end=192.168.32.254 InternalNetwork 192.168.32.0/24
    INT_SUBNET_ID=`quantum subnet-list | grep InternalSubnet | awk '{ print $2}'`
    # configure dns nameservers and gateway for dns pool
     quantum subnet-update $INT_SUBNET_ID --dns_nameservers list=true 10.10.9.6 10.10.9.7
     
    quantum subnet-update $INT_SUBNET_ID --gateway_ip 10.10.9.1
    # create external subnet
    quantum net-create --name ExternalNetwork --tenant-id $SERVICE_TENANT_ID ExternalNetwork --router:external=True
    quantum subnet-create --name ExternalSubnet --tenant-id $SERVICE_TENANT_ID --allocation-pool start=10.10.5.193,end=10.10.5.254 ExternalNetwork 10.10.5.0/24 --enable_dhcp=False
     
    # create a router
    quantum router-create --name router1 --tenant-id $SERVICE_TENANT_ID router1
    ROUTER_ID=`quantum router-list | grep router1 | awk '{ print $2}'`
     
    EXTERNAL_NETWORK_ID=`quantum net-list | grep ExternalNetwork | awk '{ print $2 }'`
    EXTERNAL_SUBNET_ID=`quantum subnet-list | grep ExternalSubnet | awk '{ print $2 }'`
    # configure the router
    quantum router-interface-add router1 $INTERNAL_SUBNET_ID
    quantum router-gateway-set router1 $EXTERNAL_NETWORK_ID
    quantum subnet-show $EXTERNAL_SUBNET_ID
    

    I now have a working setup.... Awesome

  • binarysolobinarysolo +1 -1
    Posts: 15

    My setup is similar to @rszabo50, however, my setup is still not functional.

    Specifically, my router_gateway is not active.

    Name Fixed IPs Attached Device Status Admin State
    (277c2158) 10.37.40.1 network:router_gateway DOWN UP

    "/etc/sysconfig/network-scripts/ifcfg-br-ex"

    
       DEVICE="br-ex"
       BOOTPROTO="none"
       GATEWAY="10.37.0.1"
       IPADDR="10.37.10.232"
       DNS1="10.19.1.24"
       DNS2="10.19.1.23"
       NETMASK="255.255.0.0"
       NM_CONTROLLED="NO"
       DEFROUTE="yes"
       IPV4_FAILURE_FATAL="yes"
       ONBOOT="yes"
       IPV6INIT="no"
       TYPE="OVSBridge"
       DEVICETYPE="ovs"
    
    

    "/etc/sysconfig/network-scripts/ifcfg-em1"

    
    DEVICE="em1"
    HWADDR="5C:F9:DD:CA:72:84"
    NETMASK="255.255.0.0"
    NM_CONTROLLED="no"
    ONBOOT="yes"
    TYPE="OVSPort"
    DEVICETYPE="ovs"
    OVS_BRIDGE="br-ex"
    IPV6INIT="no"
    UUID="ec044dfa-bb7f-4f42-9d3a-578fd986dd25"
    

    My network topology:

    
    quantum net-create intnet
    
    quantum subnet-create intnet 10.10.10.0/24 --dns_nameservers list=true 10.19.1.24
    
    quantum router-create router1
    
    quantum net-create extnet --router:external=True
    
    quantum subnet-create --allocation-pool start=10.37.30.1,end=10.37.30.6 extnet 10.37.0.0/16 --enable_dhcp=False
    
    quantum router-gateway-set router1 [ID]
    
    quantum router-interface-add router1 [ID]
    

  • rszabo50rszabo50 +1 -1
    Posts: 24

    nothing is accessible until you associate a floating ip.

    and try: quantum subnet-update $INT_SUBNET_ID --dns_nameservers list=true 8.8.8.8

    quantum subnet-update $INT_SUBNET_ID --gateway_ip 10.10.10.1

  • segfaultsegfault +1 -1
    Posts: 1

    I have installed openstack-grizzly via "packstack --allinone" and everything installed fine. I am looking to configure networking as I have a public interface and a secondary 10G NIC connected as a trunk to the switch. I have the 8021q module loaded. Is there any documentation anywhere which can explain how I should get started with this?

    I was hoping to be able to use the 10G trunk connection as the external interface for all my instances.