netgo

A few years ago, I have read about Quagga. It is software router which can install on the Linux such as Ubuntu. I have heard about this FRRouting which is next generation of the Quagga. In this post, I will configure some feature from FRRouting.

1. Installation

I will use Ubuntu 18.04. In this instruction, there are steps to install for Debian Linux. 

By the instruction, It will be work. 

After installation, I can check the the directory created in "/etc/frr/" like below.

2. Crash Log File.

This is the directory, "/var/tmp/frr/<daemon>[-<instance>].<pid>/crashlog" which I have refer.

3. Daemon Configuration.

After I start the FRR with "/etc/init.d/frr start" command, I can check the listen Port. I can see "staticd" and "zebra" daemon are loaded. This is the basic daemon.

When I want to more routing protocol, there are 2 things necessary, configuration file such as bgpd.conf and activate the protocol. 

The configuration sample files are located under the "/usr/share/doc/frr/examples/zebra.conf.sample". (Please not that those file has ownership by "frr". Thus I need to update with "chown frr.frr zebra.conf".)

When I handle the Quagga at first time, There are something difficult to use it. It is "Intergrated-vtysh-config". I need each configuration file to load each routing protocol. This means that I have to edit each file.

I think that FRR improve this part rather than before. Please read this. there are "frr.conf" file.

If this file "frr.conf" is existed, each configuration file will not be loaded. I have to remove "frr.conf" to load each file.

4. How to save configurartion file on VTY Shell mode.

In FRR, VTY Shell (Intergrated mode) is useful. However, there are something import. Basically, the routing protocol works on each daemon with each configuration file. It is not simple to load from common configuration file "frr.conf" to each daemon. This relationship make some consideration about the save file.

I configure "service intergrated-vtysh-config" in frr.conf and try to save with "write intergrated" insted "write file"

4. Enable BGP Feature of FRRouting

In "/etc/frr/daemon" filel there is the step to define the protocol what I want. In my case, I will active BGP Protocol It looks like below.

At this time, I only activate BGP Protocol.

5. Configure Internal BGP with Route Reflector.

In this post, I handle similar environment with bird opensource. I think it will be good comparison. This .

6. Result

After configuration, I can verify the result. It works well like below.

Host #2 and Host #3 are not connected directly. However, there can understand each other.

Reference 

[ 1 ] http://docs.frrouting.org/en/latest/ 

[ 2 ] https://deb.frrouting.org/

[ 3 ] https://blog.naver.com/happy_jhyo/70155768908

[ 4 ] http://docs.frrouting.org/en/latest/vtysh.html

[ 5 ] https://github.com/FRRouting/frr/issues/966

[ 6 ] https://createnetech.tistory.com/55

[ 7 ] https://createnetech.tistory.com/53?category=672584

In this post, I handled the External BGP routing. In this part, I will address Internal BGP routing which is working on the same AS number. This is the my environment

Top of the hosts will be the router as reflector. And the others are reflector-client. For this senario, I will set up the internal BGP routing between "router-reflector and router-reflector-client"

1. Configure Internal BGP routing.

In these configuration, there is something important. "Protocol direct" option is necessary value for this internal BGP routing protocol. Without this option, I can see the result like below. "Unreachable" is shown.

With this option, I can see the correct routing table with "birdc" command

2. Configure Router Reflector 

Now I will edit the TOP host to work as the router. It is simple. I insert the "rr client" option into the BGP configuration like below.

After then, I can see the routing table on Host #1 and Host #2

Now I can send ICMP each other.

Host #1	Host #2

Now I can use the Internal BGP and Router Reflector option.

Reference 

[ 1 ] https://createnetech.tistory.com/52?category=679927

[ 2 ] https://saidvandeklundert.net/2015-06-14-bird-bgp-route-reflection/ 

[ 3 ] www.bird.network.cz/?get_doc&v=20&f=bird-6.html#ss6.3

Recently, I need to study about Calico over Kubernetis. In fact, I like Flannel rather than this. However, this Calico is also best practice. During start this Calico, I read about the Brid. In this post, I will how to install and configure this Bird.

1. What is Bird?

In this site, it is explained "what kinds of dynamic routing is supported?". It is working on Linux also. In my environment, I will use Ubuntu.

2. Install the Bird

I will follow this instruction, It show the best way to install like below. I will use first one.

To add the repository for bird, I refer to this instruction,

After then, I can verify the version like below.

3. Configuration for Bird for the system

There are servaral configuration to work as the router or switch. First I will edit the system file "/etc/sysctl.conf". 

I will edit some configure for logging in "/etc/bird/bird.conf". In this file, it is look like below default. I will follow this instruction.

The bold above will be added. However, sometimes you will get the error message like below. Thus In my case I will update file properties.

Now I can start this bird.

4. Configure the simple BGP routing

This is my environment. I have 2 instance, which have IP address like below. I need to edit "/etc/bird/bird.conf" file

In the default configuration, there is something important variable. 

For the Simple BGP configuration, I will follow this sample instruction. I will try to configure I-BGP which mean each host has same as number. 

In my senario, I will announce the loopback IP address into the BGP routing protocol. "export" is look like "redestribute" option. In the above there is some important option "import" and "export". In this instruction, there is some explain like below.

I need to install into the BGP routing table which received. I need to edit "import all". Now I can verify the status with "birdc" command.

I can see the BGP routing table "1.1.1.1", "1.1.1.2", "2.2.2.1" and "2.2.2.2" are updated. Also I can see the status with "show protocols all" command

5. Installing over the Kernal Routing table

Even if bird has the routing table, I can not see the routing table over the system. 

Because of this, It is not easy to verify the status if this routing work or not. I need to edit "/etc/bird/bird.conf"

After then, the routing table will be like below.

I will use the AWS instance. To verify I need to change the option, "source and destination checking option". This option will drop the packet when the source and destination does not match.

And then, I can transfer ICMP packet like below.

Host #1	Host #2

This is the simple BGP routing with Bird. If I have chance to learn more deep-dive. I will handle about it next time. 

Reference 

[ 1 ] https://bird.network.cz/

[ 2 ] https://gitlab.labs.nic.cz/labs/bird

[ 3 ] https://launchpad.net/~cz.nic-labs/+archive/ubuntu/bird

[ 4 ] https://bird.network.cz/pipermail/bird-users/2011-September/002425.html

[ 5 ] https://blog.marquis.co/configuring-bgp-using-bird-on-ubuntu-14-04lts/

[ 6 ] https://skyenet.tech/bird/

How to outbound external connection of GN3 devices over remote server?

I use GNS3 simulator to study network feature sometimes. There is many instruction over internet. In this post, I will introduce how to use NAT fuction to connect to interet for outbound traffic. 

1. Pre-requisite.

I have local GNS3 client and remote server over Packet.net and AWS Baremetal cloud. In my case, my remote server is the Linux (Ubuntu 16.04) host. After then, I uploaded CiscoNX v9000 in my remote server.  

This is the my environment to re-produce in this post.

2. About NAT appliance (DHCP feature).

In GNS3, there is NAT appliance. It has the role to connect the physical interface on remote server virtually. This NAT appliance offers DHCP feature to assign IP address. This IP address determined the next hop against the Cisco switch/router. Thus, the Cisco switch/router should have DHCP feature. 

After this configuration, I can verify the interface status with "show ip int br" command

And "ip route 0.0.0.0/0 192.168.122.1" is written. In the Linux host (remote server), I can see the interfaces like below.

With this interfaces, the packet will transfer to external side. I can see the masqurade rule in "iptables table" like below.

Now, I can send traffic to outside on switch 1. Please noteh that it is not possible on switch 2.

3. Configuration NAT Inside and Outside

Switch 1 can send the packet to outside, however switch 2 can not. Because "192.168.122.0/24" network can only be masquraded in the remote server. This is the reason why the switch 1 has NAT feature. In this instruction, there is the explation for Cisco NAT concept. 

Inside source translation is for inside --> outside traffic. Outside source translation is for outside --> inside traffic. At first, I will make the direction on Cisco switch 1.

4. Create Rule (NAT Policy) for outbound traffic.

In this post, I handle only outbound traffic. The main factor is that IP address for all traffic to outside should be changed with Switch 1 interface ethernet 1/1 IP address.  In this post, It will be useful. I want all traffic to be sent.

5. Configure inside network (Switch 2).

So far, I made NAT firewall. From this part, it can assume internet network. However, I only use single switch/router simply. 

This is normal configuration. There is nothing special. After default configuration. I can verify the outbound connection like below over switch 2.

It works now.

Reference 

[ 1 ] https://docs.gns3.com/1c2Iyiczy6efnv-TS_4Hc7p11gn03-ytz9ukgwFfckDk/index.html

[ 2 ] https://docs.gns3.com/appliances/cisco-nxosv9k.html

[ 3 ] https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/interfaces/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_Interfaces_Configuration_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_Interfaces_Configuration_Guide_7x_chapter_01100.html#concept_F5BFDE98ABA749859F5AD710AC61EFCD

[ 4 ] https://onlyit.tistory.com/entry/How-to-use-a-nat-node-in-gns3

How to Configuration VXLAN in Ubuntu 16.04

In this post, I will configure VXLAN example. I will also utilize the linux-bridge to define L2 domain. Test environment is looks like below. 

The concepts are difficult to understand. However, the steps are not difficult.

1. Install the Linux Bridge and configuration.

In this step, I will create Linux Bridge and Interface on each hosts. The IP address in the same broadcasting is set on each interface.

apt-get install bridge-utils

brctl addbr vbr0

ip address add 192.168.0.1/24 dev vbr0

ifconfig vbr0 up

2. Configure VXLAN with Unicast

I will create VTEP interface with the command below. I can check the detail information with “-d” option.

3. Add VXLAN interface on Linux Bridge

However, it is not enough to communicate over tunnel. In this case, the traffic of “192.168.10.0/24” can not pass over the Linux Bridge. Thus, It is necessary for VXLAN interface to attach on the Linux Bridge.

4. Testing and analysis

I will do ping with one of “192.168.10.0/24” IP address. 

And I will dump packet during sending the packets. From the result, I can confirm “ICMP packets are encapsulated over VXLAN”

Reference Links

[ 1 ] https://serverfault.com/questions/777179/configuring-vxlan-unicast-in-linux

[ 2 ] https://www.tldp.org/HOWTO/BRIDGE-STP-HOWTO/set-up-the-bridge.html

[ 3 ] https://www.kernel.org/doc/Documentation/networking/vxlan.txt

[ 4 ] https://blog.scottlowe.org/2013/09/04/introducing-linux-network-namespaces/

[ 5 ] http://www.codeblogbt.com/archives/301596

How to Configure “ipvsadm” in Ubuntu 16.04

I recently the existence of this “ipvsadm” which used as the load-balancer.
“ipvsadm” is referred linux kernel load-balancer, which is also called “LVS, Linux Virtual Server”.
This LVS has 3 mode such as DR (Direct Routing), Tunnel and Masquerade. In this post, I will handle DSR and Masquerade (Network Address Translation, NAT).

Direct Routing : The option value is default with “-g”. The packet is send without modifying. The servers receives the packets from “ipvsadm” response to client directly.

Network Address Translation (NAT) : This option is adapted with “-m”. The packet is send with modifying the destination IP address. (The source IP address is not modified). The servers have to response the “ipvsadm”. Usually, The servers indicate “ipvsadm” as the default gateway.

My test environment is set on Ubuntu 16.04. I used AWS IaaS.

1. DR mode
 

1-1. ipvsadm configuration

Enable the IP forwarding, because the “ipvsadm” has the role to transfer and distribute received packets. To enable, edit “net.ipv4.ip_forward=1“ in “/etc/sysctl.conf” and run “sysctl -p /etc/sysctl.conf” or “sysctl –p” to apply this.
It can be done with, echo 1 > /proc/sys/net/ipv4/conf/all/forwarding, alternatively.

Configure virtual server, there are two steps. First, create the virtual server with traffic distribute method such as round-robin. Second, register servers to distribute the packets.
ipvsadm -C 
ipvsadm -A -t 10.10.0.244:80 -s rr
ipvsadm -a -t 10.10.0.244:80 -r 10.10.0.233:80 -g
After this configuration, I can confirm the status with “ipvsadm –Ln”, “ipvsadm –Lcn”, and “ipvsadm -l –-stats”

 
“ipvsadm –Ln” show the mapping information with forward method. In this case, the received packet with “10.10.0.244:80” will be routed to “10.10.0.233:80”.
 

“ipvsadm –Lcn” show the current session information. At this time, there is no con-current connection now.
 

“ipvsadm -l –-stats” show the information for in/out traffic information.

1-2. Servers configuration

In DR mode, the server received the packet without modifying. And the server response to the client directly. However, the packet drop can be happened in client side, because the client receive the packet from the server with server’s IP address. To resolve this issue, the server need to set the loopback interface with service IP address. In this case, the service IP address should be “10.10.0.244”.
ifconfig lo:0 10.10.0.244 netmask 255.255.255.255
 

LVS Direct Routing works by forwarding packets to the MAC address of servers. In this case, we have to consider “Linux ARP flux” problem. The server should not answer ARP request for “10.10.0.244”. For, this, I added in “/etc/sysctl.conf”.
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
 

1-3. Testing and analysis

Client send the request with “curl http://10.10.0.244” and get the response from server. I dumped the “ipvsadm” and “server”.
Loot at the “ipvsadm” result. I can see there is no change of source and destination IP address. 
 

This is something strange. “ipvsadm” has “10.10.0.244”, so it looks like sending to myself. This is the DR mode property, which works by MAC address of servers. Look at the connection information with “ipvsadm –Lcn”, the destination IP address can be shown.
 

At this time, what happened in the server, Look at the below. The packet was received with “10.10.0.244”. And response to this IP address. More important thing is response packet to client. The server send the packet, which has “10.10.0.244” as the source IP address. Because of this, the client does not dropt the packet.
 

2. Network Address Translation Mode

In NAT mode, the response should be return to the “ipvsadm”. However, the source IP address does not modified and sent to the server. NAT mode only modify the destination IP address.

2-1. ipvsadm configuration

Enable the IP forwarding, because the “ipvsadm” has the role to transfer and distribute received packets. To enable, edit “net.ipv4.ip_forward=1“ in “/etc/sysctl.conf” and run “sysctl -p /etc/sysctl.conf” or “sysctl –p” to apply this.
It can be done with, echo 1 > /proc/sys/net/ipv4/conf/all/forwarding, alternatively.
 
Configure virtual server, there are two steps. First, create the virtual server with traffic distribute method such as round-robin. Second, register servers to distribute the packets.
ipvsadm -C
ipvsadm -A -t 10.10.0.244:80 -s rr
ipvsadm -a -t 10.10.0.244:80 -r 10.10.0.233:80 –m
“ipvsadm –Ln” show the forward method is changed from “Route” to “Masq”
 

2-2. Server configuration

Server received the packet which is modified. Remember “ipvsadm” does not change the source IP address. In this case, the response will be return to client directly.
 
I use same network topology above. Therefore, “ipvsadm” and server are located on the same network. So, I can add some “static route” to transfer the response to “ipvsadm”.
route add -net 10.10.1.0 netmask 255.255.255.0 gw 10.10.0.244
 

2-3. Testing and analysis

Look at the “ipvsadm” packet flow from TCP dump. It show that the destination IP address is modified from 10.10.0.244 to 10.10.0.233. In response, the source IP address is also modified from 10.10.0.233 to 10.10.0.244.
 

Look at the server packet flow. The server do only normal processing. 

2-4. “ipvsadm” with SNAT (L3 mode, Proxy mode)

So far, “ipvsadm” and server are located on the same network. Therefore, It will be easy to construct LVS with NAT mode, using “static routing” method on server side. However, “ipvsadm” and servers can be located on different network.
For L3 environment, “ipvsadm” have modify the source IP address when the packet sent to server. I will add some rule in “iptables”.
Before, we add this rule, we need to add some configure in “/etc/sysctl.conf”. The iptables does not work without this options below.
net.ipv4.vs.conntrack = 1
net.ipv4.vs.snat_reroute = 1
 

After this, I add the rule into iptables with “-m”.
iptables -t nat -A POSTROUTING -o eth0 --dst 10.10.0.233 -m ipvs --ipvs --vaddr 10.10.0.244 --vport 80 --vmethod masq -j SNAT --to-source 10.10.0.244
 

Then, we can see the packet flow with TCP dump. The source IP is not client IP address, any more. The source IP address will be modified to send the server.

3. ipvsadmin with MARK of iptables

Occasionally, we need to use the MARK configuration of iptables. The PREROUTING will be used for this. Two steps are necessary. First, the received packet from client should be marked with iptables. Second, the marked packet should be distributed to servers.
To mark at the packet, I have to use mangle table. Mangle table is used for mark and QoS. In this case, I insert the rule like below
iptables  -A PREROUTING -t mangle -d 10.10.0.244/32 -j MARK --set-mark 1
 

And then, I edit the “ipvs” configuration.
ipvsadm -C 
ipvsadm -A -f 1  -s rr
ipvsadm -a -f 1 -r 10.10.0.233:0 –m
After then, I can see some change are happened. “FWM 1” mean MARK information in iptables.
 

Reference Links

[ 1 ] http://www.ultramonkey.org/papers/lvs_tutorial/html/
[ 2 ] http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.persistent_connection.html
[ 3 ] https://techiess.com/2010/09/09/load-balancing-dsr-direct-server-return/
[ 4 ] https://www.cyberciti.biz/faq/ubuntu-linux-add-static-routing/
[ 5 ] https://terrywang.net/2016/02/02/new-iptables-gotchas.html
[ 6 ] https://bugs.launchpad.net/ubuntu/+source/keepalived/+bug/1641918
[ 7 ] http://www.loadbalancer.org/blog/enabling-snat-in-lvs-xt_ipvs-and-iptables/
[ 8 ] http://manpages.ubuntu.com/manpages/trusty/man8/ipvsadm.8.html