Recently, I have some chance to study about the FRRouting. At that time, I do not have any Linux server. Thus, I decide that I use AWS Spot instance. After installation, I set up the OSPFv2 environment with simple configuration. However I can not estabilish the OSPF neighbor. 

 

1. Simple OSPFv2 Configuration of FRRouting

I have 2 hosts which is direct connected each other. In fact, these hosts are located in the same subnet of the VPC.

Host #1 Host #2
ip-10-11-0-200# show running-config 
Building configuration...

Current configuration:
!
frr version 7.3
frr defaults traditional
hostname ip-10-11-0-200
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
interface ens5
 ip address 10.11.0.200/24
 ip ospf hello-interval 1
!
interface lo
 ip address 1.1.1.1/32
!
router ospf
 ospf router-id 1.1.1.1
 network 10.11.0.0/24 area 0.0.0.0
!
line vty
!
end
ip-10-11-0-229# show running-config 
Building configuration...

Current configuration:
!
frr version 7.3
frr defaults traditional
hostname ip-10-11-0-229
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
interface ens5
 ip address 10.11.0.229/24
 ip ospf hello-interval 1
!
interface lo
 ip address 1.1.1.2/32
!
router ospf
 ospf router-id 1.1.1.2
 network 10.11.0.0/24 area 0.0.0.0
!
line vty
!
end

In FRRouting, I need one of configurations, "network <ip address> area <area-id>" and "ip ospf area <area-id>". In my case, I used "network <ip address> area <area-id>".

 

2. Multicast for OSPFv2

Have you ever heard about the multicast packet for OSPFv2? It is necessary factor to establish the connection.

[ Correct Multicast Relationship]
15:35:33.303082 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
15:35:34.275218 IP 10.11.0.229 > 224.0.0.5: OSPFv2, Hello, length 48

On the same broadcast domain, I can see the both packets from the sender. However, I can not see all of things over AWS VPC network. It look like below

15:40:05.381662 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
15:40:06.381928 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44

AWS does not support multicast default.

 

2. Enable the multicast feature for AWS VPC

Recently, AWS improve their feature of VPC with transit gateway. In this instruction, AWS show how to enable the multicast, even if there is limitation.

It is only 1 multicast source is possible. Because of this, I can not make success to establish OSPFv2 default. I need to request increase the quota. 

 

2-1. Create transit gateway with Multicast

The below is the result of creation. There is "Multicast support" option. (Please note that this feature is not opened on all of the regions, In my case, I use Virginia region.)

2-2. Attach the VPC to Transit Gateway.

I need to attach the transit gateway with mulitcast domain to the VPC.

2-3. Associate the subnet in VPC to the transit Gateway.

I have to assign the subnet which make multicast work to multicast domain. This multicast domain is created by transit gateway.

2-4. Register the source and member for multicast.

The definition of the source and memeber is below. 

For the OSPFv2, each host should be source and member. Thus I need 2 source and 2 member. However, I can not make 2 source at this time by limitation of AWS

"224.0.0.5" is the Multicast Group Address for OSPFv2. 

 

3. The result after enabling Multicast

Even if the multicast does not activate fully. I can verify the multicast effect. Host #1 is source and member. Host #2 is only member. Thus Host #2 can not transfer the mulitcast over VPC network. 

 

[Host #1 Packets]
16:03:01.614874 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
16:03:02.615055 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
16:03:03.615173 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
16:03:04.615295 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
16:03:05.615409 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
16:03:06.615963 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
16:03:07.615972 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44

root@ip-10-11-0-200:~# vtysh 

Hello, this is FRRouting (version 7.3).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

ip-10-11-0-200# show ip ospf neighbor 

Neighbor ID     Pri State           Dead Time Address         Interface                        RXmtL RqstL DBsmL
[Host #2 Packets]
15:59:34.565227 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
15:59:34.566696 IP 10.11.0.229 > 224.0.0.5: OSPFv2, Hello, length 48
15:59:35.565246 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
15:59:35.566705 IP 10.11.0.229 > 224.0.0.5: OSPFv2, Hello, length 48
15:59:36.565372 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44
15:59:36.566724 IP 10.11.0.229 > 224.0.0.5: OSPFv2, Hello, length 48
15:59:37.565376 IP 10.11.0.200 > 224.0.0.5: OSPFv2, Hello, length 44

root@ip-10-11-0-229:~# vtysh 

Hello, this is FRRouting (version 7.3).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

ip-10-11-0-229# show ip ospf neighbor 

Neighbor ID     Pri State           Dead Time Address         Interface                        RXmtL RqstL DBsmL
1.1.1.1           1 Init/DROther      39.950s 10.11.0.200     ens5:10.11.0.229                     0     0     0

This is the what I learned from the Test.

Reference

[ 1 ] https://docs.aws.amazon.com/vpc/latest/tgw/working-with-multicast.html

[ 2 ] https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html

[ 3 ] https://docs.aws.amazon.com/vpc/latest/tgw/tgw-multicast-overview.html

A few years ago, I have read about Quagga. It is software router which can install on the Linux such as Ubuntu. I have heard about this FRRouting which is next generation of the Quagga. In this post, I will configure some feature from FRRouting.

 

1. Installation

I will use Ubuntu 18.04. In this instruction, there are steps to install for Debian Linux. 

By the instruction, It will be work. 

After installation, I can check the the directory created in "/etc/frr/" like below.

 

2. Crash Log File.

This is the directory, "/var/tmp/frr/<daemon>[-<instance>].<pid>/crashlog" which I have refer.

 

3. Daemon Configuration.

After I start the FRR with "/etc/init.d/frr start" command, I can check the listen Port. I can see "staticd" and "zebra" daemon are loaded. This is the basic daemon.

When I want to more routing protocol, there are 2 things necessary, configuration file such as bgpd.conf and activate the protocol. 

The configuration sample files are located under the "/usr/share/doc/frr/examples/zebra.conf.sample". (Please not that those file has ownership by "frr". Thus I need to update with "chown frr.frr zebra.conf".)

When I handle the Quagga at first time, There are something difficult to use it. It is "Intergrated-vtysh-config". I need each configuration file to load each routing protocol. This means that I have to edit each file.

I think that FRR improve this part rather than before. Please read this. there are "frr.conf" file.

If this file "frr.conf" is existed, each configuration file will not be loaded. I have to remove "frr.conf" to load each file.

4. How to save configurartion file on VTY Shell mode.

In FRR, VTY Shell (Intergrated mode) is useful. However, there are something import. Basically, the routing protocol works on each daemon with each configuration file. It is not simple to load from common configuration file "frr.conf" to each daemon. This relationship make some consideration about the save file.

I configure "service intergrated-vtysh-config" in frr.conf and try to save with "write intergrated" insted "write file"

4. Enable BGP Feature of FRRouting

In "/etc/frr/daemon" filel there is the step to define the protocol what I want. In my case, I will active BGP Protocol It looks like below.

root@ip-10-10-0-134:~# cat /etc/frr/daemons   
bgpd=yes
ospfd=no
ospf6d=no 

At this time, I only activate BGP Protocol.

 

5. Configure Internal BGP with Route Reflector.

In this post, I handle similar environment with bird opensource. I think it will be good comparison. This .

 

Host #1 (Top) Host #2 (Left side of Bottom) Host #3 (Right side of Bottom)
ip-10-10-0-64# show running-config 
Building configuration...

Current configuration:
!
frr version 7.3
frr defaults traditional
hostname ip-10-10-0-64
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
router bgp 65001
 bgp router-id 10.10.0.64
 neighbor 10.10.0.107 remote-as 65001
 neighbor 10.10.0.134 remote-as 65001
 !
 address-family ipv4 unicast
  neighbor 10.10.0.107 route-reflector-client
  neighbor 10.10.0.134 route-reflector-client
 exit-address-family
!
line vty
!
end
ip-10-10-0-107# show running-config 
Building configuration...

Current configuration:
!
frr version 7.3
frr defaults traditional
hostname ip-10-10-0-107
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
interface lo
 ip address 1.1.1.1/32
 ip address 1.1.1.2/32
!
router bgp 65001
 bgp router-id 10.10.0.107
 neighbor 10.10.0.64 remote-as 65001
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
!
line vty
!
end
ip-10-10-0-134# show running-config 
Building configuration...

Current configuration:
!
frr version 7.3
frr defaults traditional
hostname ip-10-10-0-134
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
interface lo
 ip address 2.2.2.1/32
 ip address 2.2.2.2/32
!
router bgp 65001
 bgp router-id 10.10.0.134
 neighbor 10.10.0.64 remote-as 65001
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
!
line vty
!
end

 

6. Result

After configuration, I can verify the result. It works well like below.

Host #1 (Top)
ip-10-10-0-64# show ip bgp summary 

IPv4 Unicast Summary:
BGP router identifier 10.10.0.64, local AS number 65001 vrf-id 0
BGP table version 6
RIB entries 9, using 1656 bytes of memory
Peers 2, using 41 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.10.0.107     4      65001      10      12        0    0    0 00:04:38            3
10.10.0.134     4      65001       6       7        0    0    0 00:01:01            3

Host #2 (Left side of Bottom)
ip-10-10-0-107# show ip bgp summary 

IPv4 Unicast Summary:
BGP router identifier 10.10.0.107, local AS number 65001 vrf-id 0
BGP table version 5
RIB entries 9, using 1656 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.10.0.64      4      65001      12      10        0    0    0 00:04:34            2

Host #3 (Right side of Bottom)
ip-10-10-0-134# show ip bgp summary 

IPv4 Unicast Summary:
BGP router identifier 10.10.0.134, local AS number 65001 vrf-id 0
BGP table version 6
RIB entries 9, using 1656 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.10.0.64      4      65001       6       5        0    0    0 00:00:53            3

Host #2 and Host #3 are not connected directly. However, there can understand each other.

Host #2 (Left side of Bottom)
ip-10-10-0-107# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

K>* 0.0.0.0/0 [0/100] via 10.10.0.1, ens5, src 10.10.0.107, 00:16:04
C>* 1.1.1.1/32 is directly connected, lo, 00:07:08
C>* 1.1.1.2/32 is directly connected, lo, 00:07:04
B>* 2.2.2.1/32 [200/0] via 10.10.0.134, ens5, 00:04:47
B>* 2.2.2.2/32 [200/0] via 10.10.0.134, ens5, 00:04:47
C>* 10.10.0.0/24 is directly connected, ens5, 00:16:04
K>* 10.10.0.1/32 [0/100] is directly connected, ens5, 00:16:04

Host #3 (Right side of Bottom)
ip-10-10-0-134# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

K>* 0.0.0.0/0 [0/100] via 10.10.0.1, ens5, src 10.10.0.134, 00:15:39
B>* 1.1.1.1/32 [200/0] via 10.10.0.107, ens5, 00:04:32
B>* 1.1.1.2/32 [200/0] via 10.10.0.107, ens5, 00:04:32
C>* 2.2.2.1/32 is directly connected, lo, 00:05:16
C>* 2.2.2.2/32 is directly connected, lo, 00:05:13
C>* 10.10.0.0/24 is directly connected, ens5, 00:15:39
K>* 10.10.0.1/32 [0/100] is directly connected, ens5, 00:15:39

 

Reference 

[ 1 ] http://docs.frrouting.org/en/latest/ 

[ 2 ] https://deb.frrouting.org/

[ 3 ] https://blog.naver.com/happy_jhyo/70155768908

[ 4 ] http://docs.frrouting.org/en/latest/vtysh.html

[ 5 ] https://github.com/FRRouting/frr/issues/966

[ 6 ] https://createnetech.tistory.com/55

[ 7 ] https://createnetech.tistory.com/53?category=672584

 

 

In this post, I handled the External BGP routing. In this part, I will address Internal BGP routing which is working on the same AS number. This is the my environment

Top of the hosts will be the router as reflector. And the others are reflector-client. For this senario, I will set up the internal BGP routing between "router-reflector and router-reflector-client"

 

1. Configure Internal BGP routing.

Host #1 (Top)

Host #2 (Left bottom)

Host #3 (Right bottom)

protocol direct {

#        interface "*";

        interface "ens5";

}

protocol bgp tohost1{

        import all;

        export all;

        local as 65001;

        neighbor 10.10.0.159 as 65001;

}

protocol bgp tohost2{

        import all;

        export all;

        local as 65001;

        neighbor 10.10.0.205 as 65001;

}

protocol direct {

#        interface "*";

        interface "ens5";

}

protocol static static_bgp {

        route 1.1.1.1:255.255.255.255 via 10.10.0.159;

        route 1.1.1.2:255.255.255.255 via 10.10.0.159;

}

protocol bgp {

        import all;

        export where proto = "static_bgp";

        local as 65001;

        neighbor 10.10.0.49 as 65001;

}

protocol direct {

#        interface "*";

        interface "ens5";

}


protocol static static_bgp {

        route 2.2.2.1:255.255.255.255 via 10.10.0.205;

        route 2.2.2.2:255.255.255.255 via 10.10.0.205;

}

protocol bgp {

        import all;

        export where proto = "static_bgp";

        local as 65001;

        neighbor 10.10.0.49 as 65001;

}

In these configuration, there is something important. "Protocol direct" option is necessary value for this internal BGP routing protocol. Without this option, I can see the result like below. "Unreachable" is shown.

bird> show route

1.1.1.1/32         unreachable [bgp1 13:45:49 from 10.10.0.159] * (100/-) [i]

1.1.1.2/32         unreachable [bgp1 13:45:49 from 10.10.0.159] * (100/-) [i]

With this option, I can see the correct routing table with "birdc" command

bird> show route

1.1.1.1/32         via 10.10.0.159 on ens5 [tohost1 15:10:46] * (100/0) [i]

2.2.2.2/32         via 10.10.0.205 on ens5 [tohost2 15:10:47] * (100/0) [i]

1.1.1.2/32         via 10.10.0.159 on ens5 [tohost1 15:10:46] * (100/0) [i]

2.2.2.1/32         via 10.10.0.205 on ens5 [tohost2 15:10:47] * (100/0) [i]

10.10.0.0/24       dev ens5 [direct1 15:10:42] * (240)

bird> 

 

2. Configure Router Reflector 

 

Now I will edit the TOP host to work as the router. It is simple. I insert the "rr client" option into the BGP configuration like below.

protocol direct {
#        interface "*";
        interface "ens5";
}

protocol bgp tohost1{
        import all;
        export all;
        rr client;
        local as 65001;
        neighbor 10.10.0.159 as 65001;
}

protocol bgp tohost2{
        import all;
        export all;
        rr client;
        local as 65001;
        neighbor 10.10.0.205 as 65001;
}

After then, I can see the routing table on Host #1 and Host #2

Host #1

Host #2

root@ip-10-10-0-159:~# birdc

BIRD 1.6.8 ready.

bird> show route

1.1.1.1/32         via 10.10.0.159 on ens5 [static_bgp 15:17:59] ! (200)

2.2.2.2/32         via 10.10.0.205 on ens5 [bgp1 15:18:03 from 10.10.0.49] * (100/0) [i]

1.1.1.2/32         via 10.10.0.159 on ens5 [static_bgp 15:17:59] ! (200)

2.2.2.1/32         via 10.10.0.205 on ens5 [bgp1 15:18:03 from 10.10.0.49] * (100/0) [i]

10.10.0.0/24       dev ens5 [direct1 15:17:59] * (240)

                   via 10.10.0.49 on ens5 [bgp1 15:18:03] (100/0) [i]

root@ip-10-10-0-205:~# birdc

BIRD 1.6.8 ready.

bird> show route

2.2.2.2/32         via 10.10.0.205 on ens5 [static_bgp 15:23:13] * (200)

1.1.1.1/32         via 10.10.0.159 on ens5 [bgp1 15:23:17 from 10.10.0.49] * (100/0) [i]

2.2.2.1/32         via 10.10.0.205 on ens5 [static_bgp 15:23:13] * (200)

1.1.1.2/32         via 10.10.0.159 on ens5 [bgp1 15:23:17 from 10.10.0.49] * (100/0) [i]

10.10.0.0/24       dev ens5 [direct1 15:23:13] * (240)

                   via 10.10.0.49 on ens5 [bgp1 15:23:17] (100/0) [i]

Now I can send ICMP each other.

Host #1 Host #2

Now I can use the Internal BGP and Router Reflector option.

 

Reference 

[ 1 ] https://createnetech.tistory.com/52?category=679927

[ 2 ] https://saidvandeklundert.net/2015-06-14-bird-bgp-route-reflection/ 

[ 3 ] www.bird.network.cz/?get_doc&v=20&f=bird-6.html#ss6.3

 

+ Recent posts