In article#003 and #004 you learned how RARE/freeRouteris controlling a P4 dataplane (BMv2 or TOFINO virtual model). We also demonstrated that this setup could be integrated into real networks. However, these P4 dataplanes are not suitable for day to day real operation as it have inherent software limitations. While freeRouternative software dataplane presents the advantage to get the entire feature set and is sufficient to handle a home network traffic load, we investigated a way to improve dataplane performance. In that context we considered to study:
However, XDP model was not complete enough in order to compile router.p4 and we could not generate the corresponding kernel bypass code with ELTE T4P4S based on BMv2 V1Model.p4. (A GitHub issue is still pending). In that context, Csaba freeRouter lead developer decided to develop P4Emu a software dataplane that has the particularity to:
understand freeRouter control plane message meant to be addressed to a P4 dataplane
thus maintaining the control plane decoupled to the dataplane as it was the case with BMv2 and BF_SWITCHD
One would ask: Why write yet another software dataplane as freeRouter has already a working native software dataplane. This is a very good and valid question. The answer boils down in:
"decoupling control plane from the dataplane"
We will see in subsequent article how P4Emu unlock new valid uses cases.
Article objective
In this article we'll using freeRouter setup deployed in #004and replace bf_switchd providing freeRouter INTEL/BAREFOOT TOFINO'sdataplane by P4Emu/pcap.
It is important to note that though its name, P4Emu/pcap is not emulating V1Model. P4Emu is emulating router.p4packet processing logic and uses a packet forwarding library to effectively transmit packets at specific ingress port to the right egress port defined by freeRouter control plane message.
Diagram
[ #005 ] - Cookbook
In our example we will use the same debian stable image (buster) installed as a VirtualBox VM as in #002.
and we add a bridge network interface to or laptop RJ45 connection.
flush enp0s3 so that it can be controlled by dpdk
ip addr flush enp0s3
Add out of band management enp0s8 with Virtualbox
You can add a second Host-only interface (enp0s8) in VirtualBox in order to connect the ubuntu focal VM guest as you might lose connection when you flushed enp0s3.
mkdir -p ~/freeRouter/bin ~/freeRouter/lib ~/freeRouter/etc ~/freeRouter/log
cd ~/freeRouter/lib
wget http://freerouter.nop.hu/rtr.jar
For those you would like to rebuild these binaries you can find the compilation shell script in freeRouter cloned git repository in:~/freeRouter/src/native/c.sh
FreeRouter uses 2 configuration files in order to run, let's write these configuration files for R1 in~/freeRouter/etc
hostname pcap-freerouter
buggy
!
vrf definition v1
exit
!
interface ethernet0
description freerouter@P4_CPU_PORT[veth251]
no shutdown
no log-link-change
exit
!
interface sdn1
description freerouter@sdn1[enp0s3]
mtu 9000
vrf forwarding v1
ipv4 address 192.168.0.131 255.255.255.0
ipv6 address 2a01:e0a:159:2850::666 ffff:ffff:ffff:ffff::
ipv6 enable
no shutdown
no log-link-change
exit
!
!
!
!
!
!
!
!
!
!
!
!
!
!
server telnet tel
security protocol telnet
no exec authorization
no login authentication
vrf v1
exit
!
server p4lang p4
export-vrf v1 1
export-port sdn1 1 0
interconnect ethernet0
vrf v1
exit
!
end
Setup P4Emu dataplane communication channel via veth pair and interface adjustment (disable IPv6 at VM guest level, MTU 10240, disable TCP offload etc.)
echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/default/disable_ipv6
ip link add veth251 type veth peer name veth250
ip link set veth250 up
ip link set veth251 up
ifconfig enp0s3 promisc
ifconfig veth250 promisc
ifconfig veth251 promisc
ip link set dev veth250 up mtu 10240
ip link set dev veth251 up mtu 10240
ip link set dev enp0s3 up mtu 10240
export TOE_OPTIONS="rx tx sg tso ufo gso gro lro rxvlan txvlan rxhash"
for TOE_OPTION in $TOE_OPTIONS; do
/sbin/ethtool --offload veth250 "$TOE_OPTION" off &> /dev/null
/sbin/ethtool --offload veth251 "$TOE_OPTION" off &> /dev/null
/sbin/ethtool --offload enp0s3 "$TOE_OPTION" off &> /dev/null
done
freeRouter launch with supplied pcap-freerouter-hw.txt and pcap-freerouter-sw.txt with a console prompt
java -jar lib/rtr.jar routersc etc/pcap-freerouter-hw.txt etc/pcap-freerouter-sw.txt
info cfg.cfgInit.doInit:cfgInit.java:556 booting
info cfg.cfgInit.doInit:cfgInit.java:680 initializing hardware
info cfg.cfgInit.doInit:cfgInit.java:687 applying defaults
info cfg.cfgInit.doInit:cfgInit.java:695 applying configuration
info cfg.cfgInit.doInit:cfgInit.java:721 done
welcome
line ready
pcap-freerouter#
launch freeRouter pcapInt in order to stitch control plane and P4Emu/pcap dataplane communication
cd ~/freeRouter/bin
./pcapInt.bin veth251 22709 127.0.0.1 22710 127.0.0.1
binded to local port 127.0.0.1 22709.
will send to 127.0.0.1 22710.
pcap version: libpcap version 1.8.1
opening interface veth251 with pcap1.x api
serving others
>
Please observe the 1st ICMP packet loss that triggered IPv6 neighbor discovery for respectively 2a01:e0a:159:2850::1 and 2a01:e0a:159:2850:e23f:49ff:fe6d:1899
Initiate IPv4 ssh from freerouter -> LAN router gateway
pcap-freerouter#ssh 192.168.0.62 /vrf v1 /user my-nas
- connecting to 192.168.0.62 22
password: *******
- securing connection
Last login: Mon Jul 6 15:05:38 2020 from 192.168.0.77
FreeBSD 11.3-RELEASE-p9 (FreeNAS.amd64) #0 r325575+588899735f7(HEAD): Mon Jun 1 15:04:31 EDT 2020
FreeNAS (c) 2009-2020, The FreeNAS Development Team
All rights reserved.
FreeNAS is released under the modified BSD license.
For more information, documentation, help or support, go here:
http://freenas.org
Welcome to FreeNAS
MY-NAS%
Initiate IPv6 ssh from freerouter -> LAN router gateway
pcap-freerouter#ssh 2a01:e0a:159:2850:e23f:49ff:fe6d:1899 /vrf v1 /user my-nas
- connecting to 2a01:e0a:159:2850:e23f:49ff:fe6d:1899 22
password: *******
- securing connection
Last login: Tue Jul 7 16:01:54 2020 from 2a01:e0a:159:2850::666
FreeBSD 11.3-RELEASE-p9 (FreeNAS.amd64) #0 r325575+588899735f7(HEAD): Mon Jun 1 15:04:31 EDT 2020
FreeNAS (c) 2009-2020, The FreeNAS Development Team
All rights reserved.
FreeNAS is released under the modified BSD license.
For more information, documentation, help or support, go here:
http://freenas.org
Welcome to FreeNAS
MY-NAS%
Conclusion
In this article you:
had a demonstration of how to integratefreeRouter into a local area network (Similar to article#002)
However instead of usingbmv2 or TOFINOwe used aP4Emu/pcap dataplane
In this example the freeRouter with P4Emu/pcap has only 1 dataplane interface that is bound to enp0s3 VM interface exposed to the local network as a bridged interface
[ #005 ] RARE/FreeRouter-101 - key take-away
FreeRouteris using UNIX socket in order to forward packet dedicated to control plane + dataplane communication.
This essential paradigm is used to ensure communication betweenfreeRouterandP4Emu/pcap dataplane. It is ensured bypcapIntbinary from freeRouter net-tools that will bind freeRouter socket (veth251@locathost:22710) to a virtual network interface (veth250@localhost:22709) connected to CPU_PORT 0.
freeRouter is the control plane forP4Emu/pcap dataplane
freeRouteris doing all the control plane route computation and write/modify/remove message entry P4 entries are created/modified/removed accordingly from P4Emu/pcap tables. Although the name is P4Emu, it does not emulate BMv2 V1Model.p4, but rather router.p4
In this setup the combination of freeRouter/pcap deliver a solution for SOHO network having 1GE links
However, 1GE traffic rate require 50% of one CPU thread. Nevertheless, traffic rate achieved is higher with P4Emu/pcap than freeRouter native software packet forwarding software.
In subsequent article we will see how we can improve the latter requirement implied by P4Emu/pcap.
