Delay Measurement in Openflow-Enabled MPLS-TP Network

,


Introduction
SDN is a recent approach aimed to make networks programmable.The principle of SDN is to decouple control functionality (also known as control plane) from forwarding functionality (also known as data plane) which helps to develop a network-wide abstraction while keeping data plane as simple as possible (ONF).
Besides the network abstraction, the SDN architecture provide a set of Application Programming Interfaces API that simplifies the implementation of common network services.In SDN, the network intelligence is sitting in the Network Operating System NOS and network applications (e.g.switching, routing, load balancing, security, etc.) which use a northbound API to communicate with centralized SDN controllers.Network devices become the simple packet forwarding devices (e.g.SDN Switch) that can be programmed via an open interface: Southbound API.Openflow is one of the mostly used southbound interface (Azodolmolky et al., 2013).Figure 1 shows how Openflow is used to orchestrate intercommunication between SDN controller and SDN Switches.Applications are a generic term that in this context could cover both network and service-related functions.

Figure 1. SDN overall architecture
The dynamic and the growth of computing and storage (e.g.Data centers) urge network designers to adopt SDN as new network paradigm in order to meet the new networking requirements.An example of requirements includes changing of traffic patterns because applications that commonly access geographically distributed databases and servers through public and private clouds require extremely flexible traffic management and access to bandwidth on demand.
New technologies have been raised in order to answer this kind of new needs.MPLS-TP as Packet Transport solution can match with SDN conditions especially when considering some of its major attributes (Figure 2.) (Jarschel et al., pp -48-53, 2012): • Data Plane : remains exactly the same as MPLS to facilitate interoperability with MPLS; • Control Plane: optional, dynamic via IP based protocols or static via management platform; • OAM: transport-like OAM, monitoring and driving switches between primary and backup paths for path segments; • Protection and Resiliency: Similar operation with SDH; Figure 2. What is MPLS-TP?
The main goal of MPLS-TP is to provide connection-oriented transport for packet and time division multiplexing services over optical networks leveraging the widely deployed MPLS technology.It supports the definition and implementation of OAM and resiliency features to ensure the capabilities needed for carrier-grade transport networks -scalable operations, high availability, performance monitoring and multi-domain support (Beller et al., pp 81-92, 2009).
However, there are still some points that MPLS-TP and actual transport network are still missing (Hubbard at al., pp 9 -10, 2012): • Complexity of adding or moving devices which make it time-consuming and risky.
• Inability to scale or test new idea such as new network applications because the source code of the software running on the switches cannot be modified and long delays to introduce new features" • Vendor dependence which is traduced by lack of standard and limitation: Software bundled with hardware, and Vendor-specific interfaces.

•
Cost building, operating, scaling networks delivering innovative services remain unacceptably high.
• SDN comes to overcome such kind of limitations and permit to offer: • Abstraction: Software decoupled from hardware and Standard interface (e.g.Openflow) • Programmability: Enable innovation using open APIs, Acceleration of introduction of new features and new services

•
Centralized intelligence: Simplify provisioning, optimize performance and granular policy management In this paper, we are studying the case of an MPLS-TP network using an SDN Model from the OAM point of view.As per MPLS-TP OAM requirements, OAM should follow same path as user traffic and are initiated and managed in a centralized manner, we will then present our testbed showing the impact of having centralized OAM engine while increasing number of hops and number of monitored MPLS LSP (Busi et al., 2011).
The organization of this paper is as follows: In Section 2, we explain what different components for this testbed are and how they interact each with other.In section 3 we presents results of each test scenario using in-band DM approach as an MPLS-TP's OAM.Finally, concluding remarks are given in Section 4.

Testbed for an Openflow-Enabled MPLS-TP Network
In this chapter, we will start by talking briefly about SDN architecture and Openflow messages, and we will present our Delay Measurement model and the way how it is implemented.

Openflow Principles
The SDN based architecture is based on three principals (Schenker, 2011)   At this point, SDN does not provide any mechanism to measure Delay between Openflow switches or between switches and controller.Thus, we are proposing a proactive method which will allow controller to generate monitoring packets in order to provide a real time delay measurement.This kind of measures can then easily be used to make real time traffic engineering.

Delay Measurement Proposed Model
The proposed method of measuring delay is based on injecting a probe packet from controller, and then sent to source switch which will forward it to the end switch trough the user label switched path, and finally return back to the controller.The other delay from controller to both source and end switches are calculated separately each other.We will explain now in depth how we are calculating all these delay values.
Based on the relative standards and recommendations of MPLS-TP requirements (Bocci et al., 2009), OAM packets should run in-band and share their fate with data packets.Thus, echo packet generated by controller should take same path as user (Figure 6).The Delay is measured by computing the difference between the packets departure and arrival times, subtracting with the estimated switch-to-controller and controller-to-switch times, using the following formula: is the time to take to get from switch ( e.g.switch1 or switch2) to the controller This formula can be simplified by considering that controller-switch Round Trip Time RTT is equal to twice controller-to-switch time: This formula is not dependant of any synchronization time between nodes (between switches and controller, or between switches each other) since all these variable are controller's clock dependant only.So there is no need to have a Network Time Protocol server inside our testbed.This formula is ignoring delay caused by nodes processing.This is due to the fact that we are using virtual machines in order to simulate hosts, switches and controller.

Implementation Concept
In order to elaborate the module capable of measuring delay, we need to take profit know what messages are exchanged between switches and controller.
After starting our topology including Controller, hosts and Switches, we set default values related to each link especially bandwidth, loss and Delay.Once the Openflow switches establish tcp connection with the controller responds back with a Features_Request message.This Event EventOFPSwitchFeatures is used to populate flow tables in switch1, Switch2 and intermediated switches.The intermediate switches will mainly execute a swap MPLS label operations, while Switch1 and Switch2 will have to do push and pop MPLS Label operation.Switch1 and Switch2 flow tables should be able to deal with three kinds of packets:

•
Packets sourced from hosts (e.g.Address Resolution Protocol ARP, Normal IP traffic with or without VLAN tag) • Packets coming from "Network": from other switches ( e.g.tagged with MPLS Label in our case) • Packets coming from Controller including OAM probe packet which is used to calculate Total Delay.
This why flow tables which should be initially installed on switches in order to take in consideration different kind of flows.
Once flow tables are installed, the controller start sending Request Statistics from switches in order to calculate their related RTTs by checking Statistics event EventOFPPortStatsReply.
The Figure 7 is summarizing different timer.
Figure 7. Summary of different timers In the same time, a monitoring process is sending continuously an OAM probe message from controller to itself spoofing (Liu et al., 2008) source and destination mac address by setting them respectively to the mac source address of the switch1's port connected to host1 and respectively to the mac destination address of the switch2's port connected to host2.The Packet_In Event handler allows then to know the time of arrival of packet to the controller.Thus, we are able to know the total delay it took the probe message to go through all the switches in this Path.By matching switch1's source mac address and switch2's destination mac address couple, allow us to calculate total delay for an end to end path.Similarity, other Total delay for other paths can be easily calculated just by modifying source and destination mac address to math with their end to end path.
The flowcharts (Figure 8) bellow explain how packets are handled at controller and at switch level:

Evaluation
In this chapter, we are presenting our proposed the environment of our prototype and different scenario of testing.

Prototype
We are using two open source software to build our prototype:

•
Ryu is playing the role of the Openflow controller

Ofsoftswitch is running as Openflow Switch
Mininet is the network emulator which will run OFSOFTSWITCH13 processes.It will also emulate hosts and links.

Ryu
Ryu is a component-based, open source framework implemented entirely in Python and which is supported by NTT Labs (RYU).The Ryu messaging service does support components developed in other languages.
Components include an Openflow wire protocol support event management, messaging, in-memory state management, application management, infrastructure services and a series of reusable libraries (e.g., NETCONF library, sFlow/Netflow library).

Mininet and Ofsoftswitch
Mininet is a network emulator which is able to simulate Switches, routers and hosts on a single Linux kernel.
Mininet is an open source SDN project hosted in Github (Github).It provides an easy way to get correct system behavior and tool to create topologies.It uses virtual bridges and interfaces in order to process packets between emulated hosts (Mininet).every emulated switch or host creates its own process.Mininet offer also a straight forward and extensible Python API to allow users to develop new Openflow application and customized topologies (Lantz et al., 2010).In fact, the code developed in Mininet emulation can be moved to a real production network.
Hosts are generating UDP traffic by using iperf Linux command, which is a tool for active measurements of the maximum achievable bandwidth on IP networks.It supports tuning of various parameters related to timing, protocols, and buffers.For each test it reports the bandwidth, loss, and other parameters (Iperf).
Mininet support using different kind of Openflow switches.We choose Ofsoftswitch13 since it is an Openflow 1.3 compatible user-space software implementation supporting MPLS and statistics features (CPqD).
Ofsoftswitch13 is originally based on Ericsson TrafficLab 1.1 code.

Testbed
In this section, we present the performance of a demo application for monitoring delays.This application is developed using the Ryu API , where the controller periodically polls the switches at a constant interval to gather delays information.The topology of the testbed is also customized using Mininet API.We are using Ryu 3.13 version, and mininet 2.1.0+version.
We have run two scenarios with 8 different rates at each time (Figure 9): • Controller running one instance of OAM probe for one MPLS LSP, number of intermediate switches increased and iperf generating traffic at different rates.

•
Controller running an instance of OAM probe per MPLS LSP, number of intermediate switches increased and iperf and iperf generating traffic at different rates.
The relieved measure is the average of measures took during an interval of 100 seconds.
The emulation uses two Intel core i5 laptops with four 2.3 Ghz cores and 4 GB RAM.The first Virtual Machine is emulating the controller side while the second one is emulating switches and hosts.
Probing intervals are set to 500 ms.Regarding probing value less than 500 ms we noticed unpredictable behavior with very excessive delay value.This is due to the fact that processes in Mininet do not run in parallel, instead they use time multiplexing.
Host2 and host4 are running iperf as server: Where "-s" option mean server and "-u" option means UDP.Where "-t" means time set to 100 second and "-b" means bandwidth set to 1Mbps.The other part of the prototype concerns the application measuring the delay and which use Ryu API.This application provides a module which is running the probe and other module which react to a received event from different switches.The module running the probe help us to construct the icmp echo request using spoofed mac address, here is a sample of it: def _monitor(self): while True: for dp in self.dps.values(): self._probe(dp) hub.sleep(0.5) The function _probe is using datapath as argument which help to identify a switch in unique manner.The sleep function set the probe interval to 500 ms.The event sensitive module is handling three type of events: • EventOFPStateChange: it helps us to add or remove datapath.iddynamically and know which switch is active • EventOFPSwitchFeatures: we use this event to install flow table at switch startup • EventOFPPacketIn: is used to obtain total delay and then the Delay between Switches.

Results
We started our first test with a simple topology in order to make calibration and know Delay value when there is no traffic over the network Figure 10.For this test case we run two scenarios: The first is using OAM probe interval of 200 ms The Second is using OAM probe interval of 500 ms Then we measure delay at different rates using iperf from 1 Mbps to 90 Mbps when host 2 is playing the role of UDP server.The UDP client is sending 1470 byte datagrams and UDP buffer size is set by default to 208 Kbyte.
Each test is running during 100 seconds and the average measure is adopted.
The RTT values retrieved was measured for both Switch 1 and Switch 2 under same conditions Figure 11.In the next test case, we created a topology with Three switches, then with Five Switches and then with Ten Switches in linear manner: For each topology we have established to end-to-end LSP and run an OAM probe instance for each LSP.Then, we measured delay for different rates using iperf.Figure 12.For the last scenario we tried to stress the testbed by adding more LSPs over a ten open switches linear topology Figure 13: Figure 12.Test case #3 results: one OAM probe per LSP (number of LSP = 5, number of hops:10) During testcase#1, we don't notice any important variation delays while increasing generated bandwidth inside same user datapath.The measured value were not exceeding 1,9 ms in worst case scenario.
The second test case shows how delays is increasing two to three times when comparing the two monitored LSP's scenario with the five monitored LSP's scenario.The fact of running simultaneous OAM probe instance increase the CPU load of the controller and then capacity of treating the Packet-In event which is the place where calculation of delay is done.
The last stress scenario is confirming the result of second scenario since delay is increasing dramatically with almost forty time the calculated delay in the first scenario.

Conclusion
In this paper, we have presented a testbed which ca measure Delay between different switches inside network using in band OAM.The usage of in-band OAM is a requirement in MPLS-TP Network.We used a centralized OAM Model in our testbed where OAM are initiated from controller.This model has proven some limitation as Mininet is a very useful network emulator allowing testing proof of concept but is not adequate for stress tests.
Based in this work, we suggest implementing OAM in distributed model by adding OAM Engine at Openflow switches level.The distributed model will allow running a highest number of OAM prober among the SDN network without risking to loading controller's CPU resources.The distributed model will lead to add new messages between controller and switches related to OAM activities.
As future work, we envision using Raspberry PI as platform hosting Openflow switch software in order to avoid using Virtual machine.The next step in our work is to complete the design and implementation of other Quality of service parameters (e.g.Loss, bandwidth …).This will allow us to be able to do real time traffic engineering in a sort of network as a service platform.

Figure 4 .
Figure 4. Pipeline processing Openflow protocol has three message types: • Symmetric: Sent without solicitation, either by Switches or by Controller;  Hello: Connection startup;  Echo: Request/Reply, aims to test connections and aliveness.• Controller to Switch :Initiated by controller: may or may not require a response;  Features: Request/Reply, used to get the capabilities of a switch and to get statistics of ports/flows;  Packet-Out: Explicit instruction to forward a packet;  Flow-mod: Add/delete flows of the specified switch; • Asynchronous: Event driven messages, sent by Switches to Controller;  Packet-in: a packet in event to be notified to Controller;  Port-status: A port of a switch changed its state; Figure 5 is summarizing different types of Openflow messages:

Figure 6 .
Figure 6.Controller injecting in-band Probe packets the probe OAM packet initiated from controller is received back in controller : Sent T is the time the probe OAM packet initiated from controller is sent from controller : to take to get from controller to switch (e.g.switch1 or switch2) :

Figure 8 .
Figure 8. Flowchart explaining different flow table processing at intermediate switches are not represented since they are doing a basic operation of MPLS label swapping.

Figure 11 .
Figure 11.Test case #1 Results: One OAM for One LSP

Figure 12 .
Figure 12.Testcase #2 results: one OAM probe per LSP (number of LSP = 2, number of hops : 3, 5 and 10) As per Openflow specification (ONF, 2012), the forwarding decisions called also flow tables, inside an Openflow switch, are flow-based instead of destination-based.An entry in the flow table is composed of multiple fields (Figure3): • Matching Rule: represented by packet header fields values (e.g.input port, ip source/destination, tcp/udp port, IP dscp, vlan id, MPLS label …) • Action: the way to handle packets ( e.g.drop, flood to other switches or to controller, go to next table …) • Statistics: Number of received packets, lapse time, etc