Open vSwitch - Provisioning and Monitoring with vswitch Extended NIC stats - High Level Design

In OVS there are few areas where netdev statistics are used:

• Bridge which periodically gets the stats and puts it into OVSDB.
• Any OpenFlow query can get statistics (e.g. ovs-vsctl dump-ports command).
• sFLOW agent periodically queries statistics.

Port statistics in OVS up to this Epic were defined as a list of counters which contains:

• Normal statistics (e.g. rx_packets, tx_packets etc…)
• Extended statistics – statistics based on RFC2819.

This EPIC extends netdev interface to support “Custom statistics”.

1. Custom statistics

Custom statistics are part of new extension in netdev interface, and allows particular device implementation to expose counters in dictionary form. Such statistics are exposed through:

• OpenFlow
• OVSDB

It is up to particular netdev implementation to expose any additional statistics.

2. Custom statistics for netdev-dpdk devices

In netdev-dpdk all “Custom statistics” are taken from XSTATS. Such list is filtered to expose only dropped and error related counters. This is implemented by filtering out all counters where name doesn’t end with “_dropped” or “_error” strings, or doesn’t contain “_management_” string. Such generic solution allows to easily implement new counters on DPDK side without any new implementation in OVS itself. Because each DPDK driver can implement different counters, exposed counters by OVS can vary based on environment. In netdev-dpdk statistics are calculated in two steps:

• Counter names and IDs are taken from DPDK XSTATS – this activity is called each time port configuration will change, as we can’t guarantee that IDs and amount of counters will remain the same. This part of implementation is done in netdev-dpdk.c/h:
  static bool netdev_dpdk_configure_xstats(struct netdev_dpdk *dev) OVS_REQUIRES(dev→mutex)
• Second part of functionality is netdev interface custom statistics function implementation, so querying statistics from DPDK XSTATS (using cached IDs and names – explained above):
  static int netdev_dpdk_get_custom_stats(const struct netdev *netdev, struct netdev_custom_stats *custom_stats)

  
  

  
  

3. Generic implementation

There are two main implementation areas:

• Netdev interface update.
• OpenFlow extension and integration with ovs-ofctl.

a. Implementation – netdev

To allow each netdev device to expose new counters, main netdev interface has to be extended.

    /* Retrieves current device custom stats for 'netdev' into 'custom_stats'.

     *

     * A network device should return only available statistics (if any).

     * If there are not statistics available, empty array should be

     * returned.

     *

     * The caller initializes 'custom_stats' before calling this function.

     * The caller takes ownership over allocated array of counters inside

     * structure netdev_custom_stats.

     * */

    int (*get_custom_stats)(const struct netdev *netdev, struct netdev_custom_stats *custom_stats);

This particular function is optional for any device. If it is implemented, it will be executed each time port statistics are requested, always after ‘get_port_stats’ interface function.

b. Implementation – OpenFlow extension

“Custom statistics” in openflow are implemented using “Experimenter feature”. This feature was introduced in later revisions of OpenFlow specifications and allows to append “experimental” content to already defined message type. For more information please see section 7.3.5.5, "Port Statistics", in OpenFlow 1.5 (https://www.opennetworking.org/wp-content/uploads/2014/10/openflow-switch-v1.5.1.pdf). This feature was introduced to simplify new implementations of vendor specific functionalities. The only downside of it, is that it is supported from OpenFlow 1.4 (in this particular release experimenter feature was enabled for port statistics), it means that it is mandatory to explicitly state which OpenFlow version will be used to get the statistics. (e.g. when using ovs-ofctl it is mandatory to state OpenFlow14+ to get “Custom statistics” using “dump-ports” command).

Each new feature implemented in the open flow has to provide functions to parse and create OpenFlow message. In this particular case when statistics are requested through OpenFlow (for example through ovs-ofctl “dump ports” command), a message which contains all counters is created. To check the implementation how Open Flow message is created, please check to ofp-util.c/h:

static void ofputil_append_ofp14_port_stats(const struct ofputil_port_stats *ops, struct ovs_list *replies)

Open Flow buffer decoding (parsing) can be found also in ofp-util.c/h:

static enum ofperr parse_intel_port_custom_property(const struct ofpbuf *payload, struct ofputil_port_stats *ops)