Anuket Project

Test Environment

       
Bare Metal     VM
Ubuntu 16.04/4.4  Fedora 23/?  Same as for Bare Metal
       
DPDK 16.04DPDK 16.07 DPDK 16.04DPDK 16.07  
stat and link status events stat and link status events  stat and link status events stat and link status events   
igb, ixgbe, i40e, fm10k, vhostigb, ixgbe, i40e, fm10k, vhost igb, ixgbe, i40e, fm10k, vhostigb, ixgbe, i40e, fm10k, vhost  
Huge Pages: 1GB or 2MBHuge Pages: 1GB or 2MB Huge Pages: 1GB or 2MBHuge Pages: 1GB or 2MB  
 keep alive  keep alive  
       
OvS (latest master)  OvS (latest master)   
igb, ixgbe, i40e, fm10k, vhost  igb, ixgbe, i40e, fm10k, vhost   
Huge Pages: 1GB or 2MB  Huge Pages: 1GB or 2MB   
       
       
       
List of environments      
E1Bare Metal, U16.04, DPDK 16.04 E1,E2, E4,E5,E7,E8,E10,E11   
E2Bare Metal, U16.04, DPDK 16.07 E2,E5,E8,E11E3,E6,E9,E12  
E3Bare Metal, U16.04, OvS 2.5?     
E4Bare Metal, F23, DPDK 16.04     
E5Bare Metal, F23, DPDK 16.07     
E6Bare Metal, F23, OvS 2.5?     
E7VM, U16.04, DPDK 16.04     
E8VM, U16.04, DPDK 16.07     
E9VM, U16.04, OvS 2.5?     
E10VM, F23, DPDK 16.04     
E11VM, F23, DPDK 16.07     
E12VM, F23, OvS 2.5?     
       
Driver's info:      
IXA00390838  X-710 = igb_uio (10G/40G)   
uep2932 (host from DCRP setup)  X520-2 = ixgbe(10G)  https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Ethernet-Network-Connections-under-Linux-
RRC?  ? = fm10k(10G?/40G/100G)   
   X-710 = i40e (10G/40G)  https://downloadcenter.intel.com/download/24411/Intel-Network-Adapter-Driver-for-PCI-E-40-Gigabit-Ethernet-Network-Connections-under-Linux-
   ? = igb (1G)  https://downloadcenter.intel.com/download/13663/Intel-Network-Adapter-Driver-for-82575-6-82580-I350-and-I210-211-Based-Gigabit-Network-Connections-for-Linux-
   i1000e driver support?   

 

Tests

 Scope      
 Plugins covered: dpdk stats, dpdk events, Hugepages (HAA-928) and their interaction.  
 Preconfiguration: DPDK, collectd (with debug enabled), collectd plugins are installed. Ports are binded to DPDK.  
        
 DPDK stats tests      
TC#Tast case titlePriorityRun onStep#StepsExpected resultsComments
1DPDK collectd stats configuration testing negative '-T'/'-C'MediumE1,E2, E4,E5,E7,E8,E10,E111Change collectd plugin configuration in collectd.conf to incorrect values (coremask, ProcessType, FilePrefix, EnabledPortMask). Run testpmd, collectd.Warning/error should be logged about invalid plugin parameter value. 
    2Change collectd plugin configuration in collectd.conf to correct values. Re-run testpmd, collectd.No Errors/Warning are logged. 
    3Run testpmd. Get ports statistic.Statistic is same in collectd and system. 
        
2DPDK and collectd port counters number on different driversHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd.Number of counters is correct in relation to NIC driver. Number of counters read from collectd and system is the same.  Number of counters read from port#0 and port#1 is the same. dpdk_stats derived counters (##102), driver - i40e:
rx_low_power_idle_count
rx_low_power_idle_status
rx_priority1_xoff_packets
rx_priority2_xon_packets
rx_priority6_xon_packets
tx_low_power_idle_count
tx_low_power_idle_status
tx_priority7_xon_to_xoff_packets
mac_local_errors
mac_remote_errors
rx_dropped
rx_mac_short_dropped
rx_crc_errors
rx_errors
rx_fragmented_errors
rx_illegal_byte_errors
rx_jabber_errors
rx_length_errors
rx_mbuf_allocation_errors
rx_oversize_errors
rx_q0_errors
rx_undersized_errors
rx_error_bytes
rx_good_bytes
rx_q0_bytes
rx_packets
rx_broadcast_packets
rx_flow_director_atr_match_packets
rx_flow_director_sb_match_packets
rx_good_packets
rx_multicast_packets
rx_priority0_xoff_packets
rx_priority0_xon_packets
rx_priority1_xoff_packets
rx_priority1_xon_packets
rx_priority2_xoff_packets
rx_priority2_xon_packets
rx_priority3_xoff_packets
rx_priority3_xon_packets
rx_priority4_xoff_packets
rx_priority4_xon_packets
rx_priority5_xoff_packets
rx_priority5_xon_packets
rx_priority6_xoff_packets
rx_priority6_xon_packets
rx_priority7_xoff_packets
rx_priority7_xon_packets
rx_q0_packets
rx_size_1024_to_1522_packets
rx_size_128_to_255_packets
rx_size_1523_to_max_packets
rx_size_256_to_511_packets
rx_size_512_to_1023_packets
rx_size_64_packets
rx_size_65_to_127_packets
rx_unicast_packets
rx_unknown_protocol_packets
rx_xoff_packets
rx_xon_packets
tx_dropped
tx_link_down_dropped
tx_errors
tx_good_bytes
tx_q0_bytes
tx_broadcast_packets
tx_good_packets
tx_multicast_packets
tx_priority0_xoff_packets
tx_priority0_xon_packets
tx_priority0_xon_to_xoff_packets
tx_priority1_xoff_packets
tx_priority1_xon_packets
tx_priority1_xon_to_xoff_packets
tx_priority2_xoff_packets
tx_priority2_xon_packets
tx_priority2_xon_to_xoff_packets
tx_priority3_xoff_packets
tx_priority3_xon_packets
tx_priority3_xon_to_xoff_packets
tx_priority4_xoff_packets
tx_priority4_xon_packets
tx_priority4_xon_to_xoff_packets
tx_priority5_xoff_packets
tx_priority5_xon_packets
tx_priority5_xon_to_xoff_packets
tx_priority6_xoff_packets
tx_priority6_xon_packets
tx_priority6_xon_to_xoff_packets
tx_priority7_xoff_packets
tx_priority7_xon_packets
tx_priority7_xon_to_xoff_packets
tx_q0_packets
tx_size_1024_to_1522_packets
tx_size_128_to_255_packets
tx_size_1523_to_max_packets
tx_size_256_to_511_packets
tx_size_512_to_1023_packets
tx_size_64_packets
tx_size_65_to_127_packets
tx_unicast_packets
tx_xoff_packets
tx_xon_packets
    2Restart testpmd, collectd plugins.After restart Number of counters is same as in step above. 
    3Repeat test for the following drivers: igb, i40e, ixgbe, fm10k, vhost.Number of counter needs to be updated!!! 
        
3DPDK and collectd port counters values with testpmd on different driversHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Stop traffic.Values are not incremented for port counters read from collectd/system. 
    3Start bidirectional traffic.Counters' values started to change. Values read at the same time for each counter in collectd and system is equal for the same ports. 
    4Repeat test for the following drivers: igb, i40e, ixgbe, fm10k, vhost.Number of counter needs to be updated!!! 
        
4DPDK and collectd port counters values with testpmd and different plugin parameters (interval, etc.)HighE1,E2, E4,E5,E7,E8,E10,E111Set collectd plugin 'interval' parameter value to 10 second. Run testpmd, collectd. Start bidirectional traffic.Values are gathered by collectd plugin every 10 seconds. 
    2Change 'interval' parameter value between min,max value. Run bidirectional traffic.Values are gathered by collectd plugin by the provided interval. 
        
5DPDK and collectd port counters values upon testpmd restartHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Stop testpmd.Collectd plugin process status is ???  Values are not incremented for port counters read from collectd/system. 
    3Start testpmd. Restart collectd. Traffic is running.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    4Kill testpmd process.Collectd plugin process status is ???  Values are not incremented for port counters read from collectd/system. 
    5Start testpmd. Restart collectd.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
6DPDK and collectd port counters values upon collectd restartHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Stop collectd (plugin?).Testpmd is still running.  Values are not incremented for port counters read from collectd. Values are not incremented for port counters read from system. 
    3Start collectd (testpmd should be running). Traffic is running.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
7DPDK and collectd port counters values upon collectd plugin terminate/startHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Terminate collectd plugin process.Testpmd is still running.  Values are not incremented for port counters read from collectd. Values are not incremented for port counters read from system. 
    3Start collectd (testpmd should be running). Traffic is running.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
8DPDK and collectd port counters values upon two or more collectd processes are startedMediumE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Try to start another collectd process (with same command options as running one).Only one collectd process is running???
Values read at the same time for each counter in collectd and system is equal for the same ports???
 
    3Restart collectd (testpmd should be running). Traffic is running.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
9DPDK and collectd port counters values upon two or more testpmd apps are startedMediumE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Try to start another testpmd on the same resources (cpu, port).Testpmd not started?
Values read at the same time for each counter in collectd and system is equal for the same ports???
 
    3Restart testpmd, collectd. Traffic is running.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
10DPDK and collectd plugin statistical counters overflowHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd.All settled. 
    2Run specially prepared test application to modify system counters to boundary values.Verify values are set properly. 
    3Start bidirectional traffic.Counters are close to their boundary values. Values read at the same time for each counter in collectd and system is equal for the same ports. 
    4Wait for statistic to overlap boundary values.Counters are wrapped around and start counting from zero ???
Values read at the same time for each counter in collectd and system is equal for the same ports.
 
        
11DPDK and collectd port counters values with testpmd running after ports are unbindedMediumE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2(Stop testpmd) Unbind ports from dpdk.Testpmd, collectd processes terminated?. Ports are unbinded.  
    3Bind ports to DPDK. Start testpmd, collectd. Restart traffic.Ports are binded, processes started.
Values read at the same time for each counter in collectd and system is equal for the same ports.
 
        
        
12DPDK and collectd port counters values with l2fwd (stop/start l2fwd)LowE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run l2fwd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
13DPDK and collectd port counters values with l2fwd (stop/start l3fwd)LowE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run l3fwd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
14DPDK and collectd plugins stability over nightHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. 
    2Verify traffic statistic. (Try to increment traffic rate to the maximum possible without packets loss)Values read at the same time for each counter in collectd and system is equal for the same ports. 
    3Verify traffic statistic after a night.Collectd/testpmd process are running, port counters are incremented. Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
 DPDK events tests      
TC#Tast case titlePriorityRun onStep#StepsExpected resultsComments
1DPDK/collectd link status Up/Down events configurationHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to receive DPDK events. Run testpmd, collectd.Testpmd, collectd related processes are running. 
    2Change all parameters to some others for collectd DPDK events plugin. Restart testpmd, collectd.Changes are applied to every collectd plugin parameter. List available collectd plugin parameters. 
        
2DPDK/collectd link status Up/Down eventsHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to receive DPDK events. Run testpmd, collectd.Testpmd, collectd related processes are running. 
    2Disconnect link between Traffic generator and first DPDK binded port.Port Down event is received in time for just disconnected port. All necessary information?? is present in link down event. 
    3Connect back link between Traffic generator and first DPDK binded port.Port Up event is received in time for connected back port. All necessary information?? is present in link down event. 
    4Repeat steps #2-#3 for second port.Port Down/Up events are received in time for second port. All necessary information?? is present in link down event. 
    5Disconnect link for both ports simultaneously.Port Down events are received in time for both disconnected ports. 
    6Connect link back for both ports simultaneously.Port Up events are received in time for both connected ports. 
    7Repeat test for the following drivers: igb, i40e, ixgbe, fm10k, vhost.Link status events Up/Down are received. 
        
3DPDK/collectd link status events when collectd terminatesHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to receive DPDK events. Run testpmd, collectd.Testpmd, collectd related processes are running. 
    2Break collectd running (Cntrl+C).Collectd process is terminated. 
    3Start testpmd, collectd. Cause link Down/Up events.Link Down/Up events are received. 
    4Kill collectd process.Collectd process is terminated. 
    5Start testpmd, collectd. Cause link Down/Up events.Link Down/Up events are received. 
        
4DPDK/collectd link status events when testpmd terminatesHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to receive DPDK events. Run testpmd, collectd.Testpmd, collectd related processes are running. 
    2Terminate testpmd by sending Cntrl+C.Testpmd is terminated. 
    3Start testpmd, collectd. Cause link Down/Up events.Link Down/Up events are received. 
    4Kill testpmd process.Collectd process is terminated. 
    5Start testpmd, collectd. Cause link Down/Up events.Link Down/Up events are received. 
        
5DPDK/collectd events with testpmd running after ports were unbindedMediumE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to receive DPDK events. Run testpmd, collectd.Testpmd, collectd related processes are running. 
    2Unbind ports from DPDK.Testpmd, collectd processes terminated?. Ports are unbinded.  
    3Bind ports to DPDK. Start testpmd, collectd.Ports are binded, processes started. 
    4Cause link Down/Up events.Link Down/Up events are received. 
        
6DPDK collectd events configuration testing negative '-T'/'-C'MediumE1,E2, E4,E5,E7,E8,E10,E111Change collectd plugin configuration in collectd.conf to incorrect values (parameters to be added ?). Run testpmd, collectd.Warning/error should be logged about invalid plugin parameter value. 
    2Change collectd plugin configuration in collectd.conf to correct values. Re-run testpmd, collectd.No Errors/Warning are logged. 
    3Run testpmd. Get ports statistic.Statistic is same in collectd and system. 
        
 DPDK stats and events interaction      
TC#Tast case titlePriorityRun onStep#StepsExpected resultsComments
1DPDK and collectd plugins stats/events with 2MB Huge Pages and testpmd runningHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Configure collectd.conf to receive DPDK events. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. Verify collectd/system logs for errors. 
    2Disconnect link between Traffic generator and first DPDK binded port.Port Down event is received in time for just disconnected port.
Values read at the same time for each counter in collectd and system is equal and not incremented for the same ports.
 
    3Connect link between Traffic generator and first DPDK binded port.Port Up event is received in time for just disconnected port.
Values read at the same time for each counter in collectd and system is equal and are incremented in time for the same ports.
 
    4Repeat steps #2-#3 for second binded port.Same as for #2-#3 
        
2DPDK and collectd plugins stats/events with 1GB Huge Pages and testpmd runningHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Configure collectd.conf to receive DPDK events. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. Verify collectd/system logs for errors. 
    2Disconnect link between Traffic generator and first DPDK binded port.Port Down event is received in time for just disconnected port.
Values read at the same time for each counter in collectd and system is equal and not incremented for the same ports.
 
    3Connect link between Traffic generator and first DPDK binded port.Port Up event is received in time for just disconnected port.
Values read at the same time for each counter in collectd and system is equal and are incremented in time for the same ports.
 
    4Repeat steps #2-#3 for second binded port.Same as for #2-#3 
        
3DPDK and collectd plugins stats/events for different driversHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Configure collectd.conf to receive DPDK events. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. Verify collectd/system logs for errors. 
    2Disconnect link between Traffic generator and first DPDK binded port.Port Down event is received in time for just disconnected port.
Values read at the same time for each counter in collectd and system is equal and not incremented for the same ports.
 
    3Connect link between Traffic generator and first DPDK binded port.Port Up event is received in time for just disconnected port.
Values read at the same time for each counter in collectd and system is equal and are incremented in time for the same ports.
 
    4Repeat test for the following drivers: igb, i40e, ixgbe, fm10k, vhost.Events/stats are working correctly for each driver. 
        
4DPDK and collectd plugins stats/events upon stats plugin failureHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Configure collectd.conf to receive DPDK events. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. Verify collectd/system logs for errors. 
    2Cause failure to collectd stats plugin (How to?)Errors logged about stats plugin failure. 
    3Disconnect link between Traffic generator and first DPDK binded port.Port Down event is received in time for just disconnected port.  
    4Connect link between Traffic generator and first DPDK binded port.Port Up event is received in time for just disconnected port. 
    5Start DPDK stats collectd plugin. Repeat steps #3-#4.Ports events are received. Port counters are not updated upon disconnect. Port counters are updated upon reconnect. 
        
5DPDK and collectd plugins stats/events upon events plugin failureHighE1,E2, E4,E5,E7,E8,E10,E111Configure collectd.conf to collect DPDK stats. Configure collectd.conf to receive DPDK events. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each counter in collectd and system is equal for the same ports. Verify collectd/system logs for errors. 
    2Cause failure to collectd events plugin (How to?)Errors logged about stats plugin failure. 
    3Disconnect link between Traffic generator and first DPDK binded port.No port Down event is received. Values read at the same time for each counter in collectd and system is equal and not incremented for the same ports. 
    4Connect link between Traffic generator and first DPDK binded port.No port Up event is received. Values read at the same time for each counter in collectd and system is equal and are incremented in time for the same ports. 
    5Start DPDK events collectd plugin. Repeat steps #3-#4.Ports events are received. Port counters are not updated upon disconnect. Port counters are updated upon reconnect. 
        
 Huge Pages      
TC#Tast case titlePriorityRun onStep#StepsExpected resultsComments
1DPDK Huge Pages plugin configuration testing negative '-T'/'-C'MediumE1,E2, E4,E5,E7,E8,E10,E111Change collectd plugin configuration in collectd.conf to incorrect values. Run testpmd, collectd.Warning/error should be logged about invalid plugin parameter value. 
    2Change collectd plugin configuration in collectd.conf to correct values. Re-run testpmd, collectd.No Errors/Warning are logged. 
    3Run testpmd. Get ports statistic.Statistic is same in collectd and system. 
        
2Huge Pages plugin statistic of 2MB compare to system with testpmd running with different driversHighE1-E121Configure collectd.conf to collect Huge Pages statistic. Configure and mount 14388? 2MB pages. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each Huge Page parameter is the same. Should be tested: vmpage_number, memory and percent values. 
    2Perform values comparison for the following drivers: igb, i40e, ixgbe, fm10k, vhost.Values read at the same time for each Huge Page parameter is the same. 
        
3Huge Pages plugin statistic of 1GB compare to system with testpmd running with different driversHighE1-E121Configure collectd.conf to collect Huge Pages statistic. Configure and mount 28? 1GB pages. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each Huge Page parameter is the same. Should be tested: vmpage_number, memory and percent values. 
    2Perform values comparison for the following drivers: igb, i40e, ixgbe, fm10k, vhost.Values read at the same time for each Huge Page parameter is the same. 
        
        
4Huge Pages plugin statistic of 2MB compare to system with testpmd running over nightHighE1-E121Configure collectd.conf to collect Huge Pages statistic. Configure and mount 14388? 2MB pages. Run testpmd, collectd. Start bidirectional traffic.Values read at the same time for each Huge Page parameter is the same. 
    2Verify traffic statistic. (Try to increment traffic rate to the maximum possible without packets loss)Values read at the same time for each counter in collectd and system is equal for the same ports. 
    3Verify traffic statistic after a night.Collectd/testpmd process are running, port counters are incremented. Values read at the same time for each counter in collectd and system is equal for the same ports. 
        
5Huge Pages plugin statistic of 1GB compare to system with testpmd running and additional memory loadMediumE1-E121Configure collectd.conf to collect Huge Pages statistic. Run testpmd, collectd.Configuration successful. 
    2Run test application to consume more memory?Values read at the same time for each Huge Page parameter is the same. 
        
 DPDK Keep Alive tests      
TC#Tast case titlePriorityRun onStep#StepsExpected resultsComments
1To be defined per requirements.HighE2,E5,E8,E111To add laterTo add later 
  after DPDK16.07     
        
        
        
  • No labels