Vsperf Multi-Flow testing
Introduction
The purpose of this series of testing is to examine the impact of sending multiple streams thought the OVS bridge on the flow throughput and performance. Testing 2544 zero loss on both PVP and Phy2Phy2 modes.
Test Goals:
To document any disparity in flow processing between Layer 2,Layer 3 and Layer 4 for flows ranging between 1,000 through 1 million. Â
To determine any any significant limitations or anomalies in processing in the handling of flows based on ranges that would affect performance when deployed in specific situations. Or when using in qualifications labs. Â Â
To characterize the Multiflow performance for VSPERF that can be used to compare for future benchmarking.
Test Non-Goals:
This is not a maximum performance testing setup. The system has been tuned to provide it’s best performance to ensure consistent results for this test. These results should not be used as a  set of limitations for the software as different hardware configurations will be able to give different results.
These results are are to be used to compare within this dataset to each other.
This testing does not reflect performance limitations of the traffic generator used in this case. Other tools will have different abilities. One generate was used for consistent results.
Test Configuration:
Figure 1: Test Setup
System Configuration
Isolated all NUMA 1 CPU’s using tuned cpu parameters.
DPDK_SOCKET_MEM = ['4096', '4096']
VSWITCH_VHOST_CPU_MAP = [5,7,13,15]
VSWITCH_PMD_CPU_MASK = 'a0a0'
GUEST_SMP = ['3']
GUEST_CORE_BINDING = [('1', '9', '11')]
GUEST_TESTPMD_PARAMS = ['-l 0,1,2 -n 1 --socket-mem 2048 -- '
                       '--burst=64 -i --txqflags=0xf00 '
                       '--disable-hw-vlan --nb-cores=2, --txq=1 --rxq=1 --rxd=1024 --txd=1024']
Data Sets:
Test Data was determined by using the max number of each of the 5 iterations. This was to rule out one aberrations in the zero loss test that causes lower numbers infrequalty but impacted a flow here and there. 5 iterations were used to determine we had a level of confidence in the test numbers reported.
Test Cases:
Test Cases Performed | |||||
Run | Test Name | Description | Test Condition | Frame Size | Flow Counts |
1-5 | Baseline-001 | Establish baseline metrics using 1 flow for Phy2Phy2_TPUT | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1 |
6-10 | Baseline-002 | Establish baseline metrics using 1 flow for PVP_TPUT | PVP_TPUT | 64,128,256,512,1024,1500 | 1 |
11-15 | Multi-Src-001 | Run 2544 Zero Loss Test using modifying L2 Source Address only | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
16-20 | Multi-Src-002 | Run 2544 Zero Loss Test using modifying L2 Source Address only | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
21-25 | Multi-Src-003 | Run 2544 Zero Loss Test using modifying L3 Source Address only | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
26-30 | Multi-Src-004 | Run 2544 Zero Loss Test using modifying L3 Source Address only | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
31-35 | Multi-Src-005 | Run 2544 Zero Loss Test using modifying L4 Socket Source Address only | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 64,000 |
36-40 | Multi-Src-006 | Run 2544 Zero Loss Test using modifying L4 Socket Source Address only | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 64,000 |
41-45 | Multi-Dst-007 | Run 2544 Zero Loss Test using modifying L2 Dest Address only | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
46-50 | Multi-Dst-008 | Run 2544 Zero Loss Test using modifying L2 Dest Address only | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
51-55 | Multi-Dst-009 | Run 2544 Zero Loss Test using modifying L3 DestAddress only | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
56-60 | Multi-Dst-010 | Run 2544 Zero Loss Test using modifying L3 Dest Address only | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
61-65 | Multi-Dst-011 | Run 2544 Zero Loss Test using modifying L4 Dest Socket Address only | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 64,000 |
66-70 | Multi-Dst-012 | Run 2544 Zero Loss Test using modifying L4 Dest Socket Address only | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 64,000 |
71-75 | Multi-Both-013 | Run 2544 Zero Loss Test using modifying L2 Both src and Dest | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
76-80 | Multi-Both-014 | Run 2544 Zero Loss Test using modifying L2 Both src and Dest | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
81-85 | Multi-Both-015 | Run 2544 Zero Loss Test using modifying L3 Both src and Dest | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
86-90 | Multi-Both-016 | Run 2544 Zero Loss Test using modifying L3 Both src and Dest | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 1,000,000 |
91-95 | Multi-Both-017 | Run 2544 Zero Loss Test using modifying L4 Both src and Dest | Phy2Phy_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 64,000 |
96-100 | Multi-Both-018 | Run 2544 Zero Loss Test using modifying L4 Both src and Dest | PVP_TPUT | 64,128,256,512,1024,1500 | 1,000 10,000 64,000 |
Analysis of Test Results:
Test Results
Baseline Â
Figure 2: 1 Flow Phy2Phy RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 3: 1 Flow PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Modify Both Source and Destination Address
Figure 4: Layer 2 Src & Des PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 5: Layer 3 Src & Des PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 6: Layer 4 Src & Des PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 7: Layer 2 Src & Dst Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 8: Layer 3 Src & Dst Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 9: Layer 4 Src & Dst Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Flows Rotating Source address only
Figure 10: Layer 2 Src only PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 11: Layer 3 Src only PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 12: Layer 4 Src only PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 13: Layer 2 Src only Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 14: Layer 3 Src only Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 15: Layer 4 Src only Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Flows Rotating Destination address only
Figure 16: Layer 2 Dest only PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 17: Layer 3 Dest only PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 18: Layer 4 Dest only PVP_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 19: Layer 2 Dest only Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 20: Layer 3 Dest only Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Figure 21: Layer 4 Dest only Phy2Phy_TPUT RFC 2544 Zero Loss. Verify Duration 20 Minutes
Flow Cross Analysis
PVP_TPUT Cross Flow Analysis
Rx fps PVP_TPUT L2 1,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 5,706,159 | 5,618,036 | 5,181,619 | 4,574,736 | 1,636,512 | L2 | PVP_TPUT |
Rotate_SRC | 1,000 | 5,706,160 | 5,592,918 | 5,181,618 | 4,624,379 | 1,636,512 | L2 | PVP_TPUT |
Rotate_DST | 1,000 | 5,286,931 | 5,558,547 | 5,213,515 | 4,607,830 | 1,636,512 | L2 | PVP_TPUT |
Rx fps PVP_TPUT L2 10,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 10,000 | 5,286,930 | 5,677,521 | 5,149,722 | 4,591,304 | 1,636,512 | L2 | PVP_TPUT |
Rotate_SRC | 10,000 | 5,033,066 | 5,558,550 | 5,213,517 | 4,574,736 | 1,636,512 | L2 | PVP_TPUT |
Rotate_DST | 10,000 | 1,849,888 | 4,528,837 | 4,990,231 | 4,475,443 | 1,636,512 | L2 | PVP_TPUT |
Rx fps PVP_TPUT L2 1,000,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000,000 | 5,182,123 | 5,439,577 | 5,022,131 | 4,525,090 | 1,636,512 | L2 | PVP_TPUT |
Rotate_SRC | 1,000,000 | 5,810,968 | 5,380,094 | 4,990,232 | 4,458,897 | 1,636,512 | L2 | PVP_TPUT |
Rotate_DST | 1,000,000 | 6,020,580 | 5,737,004 | 5,245,413 | 4,425,798 | 1,636,512 | L2 | PVP_TPUT |
Rx fps PVP_TPUT L3 1,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 7,697,494 | 7,105,165 | 6,361,837 | 4,683,586 | 1,636,512 | L3 | PVP_TPUT |
Rotate_SRC | 1,000 | 8,011,913 | 7,343,100 | 6,457,529 | 4,683,587 | 1,636,512 | L3 | PVP_TPUT |
Rotate_DST | 1,000 | 4,553,281 | 4,428,330 | 4,160,887 | 3,747,308 | 1,636,512 | L3 | PVP_TPUT |
Rx fps PVP_TPUT L3 10,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 7,697,494 | 7,105,165 | 6,361,837 | 4,683,586 | 1,636,512 | L3 | PVP_TPUT |
Rotate_SRC | 1,000 | 8,011,913 | 7,343,100 | 6,457,529 | 4,683,587 | 1,636,512 | L3 | PVP_TPUT |
Rotate_DST | 1,000 | 5,182,125 | 5,023,182 | 4,735,049 | 4,161,022 | 1,636,512 | L4 | PVP_TPUT |
Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test | Test |
Rx fps PVP L3 1,000,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 10,000 | 6,859,037 | 6,153,395 | 5,691,980 | 4,683,587 | 1,636,512 | L3 | PVP_TPUT |
Rotate_SRC | 10,000 | 6,754,230 | 6,272,369 | 5,660,085 | 4,683,587 | 1,636,512 | L3 | PVP_TPUT |
Rotate_DST | 10,000 | 4,972,508 | 4,963,703 | 4,671,252 | 4,061,729 | 1,636,512 | L3 | PVP_TPUT |
Rx fps PVP L4 1,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 5,182,125 | 5,023,182 | 4,735,049 | 4,161,022 | 1,636,512 | L4 | PVP_TPUT |
Rotate_SRC | 1,000 | 7,907,108 | 7,105,161 | 6,393,736 | 4,683,587 | 1,636,512 | L4 | PVP_TPUT |
Rotate_DST | 1,000 | 5,182,125 | 5,023,182 | 4,735,049 | 4,161,022 | 1,636,512 | L4 | PVP_TPUT |
Rx fps PVP L4 65,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 10,000 | 4,972,510 | 4,606,788 | 4,256,582 | 3,846,602 | 1,636,512 | L4 | PVP_TPUT |
Rotate_SRC | 10,000 | 6,439,812 | 5,796,489 | 5,564,392 | 4,657,478 | 1,636,512 | L4 | PVP_TPUT |
Rotate_DST | 10,000 | 4,972,510 | 4,606,788 | 4,256,582 | 3,846,602 | 1,636,512 | L4 | PVP_TPUT |
Rx fps PVP_TPUT L4 64,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 65,000 | 5,182,125 | 5,023,182 | 4,607,458 | 4,028,634 | 1,636,512 | L4 | PVP_TPUT |
Rotate_SRC | 65,000 | 5,182,124 | 5,023,182 | 4,607,457 | 4,028,633 | 1,636,512 | L4 | PVP_TPUT |
Rotate_DST | 65,000 | 5,182,125 | 5,023,182 | 4,607,458 | 4,028,634 | 1,636,512 | L4 | PVP_TPUT |
Phy2Phy_TPUT Cross Flow Analysis
Rx fps Phy2Phy_TPUT L2 1,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 12,833,037 | 14,897,700 | 9,041,256 | 4,683,588 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rotate_SRC | 1,000 | 12,204,196 | 14,897,706 | 9,041,252 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rotate_DST | 1,000 | 12,204,196 | 14,897,706 | 9,041,252 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rx fps Phy2Phy_TPUT L2 10,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 10,000 | 12,937,839 | 14,897,714 | 9,041,247 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rotate_SRC | 10,000 | 12,204,195 | 14,897,705 | 9,041,250 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rotate_DST | 10,000 | 12,204,195 | 14,897,705 | 9,041,250 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rx fps Phy2Phy_TPUT L2 1,000,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000,000 | 14,824,362 | 14,957,195 | 9,041,252 | 4,683,588 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rotate_SRC | 1,000,000 | 7,487,881 | 8,592,289 | 8,626,578 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rotate_DST | 1,000,000 | 7,487,881 | 8,592,289 | 8,626,578 | 4,683,587 | 1,636,512 | L2 | Phy2Phy_TPUT |
Rx fps Phy2Phy_TPUT L3 1,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 23,628,158 | 16,716,624 | 9,041,251 | 4,683,588 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rotate_SRC | 1,000 | 23,523,356 | 16,533,085 | 9,041,248 | 4,683,585 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rotate_DST | 1,000 | 23,628,142 | 16,741,734 | 9,041,252 | 4,683,584 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rx fps Phy2Phy_TPUT L3 10,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 10,000 | 17,549,343 | 16,860,704 | 9,041,252 | 4,683,586 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rotate_SRC | 10,000 | 10,317,666 | 10,317,354 | 9,041,249 | 4,683,586 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rotate_DST | 10,000 | 10,946,502 | 10,946,570 | 9,041,246 | 4,683,584 | 1,636,512 | L3 | Phy2Phy_TPUT |
Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test | Test |
Rx fps Phy2Phy L3 1,000,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000,000 | 12,833,030 | 12,815,721 | 9,041,245 | 4,683,586 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rotate_SRC | 1,000,000 | 12,728,228 | 12,731,122 | 9,041,249 | 4,683,585 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rotate_DST | 1,000,000 | 12,728,225 | 12,756,236 | 9,041,247 | 4,683,584 | 1,636,512 | L3 | Phy2Phy_TPUT |
Rx fps Phy2Phy L4 1,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 1,000 | 23,523,355 | 16,860,711 | 9,041,250 | 4,683,586 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rotate_SRC | 1,000 | 15,243,599 | 15,314,099 | 9,041,253 | 4,683,587 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rotate_DST | 1,000 | 23,523,355 | 16,860,711 | 9,041,250 | 4,683,586 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rx fps Phy2Phy L4 65,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 10,000 | 19,540,676 | 16,801,222 | 9,041,250 | 4,683,587 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rotate_SRC | 10,000 | 19,435,885 | 16,801,230 | 9,041,253 | 4,683,587 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rotate_DST | 10,000 | 19,540,676 | 16,801,222 | 9,041,250 | 4,683,587 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rx fps Phy2Phy_TPUT L4 64,000 Flows | ||||||||
Test | Flows | 64 | 128 | 256 | 512 | 1500 | Layer | Test |
Rotate_Both | 65,000 | 13,461,878 | 13,410,574 | 9,041,247 | 4,683,585 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rotate_SRC | 65,000 | 12,518,611 | 12,552,666 | 9,041,249 | 4,683,586 | 1,636,512 | L4 | Phy2Phy_TPUT |
Rotate_DST | 65,000 | 13,461,878 | 13,410,574 | 9,041,247 | 4,683,585 | 1,636,512 | L4 | Phy2Phy_TPUT |