...
- The number of VMs in the setup is a parameter for Throughput, Latency and Delay Variation.
- The number of VMs in series on each parallel path through the DUT (PVVP means 2, see Figure 6.2-1 of the present document).
- The number of simultaneous L2/L3/L4 flows is a parameter of Throughput, Latency, and Delay Variation benchmark testing. This implies a fourth dimension. Good to clarify this in the present doc.
- All of the Test setups in clause 6.2 employ 2 physical NICs. Are there any single NIC or single vNIC VNFs envisioned (for “telco” deployment) that are critical/need to be tested immediately?
- For the VMs themselves, should the testing employ reference VNFs/VNF-Cs with specific workload profiles, as described in ETSI NFV TST001, Clause 6? Table 6.2 has VNF to workload mapping, See table 6.3 below:
- ETSI NFV INF010 recommended several metrics for the Infrastructure “service quality” provided to the VNFs instantiated there. This was an early ETSI NFV Group Specification (December 2014), and it summarizes the service quality metrics envisioned at that time in Table 1. One of the key metrics is VM Stalls, with event Frequency and Duration, which would be characterized using the methods described in clause 12.4 (Long Duration Testing) in the present document, but there is no(?) current CNTT RM requirement that matches this metric. Both ETSI NFV GS INF010 and GS TST001 envision metrics to capture this area of infrastructure behaviour.
- Power Efficiency appears in CNTT RM Table 4-10. There was IETF Benchmarking Methodology Working Group work on this topic, but it did not advance beyond the proposal stage: https://tools.ietf.org/html/draft-manral-bmwg-power-usage-04
- what else?