...
The colored blocks shows the current Features of VSPERF, the White-blocks shows the features we plan to add to VSPERF, in near future.
Significance of VSPERF
VSPERF Only
VSPERF can be used for both Lab and Production/Live Environments, as summarized in the figure below. Typically, in cloud deployments, Telcos define set of "Acceptance Tests" (AT) for cloud. Performance testcases can be part of this AT, and VSPERF is the ideal tool to be used to run those testcases.
...
Continued Contribution to Standardization
ETSI TST 008, 009, 009 and VSPERF:
Section | Compliance Comments | Missing Aspects |
Test Setups | Fully Compliant | |
Throughput | Missing capacity reporting. Only throughput is reported. | Capacity with X % Loss |
Latency | Missing Transfer time in percentile. Other 4 supported. | Transfer Time (Yth percentile, X % Thpt, Filter) |
Delay Variation | Missing FDV and IFDV with %le and %ge throughput | FDV (Yth percentile, X % Thpt, Filter) and Inter-Frame Delay Variation (X % Thpt, Filter) |
Loss | Missing loss ratio with %ge throughput and loss-free seconds. | Loss Ratio (X % Thpt, Filter), Loss-Free Seconds (X % Thpt, Filter) |
Methods of Measurement | Pre-test requirements automated. Binary Search with LV needs minor update? NDR/PDR support is lacking. Setting up Cross-Numa scenarios automated. Better configuration for Soak-Tests | Binary Search with NDR/PDR is missing |
IETF
GSMA
Container Networking Performance Benchmarking
...
Part-1: Containerizing VSPERF.
Approach-1:
This approach is currently under exploration by Unknown User (shailesh7333)
Approach-2:
This approach is currently implemented - and can be accesses from this link: https://github.com/opensource-tnbt/vsperf-docker
Controller Container: Separating Deployment from Testing
| No. | Title | Description | Comments |
|---|---|---|---|
| 1 | Auto-Deploy | This container takes the following inputs and automatically deploy VSPERF and setup the TGen if required. | Ideal for integrating with CI/CD based larger test automation Frameworks |
| 2 | Interactive-Deploy | This container, starts listening to commands from the user to perform the deployment of VSPERF and TGen | Ideal for integrating with other 'interactive' (web-based) test automation frameworks. |
| 3 | Auto-TestControl | This Container takes the following inputs and automatically runs the tests and publishes the results. | Ideal for integrating with CI/CD based larger test automation frameworks. |
| 4 | Interactive-TestControl | This container, starts listening to commands from the user to perform the testing. | Ideal for integrating with other 'interactive' (web-based) test automation frameworks. |
Part-2: Integrating VSPERF with larger Test-Automation Frameworks
We are targeting both Dovetail and X-testing.
Analytics
Basic Analytics
Reference work, which was demonstrated at ONS-NA 2019 can be accessed here: https://github.com/opensource-tnbt/vsperf-notebooks
...