This page contains an analyzis on the list of test cases listed in the CNCF CNF Testsuite to determine if RA2 should contain related workload requirements.
| Test | Note | Verdict |
|---|---|---|
To test the increasing and decreasing of capacity | Do we request horizontal scaling from all CNF-s? | |
Test if the Helm chart is published | At the moment RA2 does not mandate the usage of Helm. We should first decide on CNF packaging. RA2 can stay neutral, follow the O-RAN/ONAP ASD path or propose own solution. | |
Test if the Helm chart is valid | At the moment RA2 does not mandate the usage of Helm. | |
Test if the Helm deploys | At the moment RA2 does not mandate the usage of Helm. This should be more generic, like testing if the CNF deploys. | |
Test if the install script uses Helm v3 | At the moment RA2 does not mandate the usage of Helm. | |
To test if the CNF can perform a rolling update | Needed | |
To check if a CNF version can be downgraded through a rolling_version_change | It is not clear what is the difference between a rolling upgrade and a rolling version change. | |
To check if a CNF version can be downgraded through a rolling_downgrade | Needed | |
To check if a CNF version can be rolled back rollback | It is not clear what is the difference between a rolling downgrade and a rolled back rollback. | |
To check if the CNF is compatible with different CNIs | ||
(PoC) To check if a CNF uses Kubernetes alpha APIs | ||
To check if the CNF has a reasonable image size | ||
To check if the CNF have a reasonable startup time | ||
To check if the CNF has multiple process types within one container | ||
To check if the CNF exposes any of its containers as a service | ||
To check if the CNF has multiple microservices that share a database | ||
Test if the CNF crashes when node drain and rescheduling occurs. All configuration should be stateless | ||
To test if the CNF uses a volume host path | ||
To test if the CNF uses local storage | ||
To test if the CNF uses elastic volumes | ||
To test if the CNF uses a database with either statefulsets, elastic volumes, or both | ||
Test if the CNF crashes when network latency occurs | ||
Test if the CNF crashes when disk fill occurs | ||
Test if the CNF crashes when pod delete occurs | ||
Test if the CNF crashes when pod memory hog occurs | ||
Test if the CNF crashes when pod io stress occurs | ||
Test if the CNF crashes when pod network corruption occurs | ||
Test if the CNF crashes when pod network duplication occurs | ||
To test if there is a liveness entry in the Helm chart | ||
To test if there is a readiness entry in the Helm chart | ||
To check if logs are being sent to stdout/stderr | ||
To check if prometheus is installed and configured for the cnf | ||
To check if logs and data are being routed through fluentd | ||
To check if Open Metrics is being used and or compatible. | ||
To check if tracing is being used with Jaeger | ||
To check if a CNF is using container socket mounts | ||
To check if containers are using any tiller images | ||
To check if any containers are running in privileged mode | ||
To check if a CNF is running services with external IP's | ||
To check if any containers are running as a root user | ||
To check if any containers allow for privilege escalation | ||
To check if an attacker can use a symlink for arbitrary host file system access | ||
To check if there are service accounts that are automatically mapped | ||
To check if there is a host network attached to a pod | ||
To check if there are service accounts that are automatically mapped | ||
To check if there is an ingress and egress policy defined | ||
To check if there are any privileged containers | ||
To check for insecure capabilities | ||
To check for dangerous capabilities | ||
To check if namespaces have network policies defined | ||
To check if containers are running with non-root user with non-root membership | ||
To check if containers are running with hostPID or hostIPC privileges | ||
To check if security services are being used to harden containers | ||
To check if containers have resource limits defined | ||
To check if containers have immutable file systems | ||
To check if containers have hostPath mounts | ||
To check if containers are using labels | ||
To test if there are versioned tags on all images using OPA Gatekeeper | ||
To test if there are any (non-declarative) hardcoded IP addresses or subnet masks | ||
To test if there are node ports used in the service configuration | ||
To test if there are host ports used in the service configuration | ||
To test if there are any (non-declarative) hardcoded IP addresses or subnet masks in the K8s runtime configuration | ||
To check if a CNF version uses immutable configmaps | ||
Test if the CNF crashes when pod dns error occurs | ||