This wiki page is based on an email sent to the Anuket TSC on Sept 6, 2021.
Introduction
During the Lakelse release, some issues with RC became apparent.
...
- M1 requires a validation plan for RC. Instead of providing a validation plan, I was provided a pointer to the RC documentation. The result of this is that the community has no documented description of how RC is being validated.
- Based on feedback from the meeting, the validation testing does not include insertion of faults to ensure that test cases fail when they should. This may result in false negative tests.
- The validation testing seems to be conducted in just a single configuration/environment. This may result in configuration/optimization around a single environment and in turn, inaccurate test results in a valid, but different environment.
- There is no dashboard or other convenient means to see the results of validation testing. Also, there are no periodic reports or other updates on the results.
Conclusions
- If we intend for RC to truly be dependent on RA, then this needs to be expressed through the mapping that is described in RC's own documentation. In addition, we should work toward improving validation testing.
- If we believe that a loose coupling of RC to RA is sufficient, then we should stop stating that RC has a dependency on RA. We should also update RC documentation to remove the several references to detailed mapping of RC to RA.