And do so centrally, not randomly in individual tests. (Hopefully) fixes spurious test failures in CI. Related: #5867