stabilize e2e test case sandbox-basic #962
Conversation
LiboYu2
requested review from
roypaulin,
cchen-vertica,
fenic-fawkes,
HaoYang0000 and
qindotguan
as code owners
LiboYu2
changed the title
| @@ -13,6 +13,7 @@ | |||
|
|
|||
| apiVersion: kuttl.dev/v1beta1 | |||
| kind: TestSuite | |||
| kindNodeCache: true | |||
There was a problem hiding this comment.
This allows the downloaded images to be cached on the node. This will speed up test case execution.
| @@ -15,3 +15,4 @@ apiVersion: kuttl.dev/v1beta1 | |||
| kind: TestStep | |||
| commands: | |||
| - command: bash -c "../../../scripts/wait-for-verticadb-steady-state.sh -n verticadb-operator -t 360 $NAMESPACE" | |||
| - command: sleep 120 | |||
There was a problem hiding this comment.
Add this latency to stabilize the cluster before step 60 starts.
There was a problem hiding this comment.
You don't need to add this sleep. It will take the test longer to complete. If this frequently fails due to insufficient timeout, then just increase the timeout in the script call above.
There was a problem hiding this comment.
This is different from timeout. The timeout is the maximum wait time for the whole test step to finish. This sleep call gives the cluster some time to stabilize its state before the next step starts. It makes the test run longer but it makes the test pass.
There was a problem hiding this comment.
The script on line 17 waits for the operator to be steady(gives the cluster some time to stabilize its state before the next step) meaning there is no error and nothing going on. There is no benefit in adding another wait after. If this step fails and more time is needed, you should increase the time passed as argument to the script.
There was a problem hiding this comment.
This is from that wait-for-verticadb-steady-state.sh
timeout $TIMEOUT bash -c -- "while ! $LOG_CMD | \ grep $WEBHOOK_FILTER | \ grep $DEPRECATION_FILTER | \ grep $VDB_FILTER | \ tail -1 | grep --quiet '\"result\": {\"Requeue\":false,\"RequeueAfter\":0}, \"err\": null'; do sleep 1; done" & pid=$! wait $pid
This is from "man timeout":
timeout - run a command with a time limit
If the script runs longer than the $TIMEOUT, an error will be reported.
If the script finishes within the $TIMEOUT, the next step starts right away.
What I want to achieve is to add a latency between the two steps to make sure the latter
step is not impacted by the previous step. Increasing the timeout will not achieve that.
|
As discussed, we should use an environment variable for all extended timeout values (900), and try to print all the events on vdb. |
What do you mean by "printing all the events on vdb" |
1 removed 2 min sleep added previously 2 increased low disk space to avoid low disk volume event 3 bumped the spam filter threshold from 25 to 100 to make sure kuttl framework will receive all the events
Increased the timeout for steps that frequently fail.
Add some latency to stabilize the cluster before moving to the unsandboxing step.
I ran the test cases locally for 5 times in a roll and they all passed.