Jira Link: DB-5141
testpitrycqlconsistentrestoredml-k8s-rf3 is failing for 2.14. It started failing from 184.108.40.206-b11(it did pass in 220.127.116.11-b7 thus is a regression) and has been failing in every re-run. Note that AWS counterpart is working as expected.
1. Create a Universe with consistent restore enabled 2. Create a keyspace 3. Create a snapshot schedule 4. Note down absolute time 5. Create a table, note down absolute time 6. Perform the following DML operations repeatedly: i. Insert a new row ii. Update an existing row iii.Delete an existing row To perform this step we create a thread dml_operation_thread. 7. Start dml_operation_thread 8. Do following operations for '10' times: i. Wait for 10 seconds and collect intermediate absolute time. ii. Wait for 5 seconds and restore to collected absolute time in #8 iii. Validate successful restoration iv. Validate that expected number of inconsistencies in data is <=1. 9. Restore to empty table state using collected absolute time in #5 10. Validate successful restoration 11. Sleep for a minute. 12. Validate that expected number of inconsistencies in data is 0. 13. Restore to no table state using collected absolute time in #4 14. Wait for restore to be completed & validated 15. Wait for dml_operation_thread to complete. 16. Validate that the table doesn't exist. 17. Destroy Universe
Test is failing at 3rd iteration of step 8.iv as there are >3 jumps found in data after 3 PITRs
Noticed this failure intermittently for 2.17.1 also