I noticed that we are currently always waiting for the same amount of time during leader election (see here)
let mut tick_interval = tokio::time::interval(Duration::from_secs(lease_time / 2));
loop {
tick_interval.tick().await;
Maybe it makes sense to randomise this a bit and give different instances a slightly different tick_interval. This way we would space out requests a bit.
CC @yezizp2012
No response
No response
Raft also uses randomised timeouts during leader election to avoid collisions
@CAJan93 whats the minimum value lease_time can have? and can you tell me how to test that particular file?
Well, I think it's not that necessary to do this, because: 1. eventually we will replace it with APIs of etcd lib or (some big refactoring); 2. even in current implementation, etcd will guarantee only one node will register successfully, it will ensure sequential writing of each requests.
I agree, it is not needed, but it would still reduce the likelihood of simultaneous election-runs that may then be repeated. Therefore I introduced this randomisation in https://github.com/risingwavelabs/risingwave/pull/6466 (it really is just very few lines of code)
