1. 24 Mar, 2022 1 commit
  2. 21 Mar, 2022 1 commit
  3. 07 Feb, 2022 1 commit
  4. 03 Nov, 2021 1 commit
    • Shaya Potter's avatar
      set next_idx on each succesful raft_send_appendentries call (#65) · 02bdf818
      Shaya Potter authored
      this is to avoid sending duplicate entries each time raft_send_appendentries is called, just because they haven't been acked yet.
      
      it will still be reset backwards upon the leader receiving a raft_recv_appendentries_response() with a failure mode
      
      + a test for next_idx on append entry, which changes a number of tests which checked for this
      
      in order to do this, I had to modify the existing mock callback to return 0 instead of 1.  this needs to be verified
      
      + fixes for raft tests that this change exposed
      
      1) when we issues a sequence of entries, each was given an increasing term, so had entries whose term was higher than leaders term
      
      so had to fix that
      
      2) by setting the term of appendentries_repsonse to not be equal to leaders term (because of fix above), we now fail on receiving them
      
      do all that by not allowing raft_append_entry to append an entry that break raft semantics (i.e. term of entry > log term), but this breaks many tests that don't set term, so set them
      02bdf818
  5. 07 Oct, 2021 1 commit
    • Shaya Potter's avatar
      redo/tweak transfer leader in libraft (#61) · 63b0b1c1
      Shaya Potter authored
      this started off as a small tweak and grew a bit
      
      1) new callback for transfer leader result, along with its own enum of states
      
      2) change/remove raft_reset_transfer_leader() calls
      
      only call in places where there's a new leader or timeout (i.e. failed to transfer within period).
      
      so, we only reset in 3 places 1) recv_appendentries 2) raft_periodic (timeout) 3) raft_become_leader
      
      2a) raft_become_leader also got refactored a little to only issue the normal cb.notify_state_event() callback after it has set its state to be leader (as before, it could fail after the callback was issued)
      
      2b) in raft_periodic the raft_reset_transfer_leader() is pulled out the "if LEADER" block, as will lose leader when a vote is issued to it, we still want it to timeout if leadership isn't gained
      
      3) raft_reset_transfer_leader() does the logic for determining success/failure now, but becaue timeout is its own result which can't be determined by just looking at leader/desired, add a new flag to it to note timeout state
      
      4) leader stickiness in requestvote is only tested for prevote (as in a timeout now case, we want to drop the timeout now flag after prevotes are sent).  The idea is that we will only get to actual election if prevote passes.
      
      4) a bunch of new tests
      
      * fix issue with timeout flag to reset_transfer_leader
      
      also fix test to catch the original error
      
      * libraft/redisraft changes needed for e2e leader transfer
      
      * make leader stickiness only for prevote
      63b0b1c1
  6. 23 Sep, 2021 2 commits
    • Shaya Potter's avatar
      Adding support for TimeoutNow RPC (#7) · 575335c3
      Shaya Potter authored
      this is the small modifications needed for raftlib to support timeout now rpc
      
      1) add a flag to requestvote type to tell other nodes its overriding normal chec
      2) new send_timeoutnow callback to send a timeout now rpc at right time
      3) call said callback is we've marked the node as timeout now and in raft_appendentries_response raft_get_current_idx(me_) == r->current_idx (i.e. response says the node's idx is up to date with leader)
      4) add helper functions to set/get/reset timeout flag on node structs for targeted node.
      
      how it would be used by client
      
      1) client of libraft will call raft_transfer_leader(raft_server_t* me_, raft_node_id_t node_id) to target the node they want to transfer to.
      
      2) client of libraft will provide the send_timeoutnow() callback for the actual sending of the rpc
      
      3) client of libraft will modify their existing notify_state_event() callback to observe the result of the timeout now operation.
      
      if it receives a RAFT_STATE_LEADERSHIP_TRANSFER_FAILED, then the transfer failed
      
      if it received a RAFT_STATE_FOLLOWER, then it inspects for the actual leader to see if its the expected one and can return that it was transferred to expected node or not
      
      if it received a RAFT_STATE_CANDIDATE, then the transfer also failed, as the targeted node sent a request vote which removed leadership, but wasn't able to win the election.
      
      With PreVote, this latter case should be very rare.
      575335c3
    • Ozan Tezcan's avatar
      Define separate callback function for node id retrieval (#55) · ce8b0f02
      Ozan Tezcan authored
      Define separate callback function for node id retrieval
      ce8b0f02
  7. 15 Sep, 2021 1 commit
  8. 14 Sep, 2021 2 commits
  9. 09 Sep, 2021 1 commit
  10. 01 Sep, 2021 1 commit
    • Shaya Potter's avatar
      include the prev_log_idx in the appendentry response (#45) · 64747dee
      Shaya Potter authored
      the problem we have is that 2 appendentries can be in process in parallel
      
      Howver, the response handler doesn't expect this, it will reset the next_idx in both, but currently resets next_idx based on next_idx's value, not the old value that was the basis of prev_log_idx.
      
      Solution: return prev_log_idx in the response
      
      Problem with solution: prev_log_idx is not always based on next_idx from node, but can be based on a snapshot idx if it no longer exists. I haven't coded this yet, but perhaps can determine if the prev_log_idx that is returned is the snapshot_idx and if so, then dont increment by 1. Its possible we could pass even another flag into the appendentries struct to say if its a snapshot, but that's ugly.
      
      In addition, we increment msg_id on each appendentries so that if a response comes back out of order and we have already accepted the later response, we just ignore this response.
      
      In addition, on appendentry response, even if the appendentry wasn't a success, as long as its the same term as us (i.e. we are still the leader) then for leader/quorum purposes, that is sufficient, so move up set_last_ack to right after term test.
      64747dee
  11. 18 Aug, 2021 1 commit
  12. 11 Aug, 2021 1 commit
  13. 06 Aug, 2021 1 commit
  14. 02 Aug, 2021 1 commit
  15. 01 Aug, 2021 1 commit
  16. 30 Jul, 2021 1 commit
    • Shaya Potter's avatar
      remove REMOVE_NODE log type and include its semantics in DEMOTE (#25) · d4d15e04
      Shaya Potter authored
      this commit changes libraft's from using the DEMOTE_NODE log entry which would then initiate a REMOVE_NODE log entry on the DEMOTE's commit, to only using a REMOVE.  
      
      * add the removal of the active node flag on append of REMOVE and (add back if reverted)
      * modify is_voting to check for active state as well
      
      this enables simplifying appending and reverting nodes, as we shouldn't be touching voting state, (as a removing node might not be voting yet), only its active state.  and as the is_voting check also now checks for active state, simplifies a lot of code that manually tested both.
      
      * add a test for reversion of a non voting node removal
      
      previously, the code could not handle demoting/removing a non voting node, as demote would want to unset the voting flag (which a non voting node obviously doesn't have), and would therefore fail an assertion.
      
      with the new code, we no longer unset the voting flag in removal, and this test make sure both the is_voting and is_active return as expected through a process of add/removal/revert
      d4d15e04
  17. 26 Jul, 2021 1 commit
    • Shaya Potter's avatar
      only promote node to voting when its nonvoting has been applied (#12) · 600cf4fe
      Shaya Potter authored
      before we were able to promote a "ready" but not applied nonvoting node to a voting state, which means it would change the calculation as could vote for itself to be added.
      
      * fix tests that check for has_sufficient_logs
      
      because of a fix put in to not allow a node to be added until its "addition" as a non voter was comitted, that broke existing tests where that flag wasn't being set and testing that the nodes could be promoted
      
      * add a test that has_sufficient_logs will fail if addition not committed yet
      
      code before would add a node as a voter (via has_sufficient_logs) even if the initial non voter addition wasn't committed
      
      the test for this in the code actually tested after setting addition_committed.  add a test that does the same without that flag being set and expects that has_sufficient_logs will not be set
      600cf4fe
  18. 19 Jul, 2021 1 commit
  19. 12 Mar, 2020 1 commit
    • Yossi Gottlieb's avatar
      Add RAFT_LOGTYPE_NO_OP. · 5d7915f3
      Yossi Gottlieb authored
      The no-op entry gets committed as soon as a new leader is elected, in
      order to establish the commit index and prevent stale reads.
      5d7915f3
  20. 19 Dec, 2018 1 commit
    • Yossi Gottlieb's avatar
      Read request handling support. · 84362fa7
      Yossi Gottlieb authored
      This makes it possible to request a consistent read without having to
      commit it as a log operation.
      
      For every read request an AE heartbeat is delivered and a quorum
      response is expected.
      84362fa7
  21. 18 Dec, 2018 1 commit
  22. 09 Dec, 2018 1 commit
  23. 06 Dec, 2018 1 commit
  24. 20 Jun, 2018 2 commits
  25. 13 Jun, 2018 2 commits
  26. 11 Jun, 2018 1 commit
    • Yossi Gottlieb's avatar
      Ignore RequestVote when leader is established. (#58) · ab96a768
      Yossi Gottlieb authored
      This behavior is described in the original Raft paper and is required to
      prevent removed nodes from disrupting the cluster.
      
      This causes other problems though, as a split node may trigger election
      which is rejected but leaves it with a future term.  The generally
      accepted solution for this appears to be the use of PreVote RPC to avoid
      election that can never succeed.
      ab96a768
  27. 08 Jun, 2018 1 commit
  28. 24 May, 2018 1 commit
  29. 17 May, 2018 1 commit
  30. 16 May, 2018 1 commit
  31. 11 Apr, 2018 1 commit
  32. 07 Mar, 2018 1 commit
    • Yossi Gottlieb's avatar
      Ignore RequestVote when leader is established. · 6cac5767
      Yossi Gottlieb authored
      This behavior is described in the original Raft paper and is required to
      prevent removed nodes from disrupting the cluster.
      
      This causes other problems though, as a split node may trigger election
      which is rejected but leaves it with a future term.  The generally
      accepted solution for this appears to be the use of PreVote RPC to avoid
      election that can never succeed.
      6cac5767
  33. 08 Jan, 2018 2 commits
  34. 29 Dec, 2017 1 commit
    • Willem Thiart's avatar
      Fix: log edge cases · b912ff5a
      Willem Thiart authored
      The raft_log API is now being fuzzed tested. This change fixes all of
      the bugs detected by the fuzzer so far.
      b912ff5a
  35. 21 Nov, 2017 1 commit