1. 22 Jun, 2023 1 commit
  2. 20 Jun, 2023 1 commit
  3. 23 May, 2023 1 commit
  4. 10 May, 2023 3 commits
    • Ozan Tezcan's avatar
      Fix load_snapshot() error handling bug (#162) · 9a0c6c0e
      Ozan Tezcan authored
      Consider the snapshot delivery scenario:
      
      - Follower receives the last chunk, calls store_snapshot_chunk() 
      succesfully and updates its snapshot_recv_offset. Later it calls
      load_snapshot() but it fails. 
      
      - Follower returns failure response. In the response, "offset" will be
      zero. So, follower requests leader to start snapshot delivery from the
      beginning.
      
      - Leader sends the first chunk. Follower rejects the message as its 
      snapshot_recv_offset is not zero (Actually, follower has stored all the
      bytes, snapshot_recv_offset is equal to snapshot size). Follower returns
      this offset in the response.
      
      - Leader calls get_snapshot_chunk() callback for the received offset from
      the follower. As there are no more bytes to send, get_snapshot_chunk()
      returns RAFT_ERR_DONE and leader skips sending snapshot req.
      
      This bug causes leader to stop sending appendreq/snapshotreq to the 
      follower until leader takes another snapshot. Follower starts an 
      election as it doesn't hear from the leader but other nodes won't vote
      for it as the rest of the cluster operates fine.
      
      Solution is to consider store_snapshot_chunk() and load_snapshot() as 
      an atomic unit. So, if load_snapshot() fails, we return the response as
      if like we didn't store the last chunk. Leader will send the same chunk
      again and we can retry the operation.
      9a0c6c0e
    • Ozan Tezcan's avatar
      Add snapshot_index to raft_snapshot_resp_t struct (#163) · 9ba34e87
      Ozan Tezcan authored
      Normally, receiver side has checks if the chunk is for the correct 
      index and offset. 
      
      Adding snapshot_index to raft_snapshot_resp_t just to be sure we don't 
      update snapshot offset for the node unnecessarily.
      
      Also, it makes debugging easier as we know the snapshot index that 
      response refers to. 
      9ba34e87
    • Shaya Potter's avatar
      RR-238 - Recording and Retrieving of RAFT RPC stats · 2b7f9c7f
      Shaya Potter authored
      
      
      Record counts for raft RPC stats, sending / retrieving / success / failure and the like.
      
      cover append entry, requestvote (including prevote) and snapshots
      
      ---------
      Co-authored-by: default avatarOzan Tezcan <ozantezcan@gmail.com>
      2b7f9c7f
  5. 09 May, 2023 1 commit
  6. 04 May, 2023 1 commit
    • Ozan Tezcan's avatar
      Use raft_size_t for entry len (#160) · c5aee21b
      Ozan Tezcan authored
      Currently, entry len type is unsigned int which limits entry size to 4 gb.
      Changing it to raft_size_t (unsigned long long) to allow larger entries.
      c5aee21b
  7. 15 Apr, 2023 1 commit
  8. 13 Apr, 2023 1 commit
  9. 27 Dec, 2022 1 commit
  10. 31 Oct, 2022 1 commit
  11. 26 Oct, 2022 1 commit
  12. 25 Oct, 2022 2 commits
    • Ozan Tezcan's avatar
      Refactor higher term handling (#149) · 2ce5363f
      Ozan Tezcan authored
      This PR refactors code regarding `term` handling inside 
      `raft_recv_appendentries()` and `raft_recv_snapshot()` without 
      introducing any logic changes. 
      
      - Deleted `raft_receive_term()` function:
          This function sets the node as follower in certain cases. Although,
          it misses the pre-candidate state.
           
          We already set the node as follower inside `raft_accept_leader()`.
          (With the correct check that also handles pre-candidate state). 
      
      - Moved `raft_reset_transfer_leader()` into `raft_accept_leader()`. 
          Leader transfer is completed when we accept a leader. So, it makes
          sense to move it into `raft_accept_leader()`.
          
      - Added some tests to verify node becomes follower even on a
           pre-candidate state when a request is received with same or
           higher term.
      2ce5363f
    • Ozan Tezcan's avatar
      Refactor raft_recv_appendentries() (#148) · 7be57f36
      Ozan Tezcan authored
      Refactored raft_recv_appendentries() just to simplify the flow.
      
      This PR should not introduce any behavioral change. Only exception is
      previously, some tests were manually creating an append request with
      prev_log_idx:0 and prev_log_term:1. This is not possible and it
      cannot happen without manually creating append requests. Term of the
      entry at index 0 cannot be 1. (it is zero by definition). Fixed these tests
      as part of this PR.
      7be57f36
  13. 20 Oct, 2022 1 commit
  14. 03 Oct, 2022 1 commit
  15. 28 Sep, 2022 1 commit
  16. 22 Sep, 2022 1 commit
  17. 20 Sep, 2022 1 commit
  18. 19 Sep, 2022 4 commits
    • Ozan Tezcan's avatar
      Documentation fixes (#142) · 6ca56090
      Ozan Tezcan authored
      
      
      Documentation fixes
      Co-authored-by: default avatarHanna Fadida <hanna.fadida@redislabs.com>
      6ca56090
    • Ozan Tezcan's avatar
      Documentation (#140) · 5d42a923
      Ozan Tezcan authored
      Documentation
      5d42a923
    • Ozan Tezcan's avatar
      Handle unnecessary snapshots better (#143) · 85ae16e0
      Ozan Tezcan authored
      If a node receives a snapshot with snapshot_last_idx, it checks if 
      it already has the snapshot and informs leader accordingly.
      
      The problem is the receiver node may not have a snapshot but log 
      entries more than snapshot_last_idx:
      
      e.g
      Leader: snapshot_last_idx : 50
      Follower: snapshot_last_idx:0, last log entry index: 100
      
      If the leader tries to send snapshot to the follower 
      (with snapshot last index:50), currently, follower accepts 
      the snapshot. As follower already has entries upto index 100, snapshot
      is unnecessary. After delivery is complete, to load the snapshot, 
      follower will call raft_begin_load_snapshot(). This function realizes
      the snapshot was unnecessary and it fails. It is harmless but snapshot
      delivery was unnecessary.
      
      This PR prevents this issue earlier by checking last log index instead
       of snapshot last index.
      
      Also, added more virtraft tests with auto-flush on. 
      (This is a different pattern, it might help with detecting these issues)
      85ae16e0
    • Ozan Tezcan's avatar
      Move internal functions to the private API (#141) · c76eac81
      Ozan Tezcan authored
      Moved internal functions to raft_internal.h
      
      Still, application can include raft_private.h use internal
      functions (not suggested).
      c76eac81
  19. 12 Sep, 2022 3 commits
    • Ozan Tezcan's avatar
      Handle response error cases gracefully (#137) · 8db9930c
      Ozan Tezcan authored
      A node can receive responses from followers who are not
      aware of they have been removed from the cluster configuration.
      In this case, recv_ family functions can be called without 
      a node structure. We should ignore these messages.
      
      We should also ignore snapshot and appendentries responses
      if we are not leader anymore. Returning an error code for 
      this case does not make sense because application cannot 
      do anything but ignore the error code. 
      
      This PR changes return value to zero for these cases. 
      8db9930c
    • Ozan Tezcan's avatar
      Add raft_get_error_str() (#134) · 703e70c2
      Ozan Tezcan authored
      Added a helper function to get string representation of an error code.
      703e70c2
    • Ozan Tezcan's avatar
      Refactor callback documentation (#139) · 8857bcdd
      Ozan Tezcan authored
      Refactor callback comments.
      
      - Regroup related callbacks.
      - Move optional callbacks to the bottom and indicate "(optional)".
      8857bcdd
  20. 05 Sep, 2022 1 commit
  21. 04 Sep, 2022 1 commit
  22. 30 Aug, 2022 1 commit
    • Ozan Tezcan's avatar
      Refactor debug logging (#127) · e79d3b6a
      Ozan Tezcan authored
      Refactor debug logging and change log callback
      
      e.g:
      19 --> 13, sent snapshot_resp id:27712, t:24, s:1, o:4096, lc:0
      14 --> 17, sent requestvote_req pv:1, t:25, ci:14, lli:6848, llt:23
      14 <-- 17, recv requestvote_resp pv:1, t:25, rt:24, vg:0
      
      Breaking changes: 
      
      raft_log_f() callback will not pass node_id_t anymore.
      Sender and receiver will be part of the log line as you can see above.
      e79d3b6a
  23. 29 Aug, 2022 3 commits
    • Ozan Tezcan's avatar
      Simplify entry removal (#130) · c0270e61
      Ozan Tezcan authored
      Simplied pop callback.
      
      Previously, libraft was passing a callback(raft_entry_notify_f) to the pop callback.
      The application was supposed to call that callback for each removed entry.
      
      Libraft can loop over the entries and call the required functions itself.
      So, we can delete raft_entry_notify_f from the pop callback just to
      simplify API.
      
      In addition to that, this PR removes raft_pop_entry() function which doesn't have
      a use-case, as far as I can see.
      We can use raft_delete_entry_from_idx(me, raft_get_current_idx()) instead if required.
      c0270e61
    • Ozan Tezcan's avatar
      Add raft_get_last_applied_term() (#131) · 2279a960
      Ozan Tezcan authored
      Add raft_get_last_applied_term()
      2279a960
    • Ozan Tezcan's avatar
      Fix style (#128) · 21844ec6
      Ozan Tezcan authored
      Fix style issues in raft_log.c raft_node.c and raft_server_properties.c
      21844ec6
  24. 28 Aug, 2022 1 commit
  25. 25 Aug, 2022 2 commits
    • Ozan Tezcan's avatar
      Fix memory leak in raft_restore_log() (#126) · f1ab5203
      Ozan Tezcan authored
      f1ab5203
    • Ozan Tezcan's avatar
      Add support to rebuild configuration after a restart (#125) · 9c09a6cc
      Ozan Tezcan authored
      Configuration changes are special in Raft. They take effect 
      whenever the related log entries are appended. On a restart, 
      we should go over the log entries and rebuild the configuration.
      
      Adding raft_restore_log() function for this purpose.
      
      Overall, on a restart, application should do:
      
      Load the snapshot and call raft_restore_snapshot()
      Load the log entries and call raft_restore_log()
      Read the metadata file and call raft_restore_metadata().
      9c09a6cc
  26. 23 Aug, 2022 3 commits
  27. 22 Aug, 2022 1 commit