1. 04 Nov, 2021 1 commit
    • Eduardo Semprebon's avatar
      Replica keep serving data during repl-diskless-load=swapdb for better availability (#9323) · 91d0c758
      Eduardo Semprebon authored
      
      
      For diskless replication in swapdb mode, considering we already spend replica memory
      having a backup of current db to restore in case of failure, we can have the following benefits
      by instead swapping database only in case we succeeded in transferring db from master:
      
      - Avoid `LOADING` response during failed and successful synchronization for cases where the
        replica is already up and running with data.
      - Faster total time of diskless replication, because now we're moving from Transfer + Flush + Load
        time to Transfer + Load only. Flushing the tempDb is done asynchronously after swapping.
      - This could be implemented also for disk replication with similar benefits if consumers are willing
        to spend the extra memory usage.
      
      General notes:
      - The concept of `backupDb` becomes `tempDb` for clarity.
      - Async loading mode will only kick in if the replica is syncing from a master that has the same
        repl-id the one it had before. i.e. the data it's getting belongs to a different time of the same timeline. 
      - New property in INFO: `async_loading` to differentiate from the blocking loading
      - Slot to Key mapping is now a field of `redisDb` as it's more natural to access it from both server.db
        and the tempDb that is passed around.
      - Because this is affecting replicas only, we assume that if they are not readonly and write commands
        during replication, they are lost after SYNC same way as before, but we're still denying CONFIG SET
        here anyways to avoid complications.
      
      Considerations for review:
      - We have many cases where server.loading flag is used and even though I tried my best, there may
        be cases where async_loading should be checked as well and cases where it shouldn't (would require
        very good understanding of whole code)
      - Several places that had different behavior depending on the loading flag where actually meant to just
        handle commands coming from the AOF client differently than ones coming from real clients, changed
        to check CLIENT_ID_AOF instead.
      
      **Additional for Release Notes**
      - Bugfix - server.dirty was not incremented for any kind of diskless replication, as effect it wouldn't
        contribute on triggering next database SAVE
      - New flag for RM_GetContextFlags module API: REDISMODULE_CTX_FLAGS_ASYNC_LOADING
      - Deprecated RedisModuleEvent_ReplBackup. Starting from Redis 7.0, we don't fire this event.
        Instead, we have the new RedisModuleEvent_ReplAsyncLoad holding 3 sub-events: STARTED,
        ABORTED and COMPLETED.
      - New module flag REDISMODULE_OPTIONS_HANDLE_REPL_ASYNC_LOAD for RedisModule_SetModuleOptions
        to allow modules to declare they support the diskless replication with async loading (when absent, we fall
        back to disk-based loading).
      Co-authored-by: default avatarEduardo Semprebon <edus@saxobank.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      91d0c758
  2. 13 Apr, 2021 1 commit
  3. 06 Apr, 2021 1 commit
  4. 13 Dec, 2020 1 commit
    • Oran Agra's avatar
      Add module event for repl-diskless-load swapdb (#8153) · ab60dcf5
      Oran Agra authored
      When a replica uses the diskless-load swapdb approach, it backs up the old database,
      then attempts to load a new one, and in case of failure, it restores the backup.
      
      this means that modules with global out of keyspace data, must have an option to
      subscribe to events and backup/restore/discard their global data too.
      ab60dcf5
  5. 02 Dec, 2019 1 commit
  6. 10 Nov, 2019 1 commit
    • Oran Agra's avatar
      fix leak in module api rdb test · 02f21113
      Oran Agra authored
      recently added more reads into that function, if a later read fails, i must
      either free what's already allocated, or return the pointer so that the free
      callback will release it.
      02f21113
  7. 03 Nov, 2019 1 commit
    • Oran Agra's avatar
      Module API for loading and saving long double · 779aebc9
      Oran Agra authored
      looks like each platform implements long double differently (different bit count)
      so we can't save them as binary, and we also want to avoid creating a new RDB
      format version, so we save these are hex strings using "%La".
      
      This commit includes a change in the arguments of ld2string to support this.
      as well as tests for coverage and short reads.
      
      coded by @guybe7
      779aebc9
  8. 24 Oct, 2019 1 commit
  9. 30 Jul, 2019 1 commit
  10. 22 Jul, 2019 1 commit