1. 27 Jun, 2017 1 commit
    • antirez's avatar
      RDB modules values serialization format version 2. · 365dd037
      antirez authored
      The original RDB serialization format was not parsable without the
      module loaded, becuase the structure was managed only by the module
      itself. Moreover RDB is a streaming protocol in the sense that it is
      both produce di an append-only fashion, and is also sometimes directly
      sent to the socket (in the case of diskless replication).
      
      The fact that modules values cannot be parsed without the relevant
      module loaded is a problem in many ways: RDB checking tools must have
      loaded modules even for doing things not involving the value at all,
      like splitting an RDB into N RDBs by key or alike, or just checking the
      RDB for sanity.
      
      In theory module values could be just a blob of data with a prefixed
      length in order for us to be able to skip it. However prefixing the values
      with a length would mean one of the following:
      
      1. To be able to write some data at a previous offset. This breaks
      stremaing.
      2. To bufferize values before outputting them. This breaks performances.
      3. To have some chunked RDB output format. This breaks simplicity.
      
      Moreover, the above solution, still makes module values a totally opaque
      matter, with the fowllowing problems:
      
      1. The RDB check tool can just skip the value without being able to at
      least check the general structure. For datasets composed mostly of
      modules values this means to just check the outer level of the RDB not
      actually doing any checko on most of the data itself.
      2. It is not possible to do any recovering or processing of data for which a
      module no longer exists in the future, or is unknown.
      
      So this commit implements a different solution. The modules RDB
      serialization API is composed if well defined calls to store integers,
      floats, doubles or strings. After this commit, the parts generated by
      the module API have a one-byte prefix for each of the above emitted
      parts, and there is a final EOF byte as well. So even if we don't know
      exactly how to interpret a module value, we can always parse it at an
      high level, check the overall structure, understand the types used to
      store the information, and easily skip the whole value.
      
      The change is backward compatible: older RDB files can be still loaded
      since the new encoding has a new RDB type: MODULE_2 (of value 7).
      The commit also implements the ability to check RDB files for sanity
      taking advantage of the new feature.
      365dd037
  2. 26 Jun, 2017 1 commit
  3. 23 Jun, 2017 1 commit
    • antirez's avatar
      Issue #4027: unify comment and modify return value in freeMemoryIfNeeded(). · c9097393
      antirez authored
      It looks safer to return C_OK from freeMemoryIfNeeded() when clients are
      paused because returning C_ERR may prevent success of writes. It is
      possible that there is no difference in practice since clients cannot
      execute writes while clients are paused, but it looks more correct this
      way, at least conceptually.
      
      Related to PR #4028.
      c9097393
  4. 22 Jun, 2017 2 commits
  5. 19 Jun, 2017 1 commit
  6. 16 Jun, 2017 1 commit
  7. 15 Jun, 2017 1 commit
  8. 14 Jun, 2017 2 commits
  9. 13 Jun, 2017 1 commit
  10. 04 Jun, 2017 2 commits
  11. 01 Jun, 2017 1 commit
  12. 19 May, 2017 2 commits
  13. 15 May, 2017 1 commit
  14. 10 May, 2017 2 commits
    • antirez's avatar
      Modules TSC: use atomic var for server.unixtime. · 1f598fc2
      antirez authored
      This avoids Helgrind complaining, but we are actually not using
      atomicGet() to get the unixtime value for now: too many places where it
      is used and given tha time_t is word-sized it should be safe in all the
      archs we support as it is.
      
      On the other hand, Helgrind, when Redis is compiled with "make helgrind"
      in order to force the __sync macros, will detect the write in
      updateCachedTime() as a read (because atomic functions are used) and
      will not complain about races.
      
      This commit also includes minor refactoring of mutex initializations and
      a "helgrind" target in the Makefile.
      1f598fc2
    • antirez's avatar
      atomicvar.h: show used API in INFO. Add macro to force __sync builtin. · de786186
      antirez authored
      The __sync builtin can be correctly detected by Helgrind so to force it
      is useful for testing. The API in the INFO output can be useful for
      debugging after problems are reported.
      de786186
  15. 09 May, 2017 3 commits
  16. 04 May, 2017 2 commits
  17. 03 May, 2017 8 commits
  18. 02 May, 2017 3 commits
  19. 28 Apr, 2017 1 commit
  20. 27 Apr, 2017 1 commit
    • antirez's avatar
      PSYNC2: fix master cleanup when caching it. · 469d6e2b
      antirez authored
      The master client cleanup was incomplete: resetClient() was missing and
      the output buffer of the client was not reset, so pending commands
      related to the previous connection could be still sent.
      
      The first problem caused the client argument vector to be, at times,
      half populated, so that when the correct replication stream arrived the
      protcol got mixed to the arugments creating invalid commands that nobody
      called.
      
      Thanks to @yangsiran for also investigating this problem, after
      already providing important design / implementation hints for the
      original PSYNC2 issues (see referenced Github issue).
      
      Note that this commit adds a new function to the list library of Redis
      in order to be able to reset a list without destroying it.
      
      Related to issue #3899.
      469d6e2b
  21. 21 Apr, 2017 1 commit
    • antirez's avatar
      Check event loop creation return value. Fix #3951. · 238cebdd
      antirez authored
      Normally we never check for OOM conditions inside Redis since the
      allocator will always return a pointer or abort the program on OOM
      conditons. However we cannot have control on epool_create(), that may
      fail for kernel OOM (according to the manual page) even if all the
      parameters are correct, so the function aeCreateEventLoop() may indeed
      return NULL and this condition must be checked.
      238cebdd
  22. 19 Apr, 2017 2 commits
    • antirez's avatar
      Fix getKeysUsingCommandTable() in cluster mode. · 7d9dd80d
      antirez authored
      Close #3940.
      7d9dd80d
    • antirez's avatar
      PSYNC2: discard pending transactions from cached master. · 189a12af
      antirez authored
      During the review of the fix for #3899, @yangsiran identified an
      implementation bug: given that the offset is now relative to the applied
      part of the replication log, when we cache a master, the successive
      PSYNC2 request will be made in order to *include* the transaction that
      was not completely processed. This means that we need to discard any
      pending transaction from our replication buffer: it will be re-executed.
      189a12af