1. 06 Nov, 2019 3 commits
    • antirez's avatar
      Update PR #6537: use a fresh time outside call(). · d3f4dec4
      antirez authored
      One problem with the solution proposed so far in #6537 is that key
      lookups outside a command execution via call(), still used a cached
      time. The cached time needed to be refreshed in multiple places,
      especially because of modules callbacks from timers, cluster bus, and
      thread safe contexts, that may use RM_Open().
      
      In order to avoid this problem, this commit introduces the ability to
      detect if we are inside call(): this way we can use the reference fixed
      time only when we are in the context of a command execution or Lua
      script, but for the asynchronous lookups, we can still use mstime() to
      get a fresh time reference.
      d3f4dec4
    • antirez's avatar
      Update PR #6537 patch to for generality. · 33f42665
      antirez authored
      After the thread in #6537 and thanks to the suggestions received, this
      commit updates the original patch in order to:
      
      1. Solve the problem of updating the time in multiple places by updating
      it in call().
      2. Avoid introducing a new field but use our cached time.
      
      This required some minor refactoring to the function updating the time,
      and the introduction of a new cached time in microseconds in order to
      use less gettimeofday() calls.
      33f42665
    • zhaozhao.zz's avatar
      expires: refactoring judgment about whether a key is expired · 68d71d83
      zhaozhao.zz authored
      Calling lookupKey*() many times to search a key in one command
      may get different result.
      
      That's because lookupKey*() calls expireIfNeeded(), and delete
      the key when reach the expire time. So we can get an robj before
      the expire time, but a NULL after the expire time.
      
      The worst is that may lead to Redis crash, for example
      `RPOPLPUSH foo foo` the first time we get a list form `foo` and
      hold the pointer, but when we get `foo` again it's expired and
      deleted. Now we hold a freed memory, when execute rpoplpushHandlePush()
      redis crash.
      
      To fix it, we can refactor the judgment about whether a key is expired,
      using the same basetime `server.cmd_start_mstime` instead of calling
      mstime() everytime.
      68d71d83
  2. 11 Dec, 2018 1 commit
  3. 05 Nov, 2018 3 commits
  4. 02 Aug, 2018 1 commit
  5. 30 Jul, 2018 3 commits
  6. 03 Jul, 2018 1 commit
  7. 28 Jun, 2018 1 commit
  8. 21 Jun, 2018 1 commit
  9. 18 Jun, 2018 2 commits
    • antirez's avatar
      Streams: fix xreadGetKeys() for correctness. · 20766608
      antirez authored
      The old version could not handle the fact that "STREAMS" is a valid key
      name for streams. Now we really try to parse the command like the
      command implementation would do.
      
      Related to #5028 and 4857.
      20766608
    • antirez's avatar
      Streams: fix xreadGetKeys() buffer overflow. · a0b27dae
      antirez authored
      The loop allocated a buffer for the right number of keys positions, then
      overflowed it going past the limit.
      
      Related to #4857 and cause of the memory violation seen in #5028.
      a0b27dae
  10. 14 Jun, 2018 1 commit
  11. 13 Jun, 2018 2 commits
  12. 12 Jun, 2018 1 commit
  13. 04 Jun, 2018 1 commit
  14. 29 Apr, 2018 1 commit
  15. 27 Feb, 2018 2 commits
  16. 11 Feb, 2018 1 commit
  17. 12 Jan, 2018 1 commit
    • antirez's avatar
      Fix getKeysUsingCommandTable() in the case of nagative arity. · 2f8476df
      antirez authored
      This fixes a crash with Redis Cluster when OBJECT is mis-used, because
      getKeysUsingCommandTable() will call serverPanic() detecting we are
      accessing an invalid argument in the case "OBJECT foo" is called.
      
      This bug was introduced when OBJECT HELP was introduced, because the key
      argument is set fixed at index 2 in the command table, however now
      OBJECT may be called with an insufficient number of arguments to extract
      the key.
      
      The "Right Thing" would be to have a specific function to extract keys
      from the OBJECT command, however this is kinda of an overkill, so I
      preferred to make getKeysUsingCommandTable() more robust and just return
      no keys when it's not possible to honor the command table, because new
      commands are often added and also there are a number with an HELP
      subcommand violating the normal form, and crashing for this trivial
      reason or having many command-specific key extraction functions is not
      great.
      2f8476df
  18. 01 Dec, 2017 5 commits
  19. 27 Nov, 2017 1 commit
    • zhaozhao.zz's avatar
      LFU: do some changes about LFU to find hotkeys · 583c3147
      zhaozhao.zz authored
      Firstly, use access time to replace the decreas time of LFU.
      For function LFUDecrAndReturn,
      it should only try to get decremented counter,
      not update LFU fields, we will update it in an explicit way.
      And we will times halve the counter according to the times of
      elapsed time than server.lfu_decay_time.
      Everytime a key is accessed, we should update the LFU
      including update access time, and increment the counter after
      call function LFUDecrAndReturn.
      If a key is overwritten, the LFU should be also updated.
      Then we can use `OBJECT freq` command to get a key's frequence,
      and LFUDecrAndReturn should be called in `OBJECT freq` command
      in case of the key has not been accessed for a long time,
      because we update the access time only when the key is read or
      overwritten.
      583c3147
  20. 20 Sep, 2017 1 commit
    • zhaozhao.zz's avatar
      PSYNC2: make persisiting replication info more solid · b541ccef
      zhaozhao.zz authored
      This commit is a reinforcement of commit c1c99e9f.
      
      1. Replication information can be stored when the RDB file is
      generated by a mater using server.slaveseldb when server.repl_backlog
      is not NULL, or set repl_stream_db be -1. That's safe, because
      NULL server.repl_backlog will trigger full synchronization,
      then master will send SELECT command to replicaiton stream.
      2. Only do rdbSave* when rsiptr is not NULL,
      if we do rdbSave* without rdbSaveInfo, slave will miss repl-stream-db.
      3. Save the replication informations also in the case of
      SAVE command, FLUSHALL command and DEBUG reload.
      b541ccef
  21. 14 Jun, 2017 1 commit
  22. 13 Jun, 2017 1 commit
  23. 19 Apr, 2017 1 commit
  24. 07 Apr, 2017 1 commit
  25. 27 Mar, 2017 1 commit
  26. 30 Dec, 2016 1 commit
  27. 13 Dec, 2016 1 commit
    • antirez's avatar
      Replication: fix the infamous key leakage of writable slaves + EXPIRE. · 04542cff
      antirez authored
      BACKGROUND AND USE CASEj
      
      Redis slaves are normally write only, however the supprot a "writable"
      mode which is very handy when scaling reads on slaves, that actually
      need write operations in order to access data. For instance imagine
      having slaves replicating certain Sets keys from the master. When
      accessing the data on the slave, we want to peform intersections between
      such Sets values. However we don't want to intersect each time: to cache
      the intersection for some time often is a good idea.
      
      To do so, it is possible to setup a slave as a writable slave, and
      perform the intersection on the slave side, perhaps setting a TTL on the
      resulting key so that it will expire after some time.
      
      THE BUG
      
      Problem: in order to have a consistent replication, expiring of keys in
      Redis replication is up to the master, that synthesize DEL operations to
      send in the replication stream. However slaves logically expire keys
      by hiding them from read attempts from clients so that if the master did
      not promptly sent a DEL, the client still see logically expired keys
      as non existing.
      
      Because slaves don't actively expire keys by actually evicting them but
      just masking from the POV of read operations, if a key is created in a
      writable slave, and an expire is set, the key will be leaked forever:
      
      1. No DEL will be received from the master, which does not know about
      such a key at all.
      
      2. No eviction will be performed by the slave, since it needs to disable
      eviction because it's up to masters, otherwise consistency of data is
      lost.
      
      THE FIX
      
      In order to fix the problem, the slave should be able to tag keys that
      were created in the slave side and have an expire set in some way.
      
      My solution involved using an unique additional dictionary created by
      the writable slave only if needed. The dictionary is obviously keyed by
      the key name that we need to track: all the keys that are set with an
      expire directly by a client writing to the slave are tracked.
      
      The value in the dictionary is a bitmap of all the DBs where such a key
      name need to be tracked, so that we can use a single dictionary to track
      keys in all the DBs used by the slave (actually this limits the solution
      to the first 64 DBs, but the default with Redis is to use 16 DBs).
      
      This solution allows to pay both a small complexity and CPU penalty,
      which is zero when the feature is not used, actually. The slave-side
      eviction is encapsulated in code which is not coupled with the rest of
      the Redis core, if not for the hook to track the keys.
      
      TODO
      
      I'm doing the first smoke tests to see if the feature works as expected:
      so far so good. Unit tests should be added before merging into the
      4.0 branch.
      04542cff