1. 18 Dec, 2019 3 commits
  2. 09 Jan, 2019 2 commits
  3. 22 Oct, 2018 1 commit
  4. 16 Oct, 2018 1 commit
  5. 18 Jun, 2018 1 commit
  6. 13 Dec, 2016 1 commit
    • antirez's avatar
      Replication: fix the infamous key leakage of writable slaves + EXPIRE. · 04542cff
      antirez authored
      BACKGROUND AND USE CASEj
      
      Redis slaves are normally write only, however the supprot a "writable"
      mode which is very handy when scaling reads on slaves, that actually
      need write operations in order to access data. For instance imagine
      having slaves replicating certain Sets keys from the master. When
      accessing the data on the slave, we want to peform intersections between
      such Sets values. However we don't want to intersect each time: to cache
      the intersection for some time often is a good idea.
      
      To do so, it is possible to setup a slave as a writable slave, and
      perform the intersection on the slave side, perhaps setting a TTL on the
      resulting key so that it will expire after some time.
      
      THE BUG
      
      Problem: in order to have a consistent replication, expiring of keys in
      Redis replication is up to the master, that synthesize DEL operations to
      send in the replication stream. However slaves logically expire keys
      by hiding them from read attempts from clients so that if the master did
      not promptly sent a DEL, the client still see logically expired keys
      as non existing.
      
      Because slaves don't actively expire keys by actually evicting them but
      just masking from the POV of read operations, if a key is created in a
      writable slave, and an expire is set, the key will be leaked forever:
      
      1. No DEL will be received from the master, which does not know about
      such a key at all.
      
      2. No eviction will be performed by the slave, since it needs to disable
      eviction because it's up to masters, otherwise consistency of data is
      lost.
      
      THE FIX
      
      In order to fix the problem, the slave should be able to tag keys that
      were created in the slave side and have an expire set in some way.
      
      My solution involved using an unique additional dictionary created by
      the writable slave only if needed. The dictionary is obviously keyed by
      the key name that we need to track: all the keys that are set with an
      expire directly by a client writing to the slave are tracked.
      
      The value in the dictionary is a bitmap of all the DBs where such a key
      name need to be tracked, so that we can use a single dictionary to track
      keys in all the DBs used by the slave (actually this limits the solution
      to the first 64 DBs, but the default with Redis is to use 16 DBs).
      
      This solution allows to pay both a small complexity and CPU penalty,
      which is zero when the feature is not used, actually. The slave-side
      eviction is encapsulated in code which is not coupled with the rest of
      the Redis core, if not for the hook to track the keys.
      
      TODO
      
      I'm doing the first smoke tests to see if the feature works as expected:
      so far so good. Unit tests should be added before merging into the
      4.0 branch.
      04542cff
  7. 15 Jun, 2016 1 commit
  8. 27 Jul, 2015 1 commit
  9. 26 Jul, 2015 4 commits
  10. 03 Feb, 2015 1 commit
  11. 14 Dec, 2014 1 commit
    • Mihir Joshi's avatar
      Stricter options for SET command · 352172a7
      Mihir Joshi authored
      - As per Antirez's suggestion, this commit raises an error when mutually
      exclusive options are provided. Duplicate options are allowed.
      352172a7
  12. 11 Dec, 2014 1 commit
    • Matt Stancliff's avatar
      Bitops: Stop overallocating storage space on set · badf0f00
      Matt Stancliff authored
      Previously the string was created empty then re-sized
      to fit the offset, but sds resize causes the sds to
      over-allocate by at least 1 MB (which is a lot when
      you are operating at bit-level access).
      
      This also improves the speed of initial sets by 2% to 6%
      based on quick testing.
      
      Patch logic provided by @oranagra
      
      Fixes #1918
      badf0f00
  13. 02 Dec, 2014 1 commit
  14. 22 Nov, 2014 1 commit
    • Mihir Joshi's avatar
      stricter options for SET command · e9b014cf
      Mihir Joshi authored
      Issue: #2157
      As the SET command is parsed, it remembers which options are already set
      and if a duplicate option is found, raises an error because it is
      essentially an invalid syntax.
      
      It still allows mutually exclusive options like EX and PX because taking
      an option over another (precedence) is not essentially a syntactic
      error.
      e9b014cf
  15. 03 Oct, 2014 1 commit
    • antirez's avatar
      INCR: Modify incremented object in-place when possible. · 16559b46
      antirez authored
      However we don't try to do this if the integer is already inside a range
      representable with a shared integer.
      
      The performance gain appears to be around ~15% in micro benchmarks,
      however in the long run this also helps to improve locality, so should
      have more, hard to measure, benefits.
      16559b46
  16. 02 Sep, 2014 2 commits
    • Matt Stancliff's avatar
      Return empty string if GETRANGE of empty string · b20df972
      Matt Stancliff authored
      Previously, GETRANGE of a key containing nothing ("")
      would allocate a large (size_t)-1 return value causing
      crashes on 32bit builds when it tried to allocate the
      4 GB return string.
      b20df972
    • Matt Stancliff's avatar
      Increase size of range request in getrange · f0e306f4
      Matt Stancliff authored
      32 bit builds don't have a big enough long to capture
      the same range as a 64 bit build.  If we use "long long"
      we get proper size limits everywhere.
      
      Also updates size of unsigned comparison to fit new size of `end`.
      
      Fixes #1981
      f0e306f4
  17. 18 Aug, 2014 1 commit
  18. 07 Aug, 2014 1 commit
    • Jan-Erik Rediger's avatar
      Handle large getrange requests · 53fdfda9
      Jan-Erik Rediger authored
      Previously the end was casted to a smaller type
      which resulted in a wrong check and failed
      with values larger than handled by unsigned.
      
      Closes #1847, #1844
      53fdfda9
  19. 30 Mar, 2014 1 commit
    • antirez's avatar
      String value unsharing refactored into proper function. · 543ede03
      antirez authored
      All the Redis functions that need to modify the string value of a key in
      a destructive way (APPEND, SETBIT, SETRANGE, ...) require to make the
      object unshared (if refcount > 1) and encoded in raw format (if encoding
      is not already REDIS_ENCODING_RAW).
      
      This was cut & pasted many times in multiple places of the code. This
      commit puts the small logic needed into a function called
      dbUnshareStringValue().
      543ede03
  20. 22 Jul, 2013 1 commit
    • antirez's avatar
      Introduction of a new string encoding: EMBSTR · 894eba07
      antirez authored
      Previously two string encodings were used for string objects:
      
      1) REDIS_ENCODING_RAW: a string object with obj->ptr pointing to an sds
      stirng.
      
      2) REDIS_ENCODING_INT: a string object where the obj->ptr void pointer
      is casted to a long.
      
      This commit introduces a experimental new encoding called
      REDIS_ENCODING_EMBSTR that implements an object represented by an sds
      string that is not modifiable but allocated in the same memory chunk as
      the robj structure itself.
      
      The chunk looks like the following:
      
      +--------------+-----------+------------+--------+----+
      | robj data... | robj->ptr | sds header | string | \0 |
      +--------------+-----+-----+------------+--------+----+
                           |                       ^
                           +-----------------------+
      
      The robj->ptr points to the contiguous sds string data, so the object
      can be manipulated with the same functions used to manipulate plan
      string objects, however we need just on malloc and one free in order to
      allocate or release this kind of objects. Moreover it has better cache
      locality.
      
      This new allocation strategy should benefit both the memory usage and
      the performances. A performance gain between 60 and 70% was observed
      during micro-benchmarks, however there is more work to do to evaluate
      the performance impact and the memory usage behavior.
      894eba07
  21. 29 Mar, 2013 1 commit
  22. 28 Mar, 2013 1 commit
  23. 28 Jan, 2013 3 commits
  24. 19 Jan, 2013 2 commits
  25. 08 Nov, 2012 1 commit
  26. 31 Oct, 2012 1 commit
    • antirez's avatar
      Invert two sides of if expression in SET to avoid a lookup. · 973cb21a
      antirez authored
      Because of the short circuit behavior of && inverting the two sides of
      the if expression avoids an hash table lookup if the non-EX variant of
      SET is called.
      
      Thanks to Weibin Yao (@yaoweibin on github) for spotting this.
      973cb21a
  27. 24 May, 2012 2 commits
    • antirez's avatar
      Bit-related string operations moved to bitop.c · 760e7765
      antirez authored
      All the general string operations are implemented in t_string.c, however
      the bit operations, while targeting the string type, are better served
      in a specific file where we have the implementations of the following
      four commands and helper functions:
      
          GETBIT
          SETBIT
          BITOP
          BITCOUNT
      
      In the future this file will probably contain more code related to
      making the BITOP and BITCOUNT operations faster.
      760e7765
    • antirez's avatar
      New commands: BITOP and BITCOUNT. · 0bd6d68e
      antirez authored
      The motivation for this new commands is to be search in the usage of
      Redis for real time statistics. See the article "Fast real time metrics
      using Redis".
      
      http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/
      
      In general Redis strings when used as bitmaps using the SETBIT/GETBIT
      command provide a very space-efficient and fast way to store statistics.
      For instance in a web application with users, every user can be
      associated with a key that shows every day in which the user visited the
      web service. This information can be really valuable to extract user
      behaviour information.
      
      With Redis bitmaps doing this is very simple just saying that a given
      day is 0 (the data the service was put online) and all the next days are
      1, 2, 3, and so forth. So with SETBIT it is possible to set the bit
      corresponding to the current day every time the user visits the site.
      
      It is possible to take the count of the bit sets on the run, this is
      extremely easy using a Lua script. However a fast bit count native
      operation can be useful, especially if it can operate on ranges, or when
      the string is small like in the case of days (even if you consider many
      years it is still extremely little data).
      
      For this reason BITOP was introduced. The command counts the number of
      bits set to 1 in a string, with optional range:
      
      BITCOUNT key [start end]
      
      The start/end parameters are similar to GETRANGE. If omitted the whole
      string is tested.
      
      Population counting is more useful when bit-level operations like AND,
      OR and XOR are avaialble. For instance I can test multiple users to see
      the number of days three users visited the site at the same time. To do
      this we can take the AND of all the bitmaps, and then count the set bits.
      
      For this reason the BITOP command was introduced:
      
      BITOP [AND|OR|XOR|NOT] dest_key src_key1 src_key2 src_key3 ... src_keyN
      
      In the special case of NOT (that inverts the bits) only one source key
      can be passed.
      
      The judicious use of BITCOUNT and BITOP combined can lead to interesting
      use cases with very space efficient representation of data.
      
      The implementation provided is still not tested and optimized for speed,
      next commits will introduce unit tests. Later the implementation will be
      profiled to see if it is possible to gain an important amount of speed
      without making the code much more complex.
      0bd6d68e
  28. 21 Feb, 2012 1 commit
  29. 14 Nov, 2011 1 commit