1. 30 Nov, 2012 2 commits
    • antirez's avatar
      SDIFF is now able to select between two algorithms for speed. · f50e6584
      antirez authored
      SDIFF used an algorithm that was O(N) where N is the total number
      of elements of all the sets involved in the operation.
      
      The algorithm worked like that:
      
      ALGORITHM 1:
      
      1) For the first set, add all the members to an auxiliary set.
      2) For all the other sets, remove all the members of the set from the
      auxiliary set.
      
      So it is an O(N) algorithm where N is the total number of elements in
      all the sets involved in the diff operation.
      
      Cristobal Viedma suggested to modify the algorithm to the following:
      
      ALGORITHM 2:
      
      1) Iterate all the elements of the first set.
      2) For every element, check if the element also exists in all the other
      remaining sets.
      3) Add the element to the auxiliary set only if it does not exist in any
      of the other sets.
      
      The complexity of this algorithm on the worst case is O(N*M) where N is
      the size of the first set and M the total number of sets involved in the
      operation.
      
      However when there are elements in common, with this algorithm we stop
      the computation for a given element as long as we find a duplicated
      element into another set.
      
      I (antirez) added an additional step to algorithm 2 to make it faster,
      that is to sort the set to subtract from the biggest to the
      smallest, so that it is more likely to find a duplicate in a larger sets
      that are checked before the smaller ones.
      
      WHAT IS BETTER?
      
      None of course, for instance if the first set is much larger than the
      other sets the second algorithm does a lot more work compared to the
      first algorithm.
      
      Similarly if the first set is much smaller than the other sets, the
      original algorithm will less work.
      
      So this commit makes Redis able to guess the number of operations
      required by each algorithm, and select the best at runtime according
      to the input received.
      
      However, since the second algorithm has better constant times and can do
      less work if there are duplicated elements, an advantage is given to the
      second algorithm.
      f50e6584
    • antirez's avatar
      b4abbaf7
  2. 29 Nov, 2012 2 commits
    • antirez's avatar
      Introduced the Build ID in INFO and --version output. · 2f62c966
      antirez authored
      The idea is to be able to identify a build in a unique way, so for
      instance after a bug report we can recognize that the build is the one
      of a popular Linux distribution and perform the debugging in the same
      environment.
      2f62c966
    • antirez's avatar
      On crash memory test rewrote so that it actaully works. · b1b602a9
      antirez authored
      1) We no longer test location by location, otherwise the CPU write cache
      completely makes our business useless.
      2) We still need a memory test that operates in steps from the first to
      the last location in order to never hit the cache, but that is still
      able to retain the memory content.
      
      This was tested using a Linux box containing a bad memory module with a
      zingle bit error (always zero).
      
      So the final solution does has an error propagation step that is:
      
      1) Invert bits at every location.
      2) Swap adiacent locations.
      3) Swap adiacent locations again.
      4) Invert bits at every location.
      5) Swap adiacent locations.
      6) Swap adiacent locations again.
      
      Before and after these steps, and after step 4, a CRC64 checksum is computed.
      If the three CRC64 checksums don't match, a memory error was detected.
      b1b602a9
  3. 28 Nov, 2012 2 commits
  4. 22 Nov, 2012 5 commits
    • charsyam's avatar
      remove compile warning bioKillThreads · d7c7ac4a
      charsyam authored
      d7c7ac4a
    • antirez's avatar
      EVALSHA is now case insensitive. · 95f68f7b
      antirez authored
      EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
      Fixed using a case insensitive dictionary type for the sha -> script
      map used for replication of scripts.
      95f68f7b
    • antirez's avatar
      Fix integer overflow in zunionInterGenericCommand(). · cceb0c5b
      antirez authored
      This fixes issue #761.
      cceb0c5b
    • antirez's avatar
      Safer handling of MULTI/EXEC on errors. · 3d139127
      antirez authored
      After the transcation starts with a MULIT, the previous behavior was to
      return an error on problems such as maxmemory limit reached. But still
      to execute the transaction with the subset of queued commands on EXEC.
      
      While it is true that the client was able to check for errors
      distinguish QUEUED by an error reply, MULTI/EXEC in most client
      implementations uses pipelining for speed, so all the commands and EXEC
      are sent without caring about replies.
      
      With this change:
      
      1) EXEC fails if at least one command was not queued because of an
      error. The EXECABORT error is used.
      2) A generic error is always reported on EXEC.
      3) The client DISCARDs the MULTI state after a failed EXEC, otherwise
      pipelining multiple transactions would be basically impossible:
      After a failed EXEC the next transaction would be simply queued as
      the tail of the previous transaction.
      3d139127
    • antirez's avatar
      Make bio.c threads killable ASAP if needed. · 75369917
      antirez authored
      We use this new bio.c feature in order to stop our I/O threads if there
      is a memory test to do on crash. In this case we don't want anything
      else than the main thread to run, otherwise the other threads may mess
      with the heap and the memory test will report a false positive.
      75369917
  5. 21 Nov, 2012 2 commits
  6. 19 Nov, 2012 2 commits
    • antirez's avatar
      Children creating AOF or RDB files now report memory used by COW. · 49b64523
      antirez authored
      Finally Redis is able to report the amount of memory used by
      copy-on-write while saving an RDB or writing an AOF file in background.
      
      Note that this information is currently only logged (at NOTICE level)
      and not shown in INFO because this is less trivial (but surely doable
      with some minor form of interprocess communication).
      
      The reason we can't capture this information on the parent before we
      call wait3() is that the Linux kernel will release the child memory
      ASAP, and only retain the minimal state for the process that is useful
      to report the child termination to the parent.
      
      The COW size is obtained by summing all the Private_Dirty fields found
      in the "smap" file inside the proc filesystem for the process.
      
      All this is Linux specific and is not available on other systems.
      49b64523
    • antirez's avatar
      zmalloc_get_private_dirty() function added (Linux only). · 3bfeb9c1
      antirez authored
      For non Linux systmes it just returns 0.
      
      This function is useful to estimate copy-on-write because of childs
      saving stuff on disk.
      3bfeb9c1
  7. 14 Nov, 2012 2 commits
  8. 12 Nov, 2012 4 commits
  9. 11 Nov, 2012 1 commit
    • antirez's avatar
      MIGRATE TCP connections caching. · e23d281e
      antirez authored
      By caching TCP connections used by MIGRATE to chat with other Redis
      instances a 5x performance improvement was measured with
      redis-benchmark against small keys.
      
      This can dramatically speedup cluster resharding and other processes
      where an high load of MIGRATE commands are used.
      e23d281e
  10. 08 Nov, 2012 1 commit
  11. 07 Nov, 2012 2 commits
    • antirez's avatar
      COPY and REPLACE options for MIGRATE. · 1237d71c
      antirez authored
      With COPY now MIGRATE does not remove the key from the source instance.
      With REPLACE it uses RESTORE REPLACE on the target host so that even if
      the key already eixsts in the target instance it will be overwritten.
      
      The options can be used together.
      1237d71c
    • antirez's avatar
      REPLACE option for RESTORE. · e5b5763f
      antirez authored
      The REPLACE option deletes an existing key with the same name (if any)
      and materializes the new one. The default behavior without RESTORE is to
      return an error if a key already exists.
      e5b5763f
  12. 06 Nov, 2012 1 commit
    • antirez's avatar
      Type mismatch errors are now prefixed with WRONGTYPE. · c4b0b685
      antirez authored
      So instead to reply with a generic error like:
      
      -ERR ... wrong kind of value ...
      
      now it replies with:
      
      -WRONGTYPE ... wrong kind of value ...
      
      This makes this particular error easy to check without resorting to
      (fragile) pattern matching of the error string (however the error string
      used to be consistent already).
      
      Client libraries should return a specific exeption type for this error.
      
      Most of the commit is about fixing unit tests.
      c4b0b685
  13. 01 Nov, 2012 3 commits
  14. 31 Oct, 2012 2 commits
  15. 30 Oct, 2012 2 commits
  16. 26 Oct, 2012 1 commit
  17. 25 Oct, 2012 2 commits
  18. 24 Oct, 2012 1 commit
  19. 22 Oct, 2012 3 commits
    • antirez's avatar
      A filed called slave_read_only added in INFO output. · 89e74abf
      antirez authored
      This was an important information missing from the INFO output in the
      replication section.
      
      It obviously reflects if the slave is read only or not.
      89e74abf
    • Schuster's avatar
      redis-check-dump now understands dumps produced by Redis 2.6 · e5f794ff
      Schuster authored
      (Commit message from @antirez as it was missign in the original commits,
      also the patch was modified a bit to still work with 2.4 dumps and to
      avoid if expressions that are always true due to checked types range)
      
      This commit changes redis-check-dump to account for new encodings and
      for the new MSTIME expire format. It also refactors the test for valid
      type into a function.
      
      The code is still compatible with Redis 2.4 generated dumps.
      
      This fixes issue #709.
      e5f794ff
    • antirez's avatar
      Default memory limit for 32bit instanced moved from 3.5 GB to 3 GB. · c2661ed7
      antirez authored
      In some system, notably osx, the 3.5 GB limit was too far and not able
      to prevent a crash for out of memory. The 3 GB limit works better and it
      is still a lot of memory within a 4 GB theorical limit so it's not going
      to bore anyone :-)
      
      This fixes issue #711
      c2661ed7