1. 02 Dec, 2012 2 commits
    • antirez's avatar
      Blocking POP: use a dictionary to store keys clinet side. · 07a9f854
      antirez authored
      To store the keys we block for during a blocking pop operation, in the
      case the client is blocked for more data to arrive, we used a simple
      linear array of redis objects, in the blockingState structure:
      
          robj **keys;
          int count;
      
      However in order to fix issue #801 we also use a dictionary in order to
      avoid to end in the blocked clients queue for the same key multiple
      times with the same client.
      
      The dictionary was only temporary, just to avoid duplicates, but since
      we create / destroy it there is no point in doing this duplicated work,
      so this commit simply use a dictionary as the main structure to store
      the keys we are blocked for. So instead of the previous fields we now
      just have:
      
          dict *keys;
      
      This simplifies the code and reduces the work done by the server during
      a blocking POP operation.
      07a9f854
    • antirez's avatar
      Client should not block multiple times on the same key. · 64e69c0d
      antirez authored
      Sending a command like:
      
      BLPOP foo foo foo foo 0
      
      Resulted into a crash before this commit since the client ended being
      inserted in the waiting list for this key multiple times.
      This resulted into the function handleClientsBlockedOnLists() to fail
      because we have code like that:
      
          if (de) {
              list *clients = dictGetVal(de);
              int numclients = listLength(clients);
      
              while(numclients--) {
                  listNode *clientnode = listFirst(clients);
      
                  /* server clients here... */
              }
          }
      
      The code to serve clients used to remove the served client from the
      waiting list, so if a client is blocking multiple times, eventually the
      call to listFirst() will return NULL or worse will access random memory
      since the list may no longer exist as it is removed by the function
      unblockClientWaitingData() if there are no more clients waiting for this
      list.
      
      To avoid making the rest of the implementation more complex, this commit
      modifies blockForKeys() so that a client will be put just a single time
      into the waiting list for a given key.
      
      Since it is Saturday, I hope this fixes issue #801.
      64e69c0d
  2. 30 Nov, 2012 2 commits
    • antirez's avatar
      SDIFF is now able to select between two algorithms for speed. · ccc974d9
      antirez authored
      SDIFF used an algorithm that was O(N) where N is the total number
      of elements of all the sets involved in the operation.
      
      The algorithm worked like that:
      
      ALGORITHM 1:
      
      1) For the first set, add all the members to an auxiliary set.
      2) For all the other sets, remove all the members of the set from the
      auxiliary set.
      
      So it is an O(N) algorithm where N is the total number of elements in
      all the sets involved in the diff operation.
      
      Cristobal Viedma suggested to modify the algorithm to the following:
      
      ALGORITHM 2:
      
      1) Iterate all the elements of the first set.
      2) For every element, check if the element also exists in all the other
      remaining sets.
      3) Add the element to the auxiliary set only if it does not exist in any
      of the other sets.
      
      The complexity of this algorithm on the worst case is O(N*M) where N is
      the size of the first set and M the total number of sets involved in the
      operation.
      
      However when there are elements in common, with this algorithm we stop
      the computation for a given element as long as we find a duplicated
      element into another set.
      
      I (antirez) added an additional step to algorithm 2 to make it faster,
      that is to sort the set to subtract from the biggest to the
      smallest, so that it is more likely to find a duplicate in a larger sets
      that are checked before the smaller ones.
      
      WHAT IS BETTER?
      
      None of course, for instance if the first set is much larger than the
      other sets the second algorithm does a lot more work compared to the
      first algorithm.
      
      Similarly if the first set is much smaller than the other sets, the
      original algorithm will less work.
      
      So this commit makes Redis able to guess the number of operations
      required by each algorithm, and select the best at runtime according
      to the input received.
      
      However, since the second algorithm has better constant times and can do
      less work if there are duplicated elements, an advantage is given to the
      second algorithm.
      ccc974d9
    • antirez's avatar
      2572bb13
  3. 29 Nov, 2012 2 commits
    • antirez's avatar
      Introduced the Build ID in INFO and --version output. · 3b71404d
      antirez authored
      The idea is to be able to identify a build in a unique way, so for
      instance after a bug report we can recognize that the build is the one
      of a popular Linux distribution and perform the debugging in the same
      environment.
      3b71404d
    • antirez's avatar
      On crash memory test rewrote so that it actaully works. · cd99c14e
      antirez authored
      1) We no longer test location by location, otherwise the CPU write cache
      completely makes our business useless.
      2) We still need a memory test that operates in steps from the first to
      the last location in order to never hit the cache, but that is still
      able to retain the memory content.
      
      This was tested using a Linux box containing a bad memory module with a
      zingle bit error (always zero).
      
      So the final solution does has an error propagation step that is:
      
      1) Invert bits at every location.
      2) Swap adiacent locations.
      3) Swap adiacent locations again.
      4) Invert bits at every location.
      5) Swap adiacent locations.
      6) Swap adiacent locations again.
      
      Before and after these steps, and after step 4, a CRC64 checksum is computed.
      If the three CRC64 checksums don't match, a memory error was detected.
      cd99c14e
  4. 28 Nov, 2012 3 commits
  5. 22 Nov, 2012 6 commits
    • antirez's avatar
      EVALSHA is now case insensitive. · d0570c96
      antirez authored
      EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
      Fixed using a case insensitive dictionary type for the sha -> script
      map used for replication of scripts.
      d0570c96
    • antirez's avatar
      Fix integer overflow in zunionInterGenericCommand(). · cc85ed41
      antirez authored
      This fixes issue #761.
      cc85ed41
    • antirez's avatar
      Safer handling of MULTI/EXEC on errors. · 60f9dac6
      antirez authored
      After the transcation starts with a MULIT, the previous behavior was to
      return an error on problems such as maxmemory limit reached. But still
      to execute the transaction with the subset of queued commands on EXEC.
      
      While it is true that the client was able to check for errors
      distinguish QUEUED by an error reply, MULTI/EXEC in most client
      implementations uses pipelining for speed, so all the commands and EXEC
      are sent without caring about replies.
      
      With this change:
      
      1) EXEC fails if at least one command was not queued because of an
      error. The EXECABORT error is used.
      2) A generic error is always reported on EXEC.
      3) The client DISCARDs the MULTI state after a failed EXEC, otherwise
      pipelining multiple transactions would be basically impossible:
      After a failed EXEC the next transaction would be simply queued as
      the tail of the previous transaction.
      60f9dac6
    • antirez's avatar
      Make bio.c threads killable ASAP if needed. · dff1ec4a
      antirez authored
      We use this new bio.c feature in order to stop our I/O threads if there
      is a memory test to do on crash. In this case we don't want anything
      else than the main thread to run, otherwise the other threads may mess
      with the heap and the memory test will report a false positive.
      dff1ec4a
    • antirez's avatar
      Fast memory test on Redis crash. · d4b5c480
      antirez authored
      d4b5c480
    • antirez's avatar
      dd905417
  6. 19 Nov, 2012 2 commits
    • antirez's avatar
      Children creating AOF or RDB files now report memory used by COW. · c342b075
      antirez authored
      Finally Redis is able to report the amount of memory used by
      copy-on-write while saving an RDB or writing an AOF file in background.
      
      Note that this information is currently only logged (at NOTICE level)
      and not shown in INFO because this is less trivial (but surely doable
      with some minor form of interprocess communication).
      
      The reason we can't capture this information on the parent before we
      call wait3() is that the Linux kernel will release the child memory
      ASAP, and only retain the minimal state for the process that is useful
      to report the child termination to the parent.
      
      The COW size is obtained by summing all the Private_Dirty fields found
      in the "smap" file inside the proc filesystem for the process.
      
      All this is Linux specific and is not available on other systems.
      c342b075
    • antirez's avatar
      zmalloc_get_private_dirty() function added (Linux only). · 1d8af607
      antirez authored
      For non Linux systmes it just returns 0.
      
      This function is useful to estimate copy-on-write because of childs
      saving stuff on disk.
      1d8af607
  7. 14 Nov, 2012 1 commit
  8. 12 Nov, 2012 1 commit
    • antirez's avatar
      TTL API change: TTL returns -2 for non existing keys. · 50c41de7
      antirez authored
      The previous behavior was to return -1 if:
      
      1) Existing key but without an expire set.
      2) Non existing key.
      
      Now the second case is handled in a different, and TTL will return -2
      if the key does not exist at all.
      
      PTTL follows the same behavior as well.
      50c41de7
  9. 08 Nov, 2012 1 commit
  10. 06 Nov, 2012 1 commit
    • antirez's avatar
      Type mismatch errors are now prefixed with WRONGTYPE. · 46c5d396
      antirez authored
      So instead to reply with a generic error like:
      
      -ERR ... wrong kind of value ...
      
      now it replies with:
      
      -WRONGTYPE ... wrong kind of value ...
      
      This makes this particular error easy to check without resorting to
      (fragile) pattern matching of the error string (however the error string
      used to be consistent already).
      
      Client libraries should return a specific exeption type for this error.
      
      Most of the commit is about fixing unit tests.
      46c5d396
  11. 02 Nov, 2012 1 commit
  12. 01 Nov, 2012 3 commits
  13. 31 Oct, 2012 1 commit
    • antirez's avatar
      Invert two sides of if expression in SET to avoid a lookup. · 9fab63b3
      antirez authored
      Because of the short circuit behavior of && inverting the two sides of
      the if expression avoids an hash table lookup if the non-EX variant of
      SET is called.
      
      Thanks to Weibin Yao (@yaoweibin on github) for spotting this.
      9fab63b3
  14. 30 Oct, 2012 6 commits
  15. 22 Oct, 2012 7 commits
    • antirez's avatar
      2.7.0 branch created as a fork of 2.6.0. · f7b42b17
      antirez authored
      f7b42b17
    • antirez's avatar
      Redis 2.6.0 · 5eec376c
      antirez authored
      5eec376c
    • antirez's avatar
      A filed called slave_read_only added in INFO output. · 99d7dbe6
      antirez authored
      This was an important information missing from the INFO output in the
      replication section.
      
      It obviously reflects if the slave is read only or not.
      99d7dbe6
    • Greg Hurrell's avatar
      Fix (cosmetic) typos in dict.h · a61705dd
      Greg Hurrell authored
      a61705dd
    • Schuster's avatar
      redis-check-dump now understands dumps produced by Redis 2.6 · 16144589
      Schuster authored
      (Commit message from @antirez as it was missign in the original commits,
      also the patch was modified a bit to still work with 2.4 dumps and to
      avoid if expressions that are always true due to checked types range)
      
      This commit changes redis-check-dump to account for new encodings and
      for the new MSTIME expire format. It also refactors the test for valid
      type into a function.
      
      The code is still compatible with Redis 2.4 generated dumps.
      
      This fixes issue #709.
      16144589
    • antirez's avatar
      Default memory limit for 32bit instanced moved from 3.5 GB to 3 GB. · a25b25f4
      antirez authored
      In some system, notably osx, the 3.5 GB limit was too far and not able
      to prevent a crash for out of memory. The 3 GB limit works better and it
      is still a lot of memory within a 4 GB theorical limit so it's not going
      to bore anyone :-)
      
      This fixes issue #711
      a25b25f4
    • antirez's avatar
      Differentiate SCRIPT KILL error replies. · ab551808
      antirez authored
      When calling SCRIPT KILL currently you can get two errors:
      
      * No script in timeout (busy) state.
      * The script already performed a write.
      
      It is useful to be able to distinguish the two errors, but right now both
      start with "ERR" prefix, so string matching (that is fragile) must be used.
      
      This commit introduces two different prefixes.
      
      -NOTBUSY and -UNKILLABLE respectively to reply with an error when no
      script is busy at the moment, and when the script already executed a
      write operation and can not be killed.
      ab551808
  16. 18 Oct, 2012 1 commit
    • NanXiao's avatar
      Update src/redis-benchmark.c · a03c3270
      NanXiao authored
      The code of current implementation:
      
      if (c->pending == 0) clientDone(c);
      In clientDone function, the c's memory has been freed, then the loop will continue: while(c->pending). The memory of c has been freed now, so c->pending is invalid (c is an invalid pointer now), and this will cause memory dump in some platforams(eg: Solaris).
      
      So I think the code should be modified as:
      if (c->pending == 0)
      {
      clientDone(c);
      break;
      }
      and this will not lead to while(c->pending).
      a03c3270