1. 12 Feb, 2013 1 commit
    • antirez's avatar
      Emit SELECT to slaves in a centralized way. · 5a35e485
      antirez authored
      Before this commit every Redis slave had its own selected database ID
      state. This was not actually useful as the emitted stream of commands
      is identical for all the slaves.
      
      Now the the currently selected database is a global state that is set to
      -1 when a new slave is attached, in order to force the SELECT command to
      be re-emitted for all the slaves.
      
      This change is useful in order to implement replication partial
      resynchronization in the future, as makes sure that the stream of
      commands received by slaves, including SELECT commands, are exactly the
      same for every slave connected, at any time.
      
      In this way we could have a global offset that can identify a specific
      piece of the master -> slaves stream of commands.
      5a35e485
  2. 11 Feb, 2013 2 commits
  3. 07 Feb, 2013 1 commit
  4. 05 Feb, 2013 2 commits
    • antirez's avatar
      5f7dff4d
    • charsyam's avatar
      Turn off TCP_NODELAY on the slave socket after SYNC. · 1d80acae
      charsyam authored
      Further details from @antirez:
      
      It was reported by @StopForumSpam on Twitter that the Redis replication
      link was strangely using multiple TCP packets for multiple commands.
      This wastes a lot of bandwidth and is due to the TCP_NODELAY option we
      enable on the socket after accepting a new connection.
      
      However the master -> slave channel is a one-way channel since Redis
      replication is asynchronous, so there is no point in trying to reduce
      the latency, we should aim to reduce the bandwidth. For this reason this
      commit introduces the ability to disable the nagle algorithm on the
      socket after a successful SYNC.
      
      This feature is off by default because the delay can be up to 40
      milliseconds with normally configured Linux kernels.
      1d80acae
  5. 28 Jan, 2013 3 commits
  6. 19 Jan, 2013 4 commits
    • antirez's avatar
      Whitelist SIGUSR1 to avoid auto-triggering errors. · 39f0a33f
      antirez authored
      This commit fixes issue #875 that was caused by the following events:
      
      1) There is an active child doing BGSAVE.
      2) flushall is called (or any other condition that makes Redis killing
      the saving child process).
      3) An error is sensed by Redis as the child exited with an error (killed
      by a singal), that stops accepting write commands until a BGSAVE happens
      to be executed with success.
      
      Whitelisting SIGUSR1 and making sure Redis always uses this signal in
      order to kill its own children fixes the issue.
      39f0a33f
    • antirez's avatar
      Clear server.shutdown_asap on failed shutdown. · 1e20c939
      antirez authored
      When a SIGTERM is received Redis schedules a shutdown. However if it
      fails to perform the shutdown it must be clear the shutdown_asap flag
      otehrwise it will try again and again possibly making the server
      unusable.
      1e20c939
    • antirez's avatar
      Slowlog: don't log EXEC but just the executed commands. · d766907c
      antirez authored
      The Redis Slow Log always used to log the slow commands executed inside
      a MULTI/EXEC block. However also EXEC was logged at the end, which is
      perfectly useless.
      
      Now EXEC is no longer logged and a test was added to test this behavior.
      
      This fixes issue #759.
      d766907c
    • guiquanz's avatar
      Fixed many typos. · 1caf0939
      guiquanz authored
      Conflicts fixed, mainly because 2.8 has no cluster support / files:
      	00-RELEASENOTES
      	src/cluster.c
      	src/crc16.c
      	src/redis-trib.rb
      	src/redis.h
      1caf0939
  7. 10 Jan, 2013 1 commit
  8. 14 Dec, 2012 1 commit
    • antirez's avatar
      serverCron() frequency is now a runtime parameter (was REDIS_HZ). · a6d117b6
      antirez authored
      REDIS_HZ is the frequency our serverCron() function is called with.
      A more frequent call to this function results into less latency when the
      server is trying to handle very expansive background operations like
      mass expires of a lot of keys at the same time.
      
      Redis 2.4 used to have an HZ of 10. This was good enough with almost
      every setup, but the incremental key expiration algorithm was working a
      bit better under *extreme* pressure when HZ was set to 100 for Redis
      2.6.
      
      However for most users a latency spike of 30 milliseconds when million
      of keys are expiring at the same time is acceptable, on the other hand a
      default HZ of 100 in Redis 2.6 was causing idle instances to use some
      CPU time compared to Redis 2.4. The CPU usage was in the order of 0.3%
      for an idle instance, however this is a shame as more energy is consumed
      by the server, if not important resources.
      
      This commit introduces HZ as a runtime parameter, that can be queried by
      INFO or CONFIG GET, and can be modified with CONFIG SET. At the same
      time the default frequency is set back to 10.
      
      In this way we default to a sane value of 10, but allows users to
      easily switch to values up to 500 for near real-time applications if
      needed and if they are willing to pay this small CPU usage penalty.
      a6d117b6
  9. 29 Nov, 2012 1 commit
    • antirez's avatar
      Introduced the Build ID in INFO and --version output. · 3b71404d
      antirez authored
      The idea is to be able to identify a build in a unique way, so for
      instance after a bug report we can recognize that the build is the one
      of a popular Linux distribution and perform the debugging in the same
      environment.
      3b71404d
  10. 22 Nov, 2012 2 commits
    • antirez's avatar
      EVALSHA is now case insensitive. · d0570c96
      antirez authored
      EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
      Fixed using a case insensitive dictionary type for the sha -> script
      map used for replication of scripts.
      d0570c96
    • antirez's avatar
      Safer handling of MULTI/EXEC on errors. · 60f9dac6
      antirez authored
      After the transcation starts with a MULIT, the previous behavior was to
      return an error on problems such as maxmemory limit reached. But still
      to execute the transaction with the subset of queued commands on EXEC.
      
      While it is true that the client was able to check for errors
      distinguish QUEUED by an error reply, MULTI/EXEC in most client
      implementations uses pipelining for speed, so all the commands and EXEC
      are sent without caring about replies.
      
      With this change:
      
      1) EXEC fails if at least one command was not queued because of an
      error. The EXECABORT error is used.
      2) A generic error is always reported on EXEC.
      3) The client DISCARDs the MULTI state after a failed EXEC, otherwise
      pipelining multiple transactions would be basically impossible:
      After a failed EXEC the next transaction would be simply queued as
      the tail of the previous transaction.
      60f9dac6
  11. 08 Nov, 2012 1 commit
  12. 06 Nov, 2012 1 commit
    • antirez's avatar
      Type mismatch errors are now prefixed with WRONGTYPE. · 46c5d396
      antirez authored
      So instead to reply with a generic error like:
      
      -ERR ... wrong kind of value ...
      
      now it replies with:
      
      -WRONGTYPE ... wrong kind of value ...
      
      This makes this particular error easy to check without resorting to
      (fragile) pattern matching of the error string (however the error string
      used to be consistent already).
      
      Client libraries should return a specific exeption type for this error.
      
      Most of the commit is about fixing unit tests.
      46c5d396
  13. 01 Nov, 2012 2 commits
    • antirez's avatar
      More robust handling of AOF rewrite child. · 64be5e36
      antirez authored
      After the wait3() syscall we used to do something like that:
      
          if (pid == server.rdb_child_pid) {
              backgroundSaveDoneHandler(exitcode,bysignal);
          } else {
              ....
          }
      
      So the AOF rewrite was handled in the else branch without actually
      checking if the pid really matches. This commit makes the check explicit
      and logs at WARNING level if the pid returned by wait3() does not match
      neither the RDB or AOF rewrite child.
      64be5e36
    • Yecheng Fu's avatar
      fix typo in comments (redis.c, networking.c) · fecc8797
      Yecheng Fu authored
      fecc8797
  14. 22 Oct, 2012 2 commits
  15. 16 Oct, 2012 1 commit
    • antirez's avatar
      Fix MULTI / EXEC rendering in MONITOR output. · 21645232
      antirez authored
      Before of this commit it used to be like this:
      
      MULTI
      EXEC
      ... actual commands of the transaction ...
      
      Because after all that is the natural order of things. Transaction
      commands are queued and executed *only after* EXEC is called.
      
      However this makes debugging with MONITOR a mess, so the code was
      modified to provide a coherent output.
      
      What happens is that MULTI is rendered in the MONITOR output as far as
      possible, instead EXEC is propagated only after the transaction is
      executed, or even in the case it fails because of WATCH, so in this case
      you'll simply see:
      
      MULTI
      EXEC
      
      An empty transaction.
      21645232
  16. 11 Oct, 2012 1 commit
  17. 05 Oct, 2012 1 commit
    • antirez's avatar
      Warn when configured maxmemory value seems odd. · 05e06e15
      antirez authored
      This commit warns the user with a log at "warning" level if:
      
      1) After the server startup the maxmemory limit was found to be < 1MB.
      2) After a CONFIG SET command modifying the maxmemory setting the limit
      is set to a value that is smaller than the currently used memory.
      
      The behaviour of the Redis server is unmodified, and this wil not make
      the CONFIG SET command or a wrong configuration in redis.conf less
      likely to create problems, but at least this will make aware most users
      about a possbile error they committed without resorting to external
      help.
      
      However no warning is issued if, as a result of loading the AOF or RDB
      file, we are very near the maxmemory setting, or key eviction will be
      needed in order to go under the specified maxmemory setting. The reason
      is that in servers configured as a cache with an aggressive
      maxmemory-policy most of the times restarting the server will cause this
      condition to happen if persistence is not switched off.
      
      This fixes issue #429.
      05e06e15
  18. 27 Sep, 2012 4 commits
  19. 21 Sep, 2012 1 commit
    • antirez's avatar
      Added the SRANDMEMBER key <count> variant. · dd947715
      antirez authored
      SRANDMEMBER called with just the key argument can just return a single
      random element from a Redis Set. However many users need to return
      multiple unique elements from a Set, this is not a trivial problem to
      handle in the client side, and for truly good performance a C
      implementation was required.
      
      After many requests for this feature it was finally implemented.
      
      The problem implementing this command is the strategy to follow when
      the number of elements the user asks for is near to the number of
      elements that are already inside the set. In this case asking random
      elements to the dictionary API, and trying to add it to a temporary set,
      may result into an extremely poor performance, as most add operations
      will be wasted on duplicated elements.
      
      For this reason this implementation uses a different strategy in this
      case: the Set is copied, and random elements are returned to reach the
      specified count.
      
      The code actually uses 4 different algorithms optimized for the
      different cases.
      
      If the count is negative, the command changes behavior and allows for
      duplicated elements in the returned subset.
      dd947715
  20. 17 Sep, 2012 1 commit
    • antirez's avatar
      A reimplementation of blocking operation internals. · f444e2af
      antirez authored
      Redis provides support for blocking operations such as BLPOP or BRPOP.
      This operations are identical to normal LPOP and RPOP operations as long
      as there are elements in the target list, but if the list is empty they
      block waiting for new data to arrive to the list.
      
      All the clients blocked waiting for th same list are served in a FIFO
      way, so the first that blocked is the first to be served when there is
      more data pushed by another client into the list.
      
      The previous implementation of blocking operations was conceived to
      serve clients in the context of push operations. For for instance:
      
      1) There is a client "A" blocked on list "foo".
      2) The client "B" performs `LPUSH foo somevalue`.
      3) The client "A" is served in the context of the "B" LPUSH,
      synchronously.
      
      Processing things in a synchronous way was useful as if "A" pushes a
      value that is served by "B", from the point of view of the database is a
      NOP (no operation) thing, that is, nothing is replicated, nothing is
      written in the AOF file, and so forth.
      
      However later we implemented two things:
      
      1) Variadic LPUSH that could add multiple values to a list in the
      context of a single call.
      2) BRPOPLPUSH that was a version of BRPOP that also provided a "PUSH"
      side effect when receiving data.
      
      This forced us to make the synchronous implementation more complex. If
      client "B" is waiting for data, and "A" pushes three elemnents in a
      single call, we needed to propagate an LPUSH with a missing argument
      in the AOF and replication link. We also needed to make sure to
      replicate the LPUSH side of BRPOPLPUSH, but only if in turn did not
      happened to serve another blocking client into another list ;)
      
      This were complex but with a few of mutually recursive functions
      everything worked as expected... until one day we introduced scripting
      in Redis.
      
      Scripting + synchronous blocking operations = Issue #614.
      
      Basically you can't "rewrite" a script to have just a partial effect on
      the replicas and AOF file if the script happened to serve a few blocked
      clients.
      
      The solution to all this problems, implemented by this commit, is to
      change the way we serve blocked clients. Instead of serving the blocked
      clients synchronously, in the context of the command performing the PUSH
      operation, it is now an asynchronous and iterative process:
      
      1) If a key that has clients blocked waiting for data is the subject of
      a list push operation, We simply mark keys as "ready" and put it into a
      queue.
      2) Every command pushing stuff on lists, as a variadic LPUSH, a script,
      or whatever it is, is replicated verbatim without any rewriting.
      3) Every time a Redis command, a MULTI/EXEC block, or a script,
      completed its execution, we run the list of keys ready to serve blocked
      clients (as more data arrived), and process this list serving the
      blocked clients.
      4) As a result of "3" maybe more keys are ready again for other clients
      (as a result of BRPOPLPUSH we may have push operations), so we iterate
      back to step "3" if it's needed.
      
      The new code has a much simpler semantics, and a simpler to understand
      implementation, with the disadvantage of not being able to "optmize out"
      a PUSH+BPOP as a No OP.
      
      This commit will be tested with care before the final merge, more tests
      will be added likely.
      f444e2af
  21. 10 Sep, 2012 1 commit
  22. 04 Sep, 2012 1 commit
    • antirez's avatar
      Scripting: Force SORT BY constant determinism inside SORT itself. · 5ddee9b7
      antirez authored
      SORT is able to return (faster than when ordering) unordered output if
      the "BY" clause is used with a constant value. However we try to play
      well with scripting requirements of determinism providing always sorted
      outputs when SORT (and other similar commands) are called by Lua
      scripts.
      
      However we used the general mechanism in place in scripting in order to
      reorder SORT output, that is, if the command has the "S" flag set, the
      Lua scripting engine will take an additional step when converting a
      multi bulk reply to Lua value, calling a Lua sorting function.
      
      This is suboptimal as we can do it faster inside SORT itself.
      This is also broken as issue #545 shows us: basically when SORT is used
      with a constant BY, and additionally also GET is used, the Lua scripting
      engine was trying to order the output as a flat array, while it was
      actually a list of key-value pairs.
      
      What we do know is to recognized if the caller of SORT is the Lua client
      (since we can check this using the REDIS_LUA_CLIENT flag). If so, and if
      a "don't sort" condition is triggered by the BY option with a constant
      string, we force the lexicographical sorting.
      
      This commit fixes this bug and improves the performance, and at the same
      time simplifies the implementation. This does not mean I'm smart today,
      it means I was stupid when I committed the original implementation ;)
      5ddee9b7
  23. 31 Aug, 2012 1 commit
    • antirez's avatar
      Sentinel: Redis-side support for slave priority. · 48d26a48
      antirez authored
      A Redis slave can now be configured with a priority, that is an integer
      number that is shown in INFO output and can be get and set using the
      redis.conf file or the CONFIG GET/SET command.
      
      This field is used by Sentinel during slave election. A slave with lower
      priority is preferred. A slave with priority zero is never elected (and
      is considered to be impossible to elect even if it is the only slave
      available).
      
      A next commit will add support in the Sentinel side as well.
      48d26a48
  24. 28 Aug, 2012 1 commit
    • antirez's avatar
      Incrementally flush RDB on disk while loading it from a master. · 13732168
      antirez authored
      This fixes issue #539.
      
      Basically if there is enough free memory the OS may buffer the RDB file
      that the slave transfers on disk from the master. The file may
      actually be flused on disk at once by the operating system when it gets
      closed by Redis, causing the close system call to block for a long time.
      
      This patch is a modified version of one provided by yoav-steinberg of
      @garantiadata (the original version was posted in the issue #539
      comments), and tries to flush the OS buffers incrementally (every 8 MB
      of loaded data).
      13732168
  25. 24 Aug, 2012 1 commit
    • antirez's avatar
      Better Out of Memory handling. · 5de75120
      antirez authored
      The previous implementation of zmalloc.c was not able to handle out of
      memory in an application-specific way. It just logged an error on
      standard error, and aborted.
      
      The result was that in the case of an actual out of memory in Redis
      where malloc returned NULL (In Linux this actually happens under
      specific overcommit policy settings and/or with no or little swap
      configured) the error was not properly logged in the Redis log.
      
      This commit fixes this problem, fixing issue #509.
      Now the out of memory is properly reported in the Redis log and a stack
      trace is generated.
      
      The approach used is to provide a configurable out of memory handler
      to zmalloc (otherwise the default one logging the event on the
      standard output is used).
      5de75120
  26. 31 Jul, 2012 1 commit
    • Saj Goonatilleke's avatar
      New in INFO: aof_last_bgrewrite_status · f00b0844
      Saj Goonatilleke authored
      Behaves like rdb_last_bgsave_status -- even down to reporting 'ok' when
      no rewrite has been done yet.  (You might want to check that
      aof_last_rewrite_time_sec is not -1.)
      f00b0844
  27. 22 Jul, 2012 1 commit
    • antirez's avatar
      Allow Pub/Sub in contexts where other commands are blocked. · 3bb3f125
      antirez authored
      Redis loading data from disk, and a Redis slave disconnected from its
      master with serve-stale-data disabled, are two conditions where
      commands are normally refused by Redis, returning an error.
      
      However there is no reason to disable Pub/Sub commands as well, given
      that this layer does not interact with the dataset. To allow Pub/Sub in
      as many contexts as possible is especially interesting now that Redis
      Sentinel uses Pub/Sub of a Redis master as a communication channel
      between Sentinels.
      
      This commit allows Pub/Sub to be used in the above two contexts where
      it was previously denied.
      3bb3f125