1. 27 May, 2013 3 commits
  2. 15 May, 2013 4 commits
  3. 24 Apr, 2013 2 commits
  4. 02 Apr, 2013 1 commit
    • antirez's avatar
      Throttle BGSAVE attempt on saving error. · d6b0c18c
      antirez authored
      When a BGSAVE fails, Redis used to flood itself trying to BGSAVE at
      every next cron call, that is either 10 or 100 times per second
      depending on configuration and server version.
      
      This commit does not allow a new automatic BGSAVE attempt to be
      performed before a few seconds delay (currently 5).
      
      This avoids both the auto-flood problem and filling the disk with
      logs at a serious rate.
      
      The five seconds limit, considering a log entry of 200 bytes, will use
      less than 4 MB of disk space per day that is reasonable, the sysadmin
      should notice before of catastrofic events especially since by default
      Redis will stop serving write queries after the first failed BGSAVE.
      
      This fixes issue #849
      d6b0c18c
  5. 28 Mar, 2013 1 commit
    • antirez's avatar
      DEBUG set-active-expire added. · 10d8e6a7
      antirez authored
      We need the ability to disable the activeExpireCycle() (active
      expired key collection) call for testing purposes.
      10d8e6a7
  6. 11 Mar, 2013 2 commits
  7. 06 Mar, 2013 1 commit
    • antirez's avatar
      API to lookup commands with their original name. · bc1b2e8f
      antirez authored
      A new server.orig_commands table was added to the server structure, this
      contains a copy of the commant table unaffected by rename-command
      statements in redis.conf.
      
      A new API lookupCommandOrOriginal() was added that checks both tables,
      new first, old later, so that rewriteClientCommandVector() and friends
      can lookup commands with their new or original name in order to fix the
      client->cmd pointer when the argument vector is renamed.
      
      This fixes the segfault of issue #986, but does not fix a wider range of
      problems resulting from renaming commands that actually operate on data
      and are registered into the AOF file or propagated to slaves... That is
      command renaming should be handled with care.
      bc1b2e8f
  8. 27 Feb, 2013 1 commit
  9. 26 Feb, 2013 1 commit
    • antirez's avatar
      Set process name in ps output to make operations safer. · ac3100bc
      antirez authored
      This commit allows Redis to set a process name that includes the binding
      address and the port number in order to make operations simpler.
      
      Redis children processes doing AOF rewrites or RDB saving change the
      name into redis-aof-rewrite and redis-rdb-bgsave respectively.
      
      This in general makes harder to kill the wrong process because of an
      error and makes simpler to identify saving children.
      
      This feature was suggested by Arnaud GRANAL in the Redis Google Group,
      Arnaud also pointed me to the setproctitle.c implementation includeed in
      this commit.
      
      This feature should work on all the Linux, OSX, and all the three major
      BSD systems.
      ac3100bc
  10. 12 Feb, 2013 5 commits
    • antirez's avatar
    • antirez's avatar
      5fe2577a
    • antirez's avatar
      700e5eb4
    • antirez's avatar
      Use replicationFeedSlaves() to send PING to slaves. · 01c21f99
      antirez authored
      A Redis master sends PING commands to slaves from time to time: doing
      this ensures that even if absence of writes, the master->slave channel
      remains active and the slave can feel the master presence, instead of
      closing the connection for timeout.
      
      This commit changes the way PINGs are sent to slaves in order to use the
      standard interface used to replicate all the other commands, that is,
      the function replicationFeedSlaves().
      
      With this change the stream of commands sent to every slave is exactly
      the same regardless of their exact state (Transferring RDB for first
      synchronization or slave already online). With the previous
      implementation the PING was only sent to online slaves, with the result
      that the output stream from master to slaves was not identical for all
      the slaves: this is a problem if we want to implement partial resyncs in
      the future using a global replication stream offset.
      
      TL;DR: this commit should not change the behaviour in practical terms,
      but is just something in preparation for partial resynchronization
      support.
      01c21f99
    • antirez's avatar
      Emit SELECT to slaves in a centralized way. · 5a35e485
      antirez authored
      Before this commit every Redis slave had its own selected database ID
      state. This was not actually useful as the emitted stream of commands
      is identical for all the slaves.
      
      Now the the currently selected database is a global state that is set to
      -1 when a new slave is attached, in order to force the SELECT command to
      be re-emitted for all the slaves.
      
      This change is useful in order to implement replication partial
      resynchronization in the future, as makes sure that the stream of
      commands received by slaves, including SELECT commands, are exactly the
      same for every slave connected, at any time.
      
      In this way we could have a global offset that can identify a specific
      piece of the master -> slaves stream of commands.
      5a35e485
  11. 11 Feb, 2013 1 commit
  12. 05 Feb, 2013 2 commits
    • antirez's avatar
      5f7dff4d
    • charsyam's avatar
      Turn off TCP_NODELAY on the slave socket after SYNC. · 1d80acae
      charsyam authored
      Further details from @antirez:
      
      It was reported by @StopForumSpam on Twitter that the Redis replication
      link was strangely using multiple TCP packets for multiple commands.
      This wastes a lot of bandwidth and is due to the TCP_NODELAY option we
      enable on the socket after accepting a new connection.
      
      However the master -> slave channel is a one-way channel since Redis
      replication is asynchronous, so there is no point in trying to reduce
      the latency, we should aim to reduce the bandwidth. For this reason this
      commit introduces the ability to disable the nagle algorithm on the
      socket after a successful SYNC.
      
      This feature is off by default because the delay can be up to 40
      milliseconds with normally configured Linux kernels.
      1d80acae
  13. 28 Jan, 2013 3 commits
    • antirez's avatar
      Keyspace events: it is now possible to select subclasses of events. · b9bc4f91
      antirez authored
      When keyspace events are enabled, the overhead is not sever but
      noticeable, so this commit introduces the ability to select subclasses
      of events in order to avoid to generate events the user is not
      interested in.
      
      The events can be selected using redis.conf or CONFIG SET / GET.
      b9bc4f91
    • antirez's avatar
      Fix decrRefCount() prototype from void to robj pointer. · 2825f21f
      antirez authored
      decrRefCount used to get its argument as a void* pointer in order to be
      used as destructor where a 'void free_object(void*)' prototype is
      expected. However this made simpler to introduce bugs by freeing the
      wrong pointer. This commit fixes the argument type and introduces a new
      wrapper called decrRefCountVoid() that can be used when the void*
      argument is needed.
      2825f21f
    • antirez's avatar
      Keyspace events notification API. · 212edbc4
      antirez authored
      212edbc4
  14. 19 Jan, 2013 1 commit
    • guiquanz's avatar
      Fixed many typos. · 1caf0939
      guiquanz authored
      Conflicts fixed, mainly because 2.8 has no cluster support / files:
      	00-RELEASENOTES
      	src/cluster.c
      	src/crc16.c
      	src/redis-trib.rb
      	src/redis.h
      1caf0939
  15. 15 Jan, 2013 1 commit
    • antirez's avatar
      CLIENT GETNAME and CLIENT SETNAME introduced. · 786bd393
      antirez authored
      Sometimes it is much simpler to debug complex Redis installations if it
      is possible to assign clients a name that is displayed in the CLIENT
      LIST output.
      
      This is the case, for example, for "leaked" connections. The ability to
      provide a name to the client makes it quite trivial to understand what
      is the part of the code implementing the client not releasing the
      resources appropriately.
      
      Behavior:
      
          CLIENT SETNAME: set a name for the client, or remove the current
                          name if an empty name is set.
          CLIENT GETNAME: get the current name, or a nil.
          CLIENT LIST: now displays the client name if any.
      
      Thanks to Mark Gravell for pushing this idea forward.
      786bd393
  16. 14 Dec, 2012 1 commit
    • antirez's avatar
      serverCron() frequency is now a runtime parameter (was REDIS_HZ). · a6d117b6
      antirez authored
      REDIS_HZ is the frequency our serverCron() function is called with.
      A more frequent call to this function results into less latency when the
      server is trying to handle very expansive background operations like
      mass expires of a lot of keys at the same time.
      
      Redis 2.4 used to have an HZ of 10. This was good enough with almost
      every setup, but the incremental key expiration algorithm was working a
      bit better under *extreme* pressure when HZ was set to 100 for Redis
      2.6.
      
      However for most users a latency spike of 30 milliseconds when million
      of keys are expiring at the same time is acceptable, on the other hand a
      default HZ of 100 in Redis 2.6 was causing idle instances to use some
      CPU time compared to Redis 2.4. The CPU usage was in the order of 0.3%
      for an idle instance, however this is a shame as more energy is consumed
      by the server, if not important resources.
      
      This commit introduces HZ as a runtime parameter, that can be queried by
      INFO or CONFIG GET, and can be modified with CONFIG SET. At the same
      time the default frequency is set back to 10.
      
      In this way we default to a sane value of 10, but allows users to
      easily switch to values up to 500 for near real-time applications if
      needed and if they are willing to pay this small CPU usage penalty.
      a6d117b6
  17. 02 Dec, 2012 1 commit
    • antirez's avatar
      Blocking POP: use a dictionary to store keys clinet side. · 07a9f854
      antirez authored
      To store the keys we block for during a blocking pop operation, in the
      case the client is blocked for more data to arrive, we used a simple
      linear array of redis objects, in the blockingState structure:
      
          robj **keys;
          int count;
      
      However in order to fix issue #801 we also use a dictionary in order to
      avoid to end in the blocked clients queue for the same key multiple
      times with the same client.
      
      The dictionary was only temporary, just to avoid duplicates, but since
      we create / destroy it there is no point in doing this duplicated work,
      so this commit simply use a dictionary as the main structure to store
      the keys we are blocked for. So instead of the previous fields we now
      just have:
      
          dict *keys;
      
      This simplifies the code and reduces the work done by the server during
      a blocking POP operation.
      07a9f854
  18. 29 Nov, 2012 1 commit
    • antirez's avatar
      Introduced the Build ID in INFO and --version output. · 3b71404d
      antirez authored
      The idea is to be able to identify a build in a unique way, so for
      instance after a bug report we can recognize that the build is the one
      of a popular Linux distribution and perform the debugging in the same
      environment.
      3b71404d
  19. 22 Nov, 2012 2 commits
    • antirez's avatar
      EVALSHA is now case insensitive. · d0570c96
      antirez authored
      EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
      Fixed using a case insensitive dictionary type for the sha -> script
      map used for replication of scripts.
      d0570c96
    • antirez's avatar
      Safer handling of MULTI/EXEC on errors. · 60f9dac6
      antirez authored
      After the transcation starts with a MULIT, the previous behavior was to
      return an error on problems such as maxmemory limit reached. But still
      to execute the transaction with the subset of queued commands on EXEC.
      
      While it is true that the client was able to check for errors
      distinguish QUEUED by an error reply, MULTI/EXEC in most client
      implementations uses pipelining for speed, so all the commands and EXEC
      are sent without caring about replies.
      
      With this change:
      
      1) EXEC fails if at least one command was not queued because of an
      error. The EXECABORT error is used.
      2) A generic error is always reported on EXEC.
      3) The client DISCARDs the MULTI state after a failed EXEC, otherwise
      pipelining multiple transactions would be basically impossible:
      After a failed EXEC the next transaction would be simply queued as
      the tail of the previous transaction.
      60f9dac6
  20. 08 Nov, 2012 1 commit
  21. 02 Nov, 2012 1 commit
  22. 01 Nov, 2012 1 commit
  23. 16 Oct, 2012 1 commit
    • antirez's avatar
      Fix MULTI / EXEC rendering in MONITOR output. · 21645232
      antirez authored
      Before of this commit it used to be like this:
      
      MULTI
      EXEC
      ... actual commands of the transaction ...
      
      Because after all that is the natural order of things. Transaction
      commands are queued and executed *only after* EXEC is called.
      
      However this makes debugging with MONITOR a mess, so the code was
      modified to provide a coherent output.
      
      What happens is that MULTI is rendered in the MONITOR output as far as
      possible, instead EXEC is propagated only after the transaction is
      executed, or even in the case it fails because of WATCH, so in this case
      you'll simply see:
      
      MULTI
      EXEC
      
      An empty transaction.
      21645232
  24. 27 Sep, 2012 1 commit
  25. 17 Sep, 2012 1 commit
    • antirez's avatar
      A reimplementation of blocking operation internals. · f444e2af
      antirez authored
      Redis provides support for blocking operations such as BLPOP or BRPOP.
      This operations are identical to normal LPOP and RPOP operations as long
      as there are elements in the target list, but if the list is empty they
      block waiting for new data to arrive to the list.
      
      All the clients blocked waiting for th same list are served in a FIFO
      way, so the first that blocked is the first to be served when there is
      more data pushed by another client into the list.
      
      The previous implementation of blocking operations was conceived to
      serve clients in the context of push operations. For for instance:
      
      1) There is a client "A" blocked on list "foo".
      2) The client "B" performs `LPUSH foo somevalue`.
      3) The client "A" is served in the context of the "B" LPUSH,
      synchronously.
      
      Processing things in a synchronous way was useful as if "A" pushes a
      value that is served by "B", from the point of view of the database is a
      NOP (no operation) thing, that is, nothing is replicated, nothing is
      written in the AOF file, and so forth.
      
      However later we implemented two things:
      
      1) Variadic LPUSH that could add multiple values to a list in the
      context of a single call.
      2) BRPOPLPUSH that was a version of BRPOP that also provided a "PUSH"
      side effect when receiving data.
      
      This forced us to make the synchronous implementation more complex. If
      client "B" is waiting for data, and "A" pushes three elemnents in a
      single call, we needed to propagate an LPUSH with a missing argument
      in the AOF and replication link. We also needed to make sure to
      replicate the LPUSH side of BRPOPLPUSH, but only if in turn did not
      happened to serve another blocking client into another list ;)
      
      This were complex but with a few of mutually recursive functions
      everything worked as expected... until one day we introduced scripting
      in Redis.
      
      Scripting + synchronous blocking operations = Issue #614.
      
      Basically you can't "rewrite" a script to have just a partial effect on
      the replicas and AOF file if the script happened to serve a few blocked
      clients.
      
      The solution to all this problems, implemented by this commit, is to
      change the way we serve blocked clients. Instead of serving the blocked
      clients synchronously, in the context of the command performing the PUSH
      operation, it is now an asynchronous and iterative process:
      
      1) If a key that has clients blocked waiting for data is the subject of
      a list push operation, We simply mark keys as "ready" and put it into a
      queue.
      2) Every command pushing stuff on lists, as a variadic LPUSH, a script,
      or whatever it is, is replicated verbatim without any rewriting.
      3) Every time a Redis command, a MULTI/EXEC block, or a script,
      completed its execution, we run the list of keys ready to serve blocked
      clients (as more data arrived), and process this list serving the
      blocked clients.
      4) As a result of "3" maybe more keys are ready again for other clients
      (as a result of BRPOPLPUSH we may have push operations), so we iterate
      back to step "3" if it's needed.
      
      The new code has a much simpler semantics, and a simpler to understand
      implementation, with the disadvantage of not being able to "optmize out"
      a PUSH+BPOP as a No OP.
      
      This commit will be tested with care before the final merge, more tests
      will be added likely.
      f444e2af