1. 25 Feb, 2014 1 commit
  2. 10 Dec, 2013 1 commit
  3. 17 Sep, 2013 1 commit
  4. 07 Aug, 2013 6 commits
    • antirez's avatar
      Add per-db average TTL information in INFO output. · 487e6655
      antirez authored
      Example:
      
      db0:keys=221913,expires=221913,avg_ttl=655
      
      The algorithm uses a running average with only two samples (current and
      previous). Keys found to be expired are considered at TTL zero even if
      the actual TTL can be negative.
      
      The TTL is reported in milliseconds.
      487e6655
    • antirez's avatar
      activeExpireCycle(): fix about fast cycle early start. · 2e4ec793
      antirez authored
      We don't want to repeat a fast cycle too soon, the previous code was
      broken, we need to wait two times the period *since* the start of the
      previous cycle in order to avoid there is an even space between cycles:
      
      .-> start                   .-> second start
      |                           |
      +-------------+-------------+--------------+
      | first cycle |    pause    | second cycle |
      +-------------+-------------+--------------+
      
      The second and first start must be PERIOD*2 useconds apart hence the *2
      in the new code.
      2e4ec793
    • antirez's avatar
      Some activeExpireCycle() refactoring. · 1bcf296c
      antirez authored
      1bcf296c
    • antirez's avatar
      97f5e867
    • antirez's avatar
      Darft #2 for key collection algo: more improvements. · d9a4fd0f
      antirez authored
      This commit makes the fast collection cycle time configurable, at
      the same time it does not allow to run a new fast collection cycle
      for the same amount of time as the max duration of the fast
      collection cycle.
      d9a4fd0f
    • antirez's avatar
      Draft #1 of a new expired keys collection algorithm. · ab782615
      antirez authored
      The main idea here is that when we are no longer to expire keys at the
      rate the are created, we can't block more in the normal expire cycle as
      this would result in too big latency spikes.
      
      For this reason the commit introduces a "fast" expire cycle that does
      not run for more than 1 millisecond but is called in the beforeSleep()
      hook of the event loop, so much more often, and with a frequency bound
      to the frequency of executed commnads.
      
      The fast expire cycle is only called when the standard expiration
      algorithm runs out of time, that is, consumed more than
      REDIS_EXPIRELOOKUPS_TIME_PERC of CPU in a given cycle without being able
      to take the number of already expired keys that are yet not collected
      to a number smaller than 25% of the number of keys.
      
      You can test this commit with different loads, but a simple way is to
      use the following:
      
      Extreme load with pipelining:
      
      redis-benchmark -r 100000000 -n 100000000  \
              -P 32 set ele:rand:000000000000 foo ex 2
      
      Remove the -P32 in order to avoid the pipelining for a more real-world
      load.
      
      In another terminal tab you can monitor the Redis behavior with:
      
      redis-cli -i 0.1 -r -1 info keyspace
      
      and
      
      redis-cli --latency-history
      
      Note: this commit will make Redis printing a lot of debug messages, it
      is not a good idea to use it in production.
      ab782615
  5. 24 Apr, 2013 1 commit
  6. 19 Apr, 2013 1 commit
  7. 02 Apr, 2013 1 commit
    • antirez's avatar
      Throttle BGSAVE attempt on saving error. · ed2d9881
      antirez authored
      When a BGSAVE fails, Redis used to flood itself trying to BGSAVE at
      every next cron call, that is either 10 or 100 times per second
      depending on configuration and server version.
      
      This commit does not allow a new automatic BGSAVE attempt to be
      performed before a few seconds delay (currently 5).
      
      This avoids both the auto-flood problem and filling the disk with
      logs at a serious rate.
      
      The five seconds limit, considering a log entry of 200 bytes, will use
      less than 4 MB of disk space per day that is reasonable, the sysadmin
      should notice before of catastrofic events especially since by default
      Redis will stop serving write queries after the first failed BGSAVE.
      
      This fixes issue #849
      ed2d9881
  8. 28 Mar, 2013 2 commits
  9. 27 Mar, 2013 1 commit
  10. 26 Mar, 2013 1 commit
  11. 13 Mar, 2013 2 commits
  12. 12 Mar, 2013 1 commit
  13. 11 Mar, 2013 11 commits
    • antirez's avatar
      activeExpireCycle() smarter with many DBs and under expire pressure. · 48f4f771
      antirez authored
      activeExpireCycle() tries to test just a few DBs per iteration so that
      it scales if there are many configured DBs in the Redis instance.
      However this commit makes it a bit smarter when one a few of those DBs
      are under expiration pressure and there are many many keys to expire.
      
      What we do is to remember if in the last iteration had to return because
      we ran out of time. In that case the next iteration we'll test all the
      configured DBs so that we are sure we'll test again the DB under
      pressure.
      
      Before of this commit after some mass-expire in a given DB the function
      tested just a few of the next DBs, possibly empty, a few per iteration,
      so it took a long time for the function to reach again the DB under
      pressure. This resulted in a lot of memory being used by already expired
      keys and never accessed by clients.
      48f4f771
    • antirez's avatar
      b3281e93
    • antirez's avatar
      2677c97d
    • antirez's avatar
    • antirez's avatar
      da2dd899
    • antirez's avatar
      activeExpireCycle(): process only a small number of DBs per iteration. · 13f84841
      antirez authored
      This small number of DBs is set to 16 so actually in the default
      configuraiton Redis should behave exactly like in the past.
      However the difference is that when the user configures a very large
      number of DBs we don't do an O(N) operation, consuming a non trivial
      amount of CPU per serverCron() iteration.
      13f84841
    • antirez's avatar
      ef3a95fa
    • antirez's avatar
      Only resize/rehash a few databases per cron iteration. · 665b819e
      antirez authored
      This is the first step to lower the CPU usage when many databases are
      configured. The other is to also process a limited number of DBs per
      call in the active expire cycle.
      665b819e
    • antirez's avatar
      bb562a64
    • antirez's avatar
      e166abad
    • antirez's avatar
      serverCron() frequency is now a runtime parameter (was REDIS_HZ). · aec5ea5d
      antirez authored
      REDIS_HZ is the frequency our serverCron() function is called with.
      A more frequent call to this function results into less latency when the
      server is trying to handle very expansive background operations like
      mass expires of a lot of keys at the same time.
      
      Redis 2.4 used to have an HZ of 10. This was good enough with almost
      every setup, but the incremental key expiration algorithm was working a
      bit better under *extreme* pressure when HZ was set to 100 for Redis
      2.6.
      
      However for most users a latency spike of 30 milliseconds when million
      of keys are expiring at the same time is acceptable, on the other hand a
      default HZ of 100 in Redis 2.6 was causing idle instances to use some
      CPU time compared to Redis 2.4. The CPU usage was in the order of 0.3%
      for an idle instance, however this is a shame as more energy is consumed
      by the server, if not important resources.
      
      This commit introduces HZ as a runtime parameter, that can be queried by
      INFO or CONFIG GET, and can be modified with CONFIG SET. At the same
      time the default frequency is set back to 10.
      
      In this way we default to a sane value of 10, but allows users to
      easily switch to values up to 500 for near real-time applications if
      needed and if they are willing to pay this small CPU usage penalty.
      aec5ea5d
  14. 06 Mar, 2013 2 commits
    • antirez's avatar
      API to lookup commands with their original name. · e7a61d28
      antirez authored
      A new server.orig_commands table was added to the server structure, this
      contains a copy of the commant table unaffected by rename-command
      statements in redis.conf.
      
      A new API lookupCommandOrOriginal() was added that checks both tables,
      new first, old later, so that rewriteClientCommandVector() and friends
      can lookup commands with their new or original name in order to fix the
      client->cmd pointer when the argument vector is renamed.
      
      This fixes the segfault of issue #986, but does not fix a wider range of
      problems resulting from renaming commands that actually operate on data
      and are registered into the AOF file or propagated to slaves... That is
      command renaming should be handled with care.
      e7a61d28
    • antirez's avatar
      Allow AUTH while loading the DB in memory. · a3273dbf
      antirez authored
      While Redis is loading the AOF or RDB file in memory only a subset of
      commands are allowed. This commit adds AUTH to this subset.
      a3273dbf
  15. 04 Mar, 2013 1 commit
  16. 11 Feb, 2013 2 commits
  17. 07 Feb, 2013 1 commit
  18. 05 Feb, 2013 2 commits
    • antirez's avatar
      cf0191dc
    • charsyam's avatar
      Turn off TCP_NODELAY on the slave socket after SYNC. · 45b1b2f7
      charsyam authored
      Further details from @antirez:
      
      It was reported by @StopForumSpam on Twitter that the Redis replication
      link was strangely using multiple TCP packets for multiple commands.
      This wastes a lot of bandwidth and is due to the TCP_NODELAY option we
      enable on the socket after accepting a new connection.
      
      However the master -> slave channel is a one-way channel since Redis
      replication is asynchronous, so there is no point in trying to reduce
      the latency, we should aim to reduce the bandwidth. For this reason this
      commit introduces the ability to disable the nagle algorithm on the
      socket after a successful SYNC.
      
      This feature is off by default because the delay can be up to 40
      milliseconds with normally configured Linux kernels.
      45b1b2f7
  19. 19 Jan, 2013 2 commits
    • antirez's avatar
      Whitelist SIGUSR1 to avoid auto-triggering errors. · da540228
      antirez authored
      This commit fixes issue #875 that was caused by the following events:
      
      1) There is an active child doing BGSAVE.
      2) flushall is called (or any other condition that makes Redis killing
      the saving child process).
      3) An error is sensed by Redis as the child exited with an error (killed
      by a singal), that stops accepting write commands until a BGSAVE happens
      to be executed with success.
      
      Whitelisting SIGUSR1 and making sure Redis always uses this signal in
      order to kill its own children fixes the issue.
      da540228
    • antirez's avatar
      Clear server.shutdown_asap on failed shutdown. · 9e5b7016
      antirez authored
      When a SIGTERM is received Redis schedules a shutdown. However if it
      fails to perform the shutdown it must be clear the shutdown_asap flag
      otehrwise it will try again and again possibly making the server
      unusable.
      9e5b7016