1. 24 Mar, 2014 5 commits
    • Matt Stancliff's avatar
      Fix maxclients error handling · c138631c
      Matt Stancliff authored
      Everywhere in the Redis code base, maxclients is treated
      as an int with (int)maxclients or `maxclients = atoi(source)`,
      so let's make maxclients an int.
      
      This fixes a bug where someone could specify a negative maxclients
      on startup and it would work (as well as set maxclients very high)
      because:
      
          unsigned int maxclients;
          char *update = "-300";
          maxclients = atoi(update);
          if (maxclients < 1) goto fail;
      
      But, (maxclients < 1) can only catch the case when maxclients
      is exactly 0.  maxclients happily sets itself to -300, which isn't
      -300, but rather 4294966996, which isn't < 1, so... everything
      "worked."
      
      maxclients config parsing checks for the case of < 1, but maxclients
      CONFIG SET parsing was checking for case of < 0 (allowing
      maxclients to be set to 0).  CONFIG SET parsing is now updated to
      match config parsing of < 1.
      
      It's tempting to add a MINIMUM_CLIENTS define, but... I didn't.
      
      These changes were inspired by antirez#356, but this doesn't
      fix that issue.
      c138631c
    • antirez's avatar
      Sample and cache RSS in serverCron(). · 93253c27
      antirez authored
      Obtaining the RSS (Resident Set Size) info is slow in Linux and OSX.
      This slowed down the generation of the INFO 'memory' section.
      
      Since the RSS does not require to be a real-time measurement, we
      now sample it with server.hz frequency (10 times per second by default)
      and use this value both to show the INFO rss field and to compute the
      fragmentation ratio.
      
      Practically this does not make any difference for memory profiling of
      Redis but speeds up the INFO call significantly.
      93253c27
    • antirez's avatar
      sdscatvprintf(): Try to use a static buffer. · 30639c8c
      antirez authored
      For small content the function now tries to use a static buffer to avoid
      a malloc/free cycle that is too costly when the function is used in the
      context of performance critical code path such as INFO output generation.
      
      This change was verified to have positive effects in the execution speed
      of the INFO command.
      30639c8c
    • antirez's avatar
      Cache uname() output across INFO calls. · d3efe04c
      antirez authored
      Uname was profiled to be a slow syscall. It produces always the same
      output in the context of a single execution of Redis, so calling it at
      every INFO output generation does not make too much sense.
      
      The uname utsname structure was modified as a static variable. At the
      same time a static integer was added to check if we need to call uname
      the first time.
      d3efe04c
    • antirez's avatar
      sdscatvprintf(): guess buflen using format length. · a9caca04
      antirez authored
      sdscatvprintf() uses a loop where it tries to output the formatted
      string in a buffer of the initial length, if there was not enough room,
      a buffer of doubled size is tried and so forth.
      
      The initial guess for the buffer length was very poor, an hardcoded
      "16". This caused the printf to be processed multiple times without a
      good reason. Given that printf functions are already not fast, the
      overhead was significant.
      
      The new heuristic is to use a buffer 4 times the length of the format
      buffer, and 32 as minimal size. This appears to be a good balance for
      typical uses of the function inside the Redis code base.
      
      This change improved INFO command performances 3 times.
      a9caca04
  2. 21 Mar, 2014 3 commits
  3. 20 Mar, 2014 11 commits
    • antirez's avatar
      Use 24 bits for the lru object field and improve resolution. · a9836992
      antirez authored
      There were 2 spare bits inside the Redis object structure that are now
      used in order to enlarge 4x the range of the LRU field.
      
      At the same time the resolution was improved from 10 to 1 second: this
      still provides 194 days before the LRU counter overflows (restarting from
      zero).
      
      This is not a problem since it only causes lack of eviction precision for
      objects not touched for a very long time, and the lack of precision is
      only temporary.
      a9836992
    • antirez's avatar
      Default LRU samples is now 5. · f4da796c
      antirez authored
      f4da796c
    • antirez's avatar
      Use new dictGetRandomKeys() API to get samples for eviction. · c641b670
      antirez authored
      The eviction quality degradates a bit in my tests, but since the API is
      faster, it allows to raise the number of samples, and overall is a win.
      c641b670
    • antirez's avatar
      struct dictEntry -> dictEntry. · 82b53c65
      antirez authored
      82b53c65
    • antirez's avatar
      Added dictGetRandomKeys() to dict.c: mass get random entries. · 5317f5e9
      antirez authored
      This new function is useful to get a number of random entries from an
      hash table when we just need to do some sampling without particularly
      good distribution.
      
      It just jumps at a random place of the hash table and returns the first
      N items encountered by scanning linearly.
      
      The main usefulness of this function is to speedup Redis internal
      sampling of the key space, for example for key eviction or expiry.
      5317f5e9
    • antirez's avatar
      LRU eviction pool implementation. · 22c9cfaf
      antirez authored
      This is an improvement over the previous eviction algorithm where we use
      an eviction pool that is persistent across evictions of keys, and gets
      populated with the best candidates for evictions found so far.
      
      It allows to approximate LRU eviction at a given number of samples
      better than the previous algorithm used.
      22c9cfaf
    • antirez's avatar
      Fix OBJECT IDLETIME return value converting to seconds. · 6d5790d6
      antirez authored
      estimateObjectIdleTime() returns a value in milliseconds now, so we need
      to scale the output of OBJECT IDLETIME to seconds.
      6d5790d6
    • antirez's avatar
      Obtain LRU clock in a resolution dependent way. · ad6b0f70
      antirez authored
      For testing purposes it is handy to have a very high resolution of the
      LRU clock, so that it is possible to experiment with scripts running in
      just a few seconds how the eviction algorithms works.
      
      This commit allows Redis to use the cached LRU clock, or a value
      computed on demand, depending on the resolution. So normally we have the
      good performance of a precomputed value, and a clock that wraps in many
      days using the normal resolution, but if needed, changing a define will
      switch behavior to an high resolution LRU clock.
      ad6b0f70
    • antirez's avatar
      Specify lruclock in redisServer structure via REDIS_LRU_BITS. · 1faf8266
      antirez authored
      The padding field was totally useless: removed.
      1faf8266
    • antirez's avatar
      Specify LRU resolution in milliseconds. · d77e2316
      antirez authored
      d77e2316
    • antirez's avatar
      Set LRU parameters via REDIS_LRU_BITS define. · fe308470
      antirez authored
      fe308470
  4. 19 Mar, 2014 1 commit
  5. 18 Mar, 2014 6 commits
  6. 17 Mar, 2014 1 commit
    • antirez's avatar
      Sentinel failure detection implementation improved. · ae0b7680
      antirez authored
      Failure detection in Sentinel is ping-pong based. It used to work by
      remembering the last time a valid PONG reply was received, and checking
      if the reception time was too old compared to the current current time.
      
      PINGs were sent at a fixed interval of 1 second.
      
      This works in a decent way, but does not scale well when we want to set
      very small values of "down-after-milliseconds" (this is the node
      timeout basically).
      
      This commit reiplements the failure detection making a number of
      changes. Some changes are inspired to Redis Cluster failure detection
      code:
      
      * A new last_ping_time field is added in representation of instances.
        If non zero, we have an active ping that was sent at the specified
        time. When a valid reply to ping is received, the field is zeroed
        again.
      * last_ping_time is not reset when we reconnect the link or send a new
        ping, so from our point of view it represents the time we started
        waiting for the instance to reply to our pings without receiving a
        reply.
      * last_ping_time is now used in order to check if the instance is
        timed out. This means that we can have a node timeout of 100
        milliseconds and yet the system will work well since the new check is
        not bound to the period used to send pings.
      * Pings are now sent every second, or often if the value of
        down-after-milliseconds is less than one second. With a lower limit of
        10 HZ ping frequency.
      * Link reconnection code was improved. This is used in order to try to
        reconnect the link when we are at 50% of the node timeout without a
        valid reply received yet. However the old code triggered unnecessary
        reconnections when the node timeout was very small. Now that should be
        ok.
      
      The new code passes the tests but more testing is needed and more unit
      tests stressing the failure detector, so currently this is merged only
      in the unstable branch.
      ae0b7680
  7. 15 Mar, 2014 1 commit
  8. 14 Mar, 2014 5 commits
    • Salvatore Sanfilippo's avatar
      Merge pull request #1608 from mattsta/fix-sentinel-current-epoch-segfault · c65b75e7
      Salvatore Sanfilippo authored
      Fix segfault from accessing array out of bounds
      c65b75e7
    • Matt Stancliff's avatar
      Fix segfault from accessing array out of bounds · 584052ee
      Matt Stancliff authored
      argc == 2; argv[2] == crash
      584052ee
    • antirez's avatar
      Sentinel: be safe under crash-recovery assumptions. · ed813863
      antirez authored
      Sentinel's main safety argument is that there are no two configurations
      for the same master with the same version (configuration epoch).
      
      For this to be true Sentinels require to be authorized by a majority.
      Additionally Sentinels require to do two important things:
      
      * Never vote again for the same epoch.
      * Never exchange an old vote for a fresh one.
      
      The first prerequisite, in a crash-recovery system model, requires to
      persist the master->leader_epoch on durable storage before to reply to
      messages. This was not the case.
      
      We also make sure to persist the current epoch in order to never reply
      to stale votes requests from other Sentinels, after a recovery.
      
      The configuration is persisted by making use of fsync(), this is
      considered in the context of this code a good enough guarantee that
      after a restart our durable state is restored, however this may not
      always be the case depending on the kind of hardware and operating
      system used.
      ed813863
    • antirez's avatar
      Sentinel: fake PUBLISH command to receive HELLO messages. · 36509402
      antirez authored
      Now the way HELLO messages are received is unified.
      Now it is no longer needed for Sentinels to converge to the higher
      configuration for a master to be able to chat via some Redis instance,
      the are able to directly exchanges configurations.
      
      Note that this commit does not include the (trivial) change needed to
      send HELLO messages to Sentinel instances as well, since for an error I
      committed the change in the previous commit that refactored hello
      messages processing into a separated function.
      36509402
    • antirez's avatar
  9. 13 Mar, 2014 2 commits
  10. 11 Mar, 2014 5 commits