1. 10 Jun, 2014 1 commit
    • Matt Stancliff's avatar
      Scripting: Fix regression from #1118 · 76efe122
      Matt Stancliff authored
      The new check-for-number behavior of Lua arguments broke
      users who use large strings of just integers.
      
      The Lua number check would convert the string to a number, but
      that breaks user data because
      Lua numbers have limited precision compared to an arbitrarily
      precise number wrapped in a string.
      
      Regression fixed and new test added.
      
      Fixes #1118 again.
      76efe122
  2. 04 Jun, 2014 2 commits
    • antirez's avatar
      Fixed dbuf variable scope in luaRedisGenericCommand(). · 3758f27b
      antirez authored
      I'm not sure if while the visibility is the inner block, the fact we
      point to 'dbuf' is a problem or not, probably the stack var isx
      guaranteed to live until the function returns. However obvious code is
      better anyway.
      3758f27b
    • antirez's avatar
      Scripting: better Lua number -> string conversion in luaRedisGenericCommand(). · 072982d8
      antirez authored
      The lua_to*string() family of functions use a non optimal format
      specifier when converting integers to strings. This has both the problem
      of the number being converted in exponential notation, which we don't
      use as a Redis return value when floating point numbers are involed,
      and, moreover, there is a loss of precision since the default format
      specifier is not able to represent numbers that must be represented
      exactly in the IEEE 754 number mantissa.
      
      The new code handles it as a special case using a saner conversion.
      
      This fixes issue #1118.
      072982d8
  3. 20 May, 2014 1 commit
  4. 19 May, 2014 1 commit
    • michael-grunder's avatar
      Fix LUA_OBJCACHE segfault. · ea0e2524
      michael-grunder authored
      When scanning the argument list inside of a redis.call() invocation
      for pre-cached values, there was no check being done that the
      argument we were on was in fact within the bounds of the cache size.
      
      So if a redis.call() command was ever executed with more than 32
      arguments (current cache size #define setting) redis-server could
      segfault.
      ea0e2524
  5. 07 May, 2014 8 commits
  6. 24 Apr, 2014 1 commit
    • antirez's avatar
      Process events with processEventsWhileBlocked() when blocked. · e29d3307
      antirez authored
      When we are blocked and a few events a processed from time to time, it
      is smarter to call the event handler a few times in order to handle the
      accept, read, write, close cycle of a client in a single pass, otherwise
      there is too much latency added for clients to receive a reply while the
      server is busy in some way (for example during the DB loading).
      e29d3307
  7. 13 Feb, 2014 1 commit
    • antirez's avatar
      Fix script cache bug in the scripting engine. · 21e6b0fb
      antirez authored
      This commit fixes a serious Lua scripting replication issue, described
      by Github issue #1549. The root cause of the problem is that scripts
      were put inside the script cache, assuming that slaves and AOF already
      contained it, even if the scripts sometimes produced no changes in the
      data set, and were not actaully propagated to AOF/slaves.
      
      Example:
      
          eval "if tonumber(KEYS[1]) > 0 then redis.call('incr', 'x') end" 1 0
      
      Then:
      
          evalsha <sha1 step 1 script> 1 0
      
      At this step sha1 of the script is added to the replication script cache
      (the script is marked as known to the slaves) and EVALSHA command is
      transformed to EVAL. However it is not dirty (there is no changes to db),
      so it is not propagated to the slaves. Then the script is called again:
      
          evalsha <sha1 step 1 script> 1 1
      
      At this step master checks that the script already exists in the
      replication script cache and doesn't transform it to EVAL command. It is
      dirty and propagated to the slaves, but they fail to evaluate the script
      as they don't have it in the script cache.
      
      The fix is trivial and just uses the new API to force the propagation of
      the executed command regardless of the dirty state of the data set.
      
      Thank you to @minus-infinity on Github for finding the issue,
      understanding the root cause, and fixing it.
      21e6b0fb
  8. 03 Feb, 2014 1 commit
  9. 05 Dec, 2013 1 commit
  10. 29 Aug, 2013 1 commit
    • antirez's avatar
      Fixed critical memory leak from EVAL. · 41d31473
      antirez authored
      Multiple missing calls to lua_pop prevented the error handler function
      pushed on the stack for lua_pcall() to be popped before returning,
      causing a memory leak in almost all the code paths of EVAL (both
      successful calls and calls returning errors).
      
      This caused two issues: Lua leaking memory (and this was very visible
      from INFO memory output, as the 'used_memory_lua' field reported an
      always increasing amount of memory used), and as a result slower and
      slower GC cycles resulting in all the CPU being used.
      
      Thanks to Tanguy Le Barzic for noticing something was wrong with his 2.8
      slave, and for creating a testing EC2 environment where I was able to
      investigate the issue.
      41d31473
  11. 25 Jun, 2013 3 commits
  12. 24 Jun, 2013 2 commits
  13. 21 Jun, 2013 1 commit
    • antirez's avatar
      New API to force propagation. · 515a26bb
      antirez authored
      The old REDIS_CMD_FORCE_REPLICATION flag was removed from the
      implementation of Redis, now there is a new API to force specific
      executions of a command to be propagated to AOF / Replication link:
      
          void forceCommandPropagation(int flags);
      
      The new API is also compatible with Lua scripting, so a script that will
      execute commands that are forced to be propagated, will also be
      propagated itself accordingly even if no change to data is operated.
      
      As a side effect, this new design fixes the issue with scripts not able
      to propagate PUBLISH to slaves (issue #873).
      515a26bb
  14. 19 Jun, 2013 1 commit
  15. 18 Jun, 2013 2 commits
  16. 22 May, 2013 1 commit
  17. 19 Jan, 2013 1 commit
  18. 10 Jan, 2013 1 commit
    • antirez's avatar
      Multiple fixes for EVAL (issue #872). · baee5650
      antirez authored
      1) The event handler was no restored after a timeout condition if the
         command was eventually executed with success.
      2) The command was not converted to EVAL in case of errors in the middle
         of the execution.
      3) Terrible duplication of code without any apparent reason.
      baee5650
  19. 22 Nov, 2012 1 commit
    • antirez's avatar
      EVALSHA is now case insensitive. · 95f68f7b
      antirez authored
      EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
      Fixed using a case insensitive dictionary type for the sha -> script
      map used for replication of scripts.
      95f68f7b
  20. 08 Nov, 2012 1 commit
  21. 22 Oct, 2012 1 commit
    • antirez's avatar
      Differentiate SCRIPT KILL error replies. · acfe3675
      antirez authored
      When calling SCRIPT KILL currently you can get two errors:
      
      * No script in timeout (busy) state.
      * The script already performed a write.
      
      It is useful to be able to distinguish the two errors, but right now both
      start with "ERR" prefix, so string matching (that is fragile) must be used.
      
      This commit introduces two different prefixes.
      
      -NOTBUSY and -UNKILLABLE respectively to reply with an error when no
      script is busy at the moment, and when the script already executed a
      write operation and can not be killed.
      acfe3675
  22. 01 Oct, 2012 1 commit
  23. 28 Sep, 2012 2 commits
    • antirez's avatar
      Scripting: add helper functions redis.error_reply() and redis.status_reply(). · 9c21b72b
      antirez authored
      A previous commit introduced Redis.NIL. This commit adds similar helper
      functions to return tables with a single field set to the specified
      string so that instead of using 'return {err="My Error"}' it is possible
      to use a more idiomatic form:
      
          return redis.error_reply("My Error")
          return redis.status_reply("OK")
      9c21b72b
    • antirez's avatar
      Scripting: redis.NIL to return nil bulk replies. · 6dd1693c
      antirez authored
      Lua arrays can't contain nil elements (see
      http://www.lua.org/pil/19.1.html for more information), so Lua scripts
      were not able to return a multi-bulk reply containing nil bulk
      elements inside.
      
      This commit introduces a special conversion: a table with just
      a "nilbulk" field set to a boolean value is converted by Redis as a nil
      bulk reply, but at the same time for Lua this type is not a "nil" so can
      be used inside Lua arrays.
      
      This type is also assigned to redis.NIL, so the following two forms
      are equivalent and will be able to return a nil bulk reply as second
      element of a three elements array:
      
          EVAL "return {1,redis.NIL,3}" 0
          EVAL "return {1,{nilbulk=true},3}" 0
      
      The result in redis-cli will be:
      
          1) (integer) 1
          2) (nil)
          3) (integer) 3
      6dd1693c
  24. 04 Sep, 2012 1 commit
    • antirez's avatar
      Scripting: Force SORT BY constant determinism inside SORT itself. · 36741b2c
      antirez authored
      SORT is able to return (faster than when ordering) unordered output if
      the "BY" clause is used with a constant value. However we try to play
      well with scripting requirements of determinism providing always sorted
      outputs when SORT (and other similar commands) are called by Lua
      scripts.
      
      However we used the general mechanism in place in scripting in order to
      reorder SORT output, that is, if the command has the "S" flag set, the
      Lua scripting engine will take an additional step when converting a
      multi bulk reply to Lua value, calling a Lua sorting function.
      
      This is suboptimal as we can do it faster inside SORT itself.
      This is also broken as issue #545 shows us: basically when SORT is used
      with a constant BY, and additionally also GET is used, the Lua scripting
      engine was trying to order the output as a flat array, while it was
      actually a list of key-value pairs.
      
      What we do know is to recognized if the caller of SORT is the Lua client
      (since we can check this using the REDIS_LUA_CLIENT flag). If so, and if
      a "don't sort" condition is triggered by the BY option with a constant
      string, we force the lexicographical sorting.
      
      This commit fixes this bug and improves the performance, and at the same
      time simplifies the implementation. This does not mean I'm smart today,
      it means I was stupid when I committed the original implementation ;)
      36741b2c
  25. 31 Aug, 2012 2 commits
    • antirez's avatar
      Scripting: Reset Lua fake client reply_bytes after command execution. · e323635c
      antirez authored
      Lua scripting uses a fake client in order to run commands in the context
      of a client, accumulate the reply, and convert it into a Lua object
      to return to the caller. This client is reused again and again, and is
      referenced by the server.lua_client globally accessible pointer.
      
      However after every call to redis.call() or redis.pcall(), that is
      handled by the luaRedisGenericCommand() function, the reply_bytes field
      of the client was not set back to zero. This filed is used to estimate
      the amount of memory currently used in the reply. Because of the lack of
      reset, script after script executed, this value used to get bigger and
      bigger, and in the end on 32 bit systems it triggered the following
      assert:
      
          redisAssert(c->reply_bytes < ULONG_MAX-(1024*64));
      
      On 64 bit systems this does not happen because it takes too much time to
      reach values near to 2^64 for users to see the practical effect of the
      bug.
      
      Now in the cleanup stage of luaRedisGenericCommand() we reset the
      reply_bytes counter to zero, avoiding the issue. It is not practical to
      add a test for this bug, but the fix was manually tested using a
      debugger.
      
      This commit fixes issue #656.
      e323635c
    • antirez's avatar
      Scripting: require at least one argument for redis.call(). · 46c31a15
      antirez authored
      Redis used to crash with a call like the following:
      
          EVAL "redis.call()" 0
      
      Now the explicit check for at least one argument prevents the problem.
      
      This commit fixes issue #655.
      46c31a15
  26. 27 Apr, 2012 1 commit
    • antirez's avatar
      Set LUA_MASKCOUNT hook more selectively. Fixes issue #480. · 0ad10db2
      antirez authored
      An user reported a crash with Redis scripting (see issue #480 on
      github), inspection of the kindly provided strack trace showed that
      server.lua_caller was probably set to NULL. The stack trace also slowed
      that the call to the hook was originating from a point where we just
      used to set/get a few global variables in the Lua state.
      
      What was happening is that we did not set the timeout hook selectively
      only when the user script was called. Now we set it more selectively,
      specifically only in the context of the lua_pcall() call, and make sure
      to remove the hook when the call returns. Otherwise the hook can get
      called in random contexts every time we do something with the Lua
      state.
      0ad10db2