1. 13 Jun, 2018 3 commits
    • antirez's avatar
      Security: more cmsgpack fixes by @soloestoy. · a57595ca
      antirez authored
      @soloestoy sent me this additional fixes, after searching for similar
      problems to the one reported in mp_pack(). I'm committing the changes
      because it was not possible during to make a public PR to protect Redis
      users and give Redis providers some time to patch their systems.
      a57595ca
    • antirez's avatar
      Security: update Lua struct package for security. · 8783fb94
      antirez authored
      During an auditing Apple found that the "struct" Lua package
      we ship with Redis (http://www.inf.puc-rio.br/~roberto/struct/) contains
      a security problem. A bound-checking statement fails because of integer
      overflow. The bug exists since we initially integrated this package with
      Lua, when scripting was introduced, so every version of Redis with
      EVAL/EVALSHA capabilities exposed is affected.
      
      Instead of just fixing the bug, the library was updated to the latest
      version shipped by the author.
      8783fb94
    • antirez's avatar
      Security: fix Lua cmsgpack library stack overflow. · 8cb9344b
      antirez authored
      During an auditing effort, the Apple Vulnerability Research team discovered
      a critical Redis security issue affecting the Lua scripting part of Redis.
      
      -- Description of the problem
      
      Several years ago I merged a pull request including many small changes at
      the Lua MsgPack library (that originally I authored myself). The Pull
      Request entered Redis in commit 90b6337c, in 2014.
      Unfortunately one of the changes included a variadic Lua function that
      lacked the check for the available Lua C stack. As a result, calling the
      "pack" MsgPack library function with a large number of arguments, results
      into pushing into the Lua C stack a number of new values proportional to
      the number of arguments the function was called with. The pushed values,
      moreover, are controlled by untrusted user input.
      
      This in turn causes stack smashing which we believe to be exploitable,
      while not very deterministic, but it is likely that an exploit could be
      created targeting specific versions of Redis executables. However at its
      minimum the issue results in a DoS, crashing the Redis server.
      
      -- Versions affected
      
      Versions greater or equal to Redis 2.8.18 are affected.
      
      -- Reproducing
      
      Reproduce with this (based on the original reproduction script by
      Apple security team):
      
      https://gist.github.com/antirez/82445fcbea6d9b19f97014cc6cc79f8a
      
      -- Verification of the fix
      
      The fix was tested in the following way:
      
      1) I checked that the problem is no longer observable running the trigger.
      2) The Lua code was analyzed to understand the stack semantics, and that
      actually enough stack is allocated in all the cases of mp_pack() calls.
      3) The mp_pack() function was modified in order to show exactly what items
      in the stack were being set, to make sure that there is no silent overflow
      even after the fix.
      
      -- Credits
      
      Thank you to the Apple team and to the other persons that helped me
      checking the patch and coordinating this communication.
      8cb9344b
  2. 29 May, 2018 1 commit
  3. 22 Apr, 2017 1 commit
  4. 30 Jan, 2017 1 commit
    • antirez's avatar
      Jemalloc updated to 4.4.0. · 36c1acc2
      antirez authored
      The original jemalloc source tree was modified to:
      
      1. Remove the configure error that prevents nested builds.
      2. Insert the Redis private Jemalloc API in order to allow the
      Redis fragmentation function to work.
      36c1acc2
  5. 12 Jan, 2017 2 commits
  6. 21 Dec, 2016 1 commit
  7. 29 Jul, 2016 1 commit
  8. 06 Jul, 2016 1 commit
    • antirez's avatar
      geohash.c and geohash_helper.c are part of Redis. · eaa713e9
      antirez authored
      They were under /deps since they originate from a different source tree,
      however at this point they are very modified and we took ownership of
      both the files making changes, fixing bugs, so there is no upgrade path
      from the original code tree.
      
      Given that, better to move the code under /src with proper dependencies
      and with a more simpler editing experience.
      eaa713e9
  9. 30 May, 2016 2 commits
    • antirez's avatar
      Geo: fix typo in geohashEstimateStepsByRadius(). · 5d4b5fbd
      antirez authored
      I'm the author of this line but I can't see a good reason for it to
      don't be a typo, a step of 26 should be valid with 52 bits per
      coordinate, moreover the line was:
      
          if (step > 26) step = 25;
      
      So a step of 26 was actually already used, except when one of 27 was
      computed (which is invalid) only then it was trimmed to 25 instead of
      26.
      
      All tests passing after the change.
      5d4b5fbd
    • antirez's avatar
      Fix GEORADIUS wrong output with radius > Earth radius. · 4eff3dc4
      antirez authored
      Close #3266
      4eff3dc4
  10. 18 May, 2016 1 commit
  11. 13 Apr, 2016 1 commit
  12. 12 Apr, 2016 1 commit
  13. 06 Apr, 2016 1 commit
    • antirez's avatar
      Linenoise updated. · 8a98b8d0
      antirez authored
      As a side effect, cat commands.txt | redis-cli now is able to handle
      lines more than 4096 bytes.
      8a98b8d0
  14. 10 Feb, 2016 1 commit
  15. 07 Feb, 2016 1 commit
  16. 13 Dec, 2015 1 commit
  17. 17 Nov, 2015 1 commit
    • antirez's avatar
      Lua debugger: use sds_malloc() to allocate eval cli array. · e57cccde
      antirez authored
      Redis-cli handles the debugger "eval" command in a special way since
      sdssplitargs() would not be ok: we need to send the Redis debugger the
      whole Lua script without any parsing. However in order to later free the
      argument vector inside redis-cli using just sdsfreesplitres(), we need
      to allocate the array of SDS pointers using the same allocator SDS is
      using, that may differ to what Redis is using.
      
      So now a newer version of SDS exports sds_malloc() and other allocator
      functions to give access, to the program it is linked to, the allocator
      used internally by SDS.
      e57cccde
  18. 07 Oct, 2015 1 commit
  19. 06 Oct, 2015 2 commits
  20. 25 Jul, 2015 1 commit
  21. 24 Jul, 2015 1 commit
    • antirez's avatar
      Jemalloc: use LG_QUANTUM of 3 for AMD64 and I386. · 6b836b6b
      antirez authored
      This gives us a 24 bytes size class which is dict.c dictEntry size, thus
      improving the memory efficiency of Redis significantly.
      Moreover other non 16 bytes aligned tiny classes are added that further
      reduce the fragmentation of the allocator.
      
      Technically speaking LG_QUANTUM should be 4 on i386 / AMD64 because of
      SSE types and other 16 bytes types, however we don't use those, and our
      jemalloc only targets Redis.
      
      New versions of Jemalloc will have an explicit configure switch in order
      to specify the quantum value for a platform without requiring any change
      to the Jemalloc source code: we'll switch to this system when available.
      
      This change was originally proposed by Oran Agra (@oranagra) as a change
      to the Jemalloc script to generate the size classes define. We ended
      doing it differently by changing LG_QUANTUM since it is apparently the
      supported Jemalloc method to obtain a 24 bytes size class, moreover it
      also provides us other potentially useful size classes.
      
      Related to issue #2510.
      6b836b6b
  22. 09 Jul, 2015 2 commits
  23. 06 Jul, 2015 1 commit
  24. 01 Jul, 2015 1 commit
  25. 29 Jun, 2015 2 commits
  26. 26 Jun, 2015 1 commit
    • antirez's avatar
      Geo: from lat,lon API to lon,lat API according to GIS standard · fa9d62d3
      antirez authored
      The GIS standard and all the major DBs implementing GIS related
      functions take coordinates as x,y that is longitude,latitude.
      It was a bad start for Redis to do things differently, so even if this
      means that existing users of the Geo module will be required to change
      their code, Redis now conforms to the standard.
      
      Usually Redis is very backward compatible, but this is not an exception
      to this rule, since this is the first Geo implementation entering the
      official Redis source code. It is not wise to try to be backward
      compatible with code forks... :-)
      
      Close #2637.
      fa9d62d3
  27. 24 Jun, 2015 1 commit
    • antirez's avatar
      Geo: Fix geohashEstimateStepsByRadius() step underestimation. · 55c4a365
      antirez authored
      The returned step was in some case not enough towards normal
      coordinates (for example when our search position was was already near the
      margin of the central area, and we had to match, using the east or west
      neighbor, a very far point). Example:
      
          geoadd points 67.575457940146066 -62.001317572780565 far
          geoadd points 66.685439060295664 -58.925040587282297 center
          georadius points 66.685439060295664 -58.925040587282297 200 km
      
      In the above case the code failed to find a match (happens at smaller
      latitudes too) even if far and center are at less than 200km.
      
      Another fix introduced by this commit is a progressively larger area
      towards the poles, since meridians are a lot less far away, so we need
      to compensate for this.
      
      The current implementation works comparably to the Tcl brute-force
      stress tester implemented in the fuzzy test in the geo.tcl unit for
      latitudes between -70 and 70, and is pretty accurate over +/-80 too,
      with sporadic false negatives.
      
      A more mathematically clean implementation is possible by computing the
      meridian distance at the specified latitude and computing the step
      according to it.
      55c4a365
  28. 23 Jun, 2015 1 commit
    • antirez's avatar
      Geo: big refactoring of geo.c, zset.[ch] removed. · 0b931390
      antirez authored
      This commit simplifies the implementation in a few ways:
      
      1. zsetScore implementation improved a bit and moved into t_zset.c where
         is now also used to implement the ZSCORE command.
      
      2. Range extraction from the sorted set remains a separated
         implementation from the one in t_zset.c, but was hyper-specialized in
         order to avoid accumulating results into a list and remove the ones
         outside the radius.
      
      3. A new type is introduced: geoArray, which can accumulate geoPoint
         structures in a vector with power of two expansion policy. This is
         useful since we have to call qsort() against it before returning the
         result to the user.
      
      4. As a result of 1, 2, 3, the two files zset.c and zset.h are now
         removed, including the function to merge two lists (now handled with
         functions that can add elements to existing geoArray arrays) and
         the machinery used in order to pass zset results.
      
      5. geoPoint structure simplified because of the general code structure
         simplification, so we no longer need to take references to objects.
      
      6. Not counting the JSON removal the refactoring removes 200 lines of
         code for the same functionalities, with a simpler to read
         implementation.
      
      7. GEORADIUS is now 2.5 times faster testing with 10k elements and a
         radius resulting in 124 elements returned. However this is mostly a
         side effect of the refactoring and simplification. More speed gains
         can be achieved by trying to optimize the code.
      0b931390
  29. 22 Jun, 2015 2 commits
    • antirez's avatar
    • Matt Stancliff's avatar
      [In-Progress] Add Geo Commands · 7f4ac3d1
      Matt Stancliff authored
      Current todo:
        - replace functions in zset.{c,h} with a new unified Redis
          zset access API.
      
      Once we get the zset interface fixed, we can squash
      relevant commits in this branch and have one nice commit
      to merge into unstable.
      
      This commit adds:
        - Geo commands
        - Tests; runnable with: ./runtest --single unit/geo
        - Geo helpers in deps/geohash-int/
        - src/geo.{c,h} and src/geojson.{c,h} implementing geo commands
        - Updated build configurations to get everything working
        - TEMPORARY: src/zset.{c,h} implementing zset score and zset
          range reading without writing to client output buffers.
        - Modified linkage of one t_zset.c function for use in zset.c
      
      Conflicts:
      	src/Makefile
      	src/redis.c
      7f4ac3d1
  30. 03 Jun, 2015 2 commits
  31. 28 Apr, 2015 1 commit
    • antirez's avatar
      Fix Sentinel memory leak (hiredis bug) · c806dd79
      antirez authored
      This fixes issue #2535, that was actually an hiredis library bug (I
      submitted an issue and fix to the redis/hiredis repo as well).
      
      When an asynchronous hiredis connection subscribes to a Pub/Sub channel
      and gets an error, and in other related conditions, the function
      redisProcessCallbacks() enters a code path where the link is
      disconnected, however the function returns before freeing the allocated
      reply object. This causes a memory leak. The memory leak was trivial to
      trigger in Redis Sentinel, which uses hiredis, every time we tried to
      subscribe to an instance that required a password, in case the Sentinel
      was configured either with the wrong password or without password at
      all. In this case, the -AUTH error caused the leaking code path to be
      executed.
      
      It was verified with Valgrind that after this change the leak no longer
      happens in Sentinel with a misconfigured authentication password.
      c806dd79