1. 20 Jun, 2018 1 commit
    • Guy Benoish's avatar
      Enhance RESTORE with RDBv9 new features · b5197f1f
      Guy Benoish authored
      RESTORE now supports:
      1. Setting LRU/LFU
      2. Absolute-time TTL
      
      Other related changes:
      1. RDB loading will not override LRU bits when RDB file
         does not contain the LRU opcode.
      2. RDB loading will not set LRU/LFU bits if the server's
         maxmemory-policy does not match.
      b5197f1f
  2. 19 Jun, 2018 10 commits
    • Salvatore Sanfilippo's avatar
      Merge pull request #5042 from oranagra/malloc_usable_size_libc · c6fdebf5
      Salvatore Sanfilippo authored
      add malloc_usable_size for libc malloc
      c6fdebf5
    • Oran Agra's avatar
      add malloc_usable_size for libc malloc · 482785ac
      Oran Agra authored
      this reduces the extra 8 bytes we save before each pointer.
      but more importantly maybe, it makes the valgrind runs to be more similiar
      to our normal runs.
      
      note: the change in malloc_stats struct in server.h is to eliminate an name conflict.
      structs that are not typedefed are resolved from a separate name space.
      482785ac
    • Salvatore Sanfilippo's avatar
      Merge pull request #5023 from FX-HAO/unstable · 4da29630
      Salvatore Sanfilippo authored
      Fix update_zmalloc_stat_alloc in zrealloc
      4da29630
    • antirez's avatar
      Test RDB stream encoding saving/loading. · e7219025
      antirez authored
      e7219025
    • Salvatore Sanfilippo's avatar
      Merge pull request #5041 from oranagra/redis-rdb-check_rdbLoadMillisecondTime · 5f5e1199
      Salvatore Sanfilippo authored
      fix redis-rdb-check to provide proper arguments to rdbLoadMillisecondTime
      5f5e1199
    • antirez's avatar
      4848fbec
    • Oran Agra's avatar
      fix redis-rdb-check to provide proper arguments to rdbLoadMillisecondTime · f31b0405
      Oran Agra authored
      due to incorrect forward declaration, it didn't provide all arguments.
      this lead to random value being read from the stack and return of incorrect time,
      which in this case doesn't matter since no one uses it.
      f31b0405
    • antirez's avatar
      AOF: remove no longer used variable "now". · 333c98c4
      antirez authored
      333c98c4
    • antirez's avatar
      e94b2053
    • antirez's avatar
      Remove AOF optimization to skip expired keys. · ba92b517
      antirez authored
      Basically we cannot be sure that if the key is expired while writing the
      AOF, the main thread will surely find the key expired. There are
      possible race conditions like the moment at which the "now" is sampled,
      and the fact that time may jump backward.
      
      Think about the following:
      
      SET a 5
      EXPIRE a 1
      
      AOF rewrite starts after about 1 second. The child process finds the key
      expired, while in the main thread instead an INCR command is called
      against the key "a" immediately after a fork, and the scheduler was
      faster to give execution time to the main thread, so "a" is yet not
      expired.
      
      The main thread will generate an INCR a command to the AOF log that will
      be appended to the rewritten AOF file, but that INCR command will target
      a non existin "a" key, so a new non volatile key "a" will be created.
      
      Two observations:
      
      A) In theory by computing "now" before the fork, we should be sure that
      if a key is expired at that time, it will be expired later when the
      main thread will try to access to such key. However this does not take
      into account the fact that the computer time may jump backward.
      
      B) Technically we may still make the process safe by using a monotonic
      time source.
      
      However there were other similar related bugs, and in general the new
      "vision" is that Redis persistence files should represent the memory
      state without trying to be too smart: this makes the design more
      consistent, bugs less likely to arise from complex interactions, and in
      the end what is to fix is the Redis expire process to have less expired
      keys in RAM.
      
      Thanks to Oran Agra and Guy Benoish for writing me an email outlining
      this problem, after they conducted a Redis 5 code review.
      ba92b517
  3. 18 Jun, 2018 9 commits
  4. 17 Jun, 2018 1 commit
  5. 16 Jun, 2018 2 commits
  6. 14 Jun, 2018 8 commits
  7. 13 Jun, 2018 8 commits
    • zhaozhao.zz's avatar
      fix exists command on slave · 2ffa533f
      zhaozhao.zz authored
      2ffa533f
    • shenlongxing's avatar
      e4e5a670
    • zhaozhao.zz's avatar
      optimize reply list memory usage · 963002d7
      zhaozhao.zz authored
      963002d7
    • antirez's avatar
      Security: fix redis-cli buffer overflow. · ce17f76b
      antirez authored
      Thanks to Fakhri Zulkifli for reporting it.
      
      The fix switched to dynamic allocation, copying the final prompt in the
      static buffer only at the end.
      ce17f76b
    • antirez's avatar
      Security: fix Lua struct package offset handling. · e89086e0
      antirez authored
      After the first fix to the struct package I found another similar
      problem, which is fixed by this patch. It could be reproduced easily by
      running the following script:
      
          return struct.unpack('f', "xxxxxxxxxxxxx",-3)
      
      The above will access bytes before the 'data' pointer.
      e89086e0
    • antirez's avatar
      Security: more cmsgpack fixes by @soloestoy. · 5ccb6f7a
      antirez authored
      @soloestoy sent me this additional fixes, after searching for similar
      problems to the one reported in mp_pack(). I'm committing the changes
      because it was not possible during to make a public PR to protect Redis
      users and give Redis providers some time to patch their systems.
      5ccb6f7a
    • antirez's avatar
      Security: update Lua struct package for security. · 1eb08bcd
      antirez authored
      During an auditing Apple found that the "struct" Lua package
      we ship with Redis (http://www.inf.puc-rio.br/~roberto/struct/) contains
      a security problem. A bound-checking statement fails because of integer
      overflow. The bug exists since we initially integrated this package with
      Lua, when scripting was introduced, so every version of Redis with
      EVAL/EVALSHA capabilities exposed is affected.
      
      Instead of just fixing the bug, the library was updated to the latest
      version shipped by the author.
      1eb08bcd
    • antirez's avatar
      Security: fix Lua cmsgpack library stack overflow. · 52a00201
      antirez authored
      During an auditing effort, the Apple Vulnerability Research team discovered
      a critical Redis security issue affecting the Lua scripting part of Redis.
      
      -- Description of the problem
      
      Several years ago I merged a pull request including many small changes at
      the Lua MsgPack library (that originally I authored myself). The Pull
      Request entered Redis in commit 90b6337c, in 2014.
      Unfortunately one of the changes included a variadic Lua function that
      lacked the check for the available Lua C stack. As a result, calling the
      "pack" MsgPack library function with a large number of arguments, results
      into pushing into the Lua C stack a number of new values proportional to
      the number of arguments the function was called with. The pushed values,
      moreover, are controlled by untrusted user input.
      
      This in turn causes stack smashing which we believe to be exploitable,
      while not very deterministic, but it is likely that an exploit could be
      created targeting specific versions of Redis executables. However at its
      minimum the issue results in a DoS, crashing the Redis server.
      
      -- Versions affected
      
      Versions greater or equal to Redis 2.8.18 are affected.
      
      -- Reproducing
      
      Reproduce with this (based on the original reproduction script by
      Apple security team):
      
      https://gist.github.com/antirez/82445fcbea6d9b19f97014cc6cc79f8a
      
      -- Verification of the fix
      
      The fix was tested in the following way:
      
      1) I checked that the problem is no longer observable running the trigger.
      2) The Lua code was analyzed to understand the stack semantics, and that
      actually enough stack is allocated in all the cases of mp_pack() calls.
      3) The mp_pack() function was modified in order to show exactly what items
      in the stack were being set, to make sure that there is no silent overflow
      even after the fix.
      
      -- Credits
      
      Thank you to the Apple team and to the other persons that helped me
      checking the patch and coordinating this communication.
      52a00201
  8. 12 Jun, 2018 1 commit
    • antirez's avatar
      RDB: Apply fix to rdbLoadMillisecondTime() only for new RDB versions. · 032ea657
      antirez authored
      This way we let big endian systems to still load old RDB versions.
      However newver versions will be saved and loaded in a way that make RDB
      expires cross-endian again. Thanks to @oranagra for the reporting and
      the discussion about this problem, leading to this fix.
      032ea657