1. 22 Mar, 2017 4 commits
  2. 21 Feb, 2017 9 commits
    • antirez's avatar
      SipHash 2-4 -> SipHash 1-2. · 5a413303
      antirez authored
      For performance reasons we use a reduced rounds variant of
      SipHash. This should still provide enough protection and the
      effects in the hash table distribution are non existing.
      If some real world attack on SipHash 1-2 will be found we can
      trivially switch to something more secure. Anyway it is a
      big step forward from Murmurhash, for which it is trivial to
      generate *seed independent* colliding keys... The speed
      penatly introduced by SipHash 2-4, around 4%, was a too big
      price to pay compared to the effectiveness of the HashDoS
      attack against SipHash 1-2, and considering so far in the
      Redis history, no such an incident ever happened even while
      using trivially to collide hash functions.
      5a413303
    • antirez's avatar
      freeMemoryIfNeeded(): improve code and lazyfree handling. · a8cbc3ec
      antirez authored
      1. Refactor memory overhead computation into a function.
      2. Every 10 keys evicted, check if memory usage already reached
         the target value directly, since we otherwise don't count all
         the memory reclaimed by the background thread right now.
      a8cbc3ec
    • antirez's avatar
      857e6d56
    • antirez's avatar
      SipHash x86 optimizations. · 34387cea
      antirez authored
      34387cea
    • antirez's avatar
      Use SipHash hash function to mitigate HashDos attempts. · ba647598
      antirez authored
      This change attempts to switch to an hash function which mitigates
      the effects of the HashDoS attack (denial of service attack trying
      to force data structures to worst case behavior) while at the same time
      providing Redis with an hash function that does not expect the input
      data to be word aligned, a condition no longer true now that sds.c
      strings have a varialbe length header.
      
      Note that it is possible sometimes that even using an hash function
      for which collisions cannot be generated without knowing the seed,
      special implementation details or the exposure of the seed in an
      indirect way (for example the ability to add elements to a Set and
      check the return in which Redis returns them with SMEMBERS) may
      make the attacker's life simpler in the process of trying to guess
      the correct seed, however the next step would be to switch to a
      log(N) data structure when too many items in a single bucket are
      detected: this seems like an overkill in the case of Redis.
      
      SPEED REGRESION TESTS:
      
      In order to verify that switching from MurmurHash to SipHash had
      no impact on speed, a set of benchmarks involving fast insertion
      of 5 million of keys were performed.
      
      The result shows Redis with SipHash in high pipelining conditions
      to be about 4% slower compared to using the previous hash function.
      However this could partially be related to the fact that the current
      implementation does not attempt to hash whole words at a time but
      reads single bytes, in order to have an output which is endian-netural
      and at the same time working on systems where unaligned memory accesses
      are a problem.
      
      Further X86 specific optimizations should be tested, the function
      may easily get at the same level of MurMurHash2 if a few optimizations
      are performed.
      ba647598
    • Salvatore Sanfilippo's avatar
      ARM: Avoid fast path for BITOP. · 2ee19d98
      Salvatore Sanfilippo authored
      GCC will produce certain unaligned multi load-store instructions
      that will be trapped by the Linux kernel since ARM v6 cannot
      handle them with unaligned addresses. Better to use the slower
      but safer implementation instead of generating the exception which
      should be anyway very slow.
      2ee19d98
    • Salvatore Sanfilippo's avatar
      ARM: Use libc malloc by default. · eb62cfea
      Salvatore Sanfilippo authored
      I'm not sure how much test Jemalloc gets on ARM, moreover
      compiling Redis with Jemalloc support in not very powerful
      devices, like most ARMs people will build Redis on, is extremely
      slow. It is possible to enable Jemalloc build anyway if needed
      by using "make MALLOC=jemalloc".
      eb62cfea
    • Salvatore Sanfilippo's avatar
      ARM: Avoid memcpy() in MurmurHash64A() if we are using 64 bit ARM. · 620e48b1
      Salvatore Sanfilippo authored
      However note that in architectures supporting 64 bit unaligned
      accesses memcpy(...,...,8) is likely translated to a simple
      word memory movement anyway.
      620e48b1
    • Salvatore Sanfilippo's avatar
  3. 20 Feb, 2017 2 commits
  4. 09 Feb, 2017 1 commit
    • antirez's avatar
      Fix MIGRATE closing of cached socket on error. · 8d55aeb5
      antirez authored
      After investigating issue #3796, it was discovered that MIGRATE
      could call migrateCloseSocket() after the original MIGRATE c->argv
      was already rewritten as a DEL operation. As a result the host/port
      passed to migrateCloseSocket() could be anything, often a NULL pointer
      that gets deferenced crashing the server.
      
      Now the socket is closed at an earlier time when there is a socket
      error in a later stage where no retry will be performed, before we
      rewrite the argument vector. Moreover a check was added so that later,
      in the socket_err label, there is no further attempt at closing the
      socket if the argument was rewritten.
      
      This fix should resolve the bug reported in #3796.
      8d55aeb5
  5. 01 Feb, 2017 3 commits
    • antirez's avatar
      Fix ziplist fix... · 7c22d768
      antirez authored
      7c22d768
    • antirez's avatar
      Ziplist: insertion bug under particular conditions fixed. · 8327b813
      antirez authored
      Ziplists had a bug that was discovered while investigating a different
      issue, resulting in a corrupted ziplist representation, and a likely
      segmentation foult and/or data corruption of the last element of the
      ziplist, once the ziplist is accessed again.
      
      The bug happens when a specific set of insertions / deletions is
      performed so that an entry is encoded to have a "prevlen" field (the
      length of the previous entry) of 5 bytes but with a count that could be
      encoded in a "prevlen" field of a since byte. This could happen when the
      "cascading update" process called by ziplistInsert()/ziplistDelete() in
      certain contitious forces the prevlen to be bigger than necessary in
      order to avoid too much data moving around.
      
      Once such an entry is generated, inserting a very small entry
      immediately before it will result in a resizing of the ziplist for a
      count smaller than the current ziplist length (which is a violation,
      inserting code expects the ziplist to get bigger actually). So an FF
      byte is inserted in a misplaced position. Moreover a realloc() is
      performed with a count smaller than the ziplist current length so the
      final bytes could be trashed as well.
      
      SECURITY IMPLICATIONS:
      
      Currently it looks like an attacker can only crash a Redis server by
      providing specifically choosen commands. However a FF byte is written
      and there are other memory operations that depend on a wrong count, so
      even if it is not immediately apparent how to mount an attack in order
      to execute code remotely, it is not impossible at all that this could be
      done. Attacks always get better... and we did not spent enough time in
      order to think how to exploit this issue, but security researchers
      or malicious attackers could.
      8327b813
    • antirez's avatar
      ziplist: better comments, some refactoring. · 1688ccff
      antirez authored
      1688ccff
  6. 30 Jan, 2017 5 commits
  7. 27 Jan, 2017 2 commits
    • antirez's avatar
      Add panic() into redisassert.h. · 7c2153da
      antirez authored
      This header file is for libs, like ziplist.c, that we want to leave
      almost separted from the core. The panic() calls will be easy to delete
      in order to use such files outside, but the debugging info we gain are
      very valuable compared to simple assertions where it is not possible to
      print debugging info.
      7c2153da
    • antirez's avatar
      serverPanic(): allow printf() alike formatting. · dc83ddf0
      antirez authored
      This is of great interest because allows us to print debugging
      informations that could be of useful when debugging, like in the
      following example:
      
          serverPanic("Unexpected encoding for object %d, %d",
              obj->type, obj->encoding);
      dc83ddf0
  8. 13 Jan, 2017 3 commits
  9. 12 Jan, 2017 11 commits