1. 06 Dec, 2020 3 commits
    • Oran Agra's avatar
      Sanitize dump payload: validate no duplicate records in hash/zset/intset · 3716950c
      Oran Agra authored
      If RESTORE passes successfully with full sanitization, we can't affort
      to crash later on assertion due to duplicate records in a hash when
      converting it form ziplist to dict.
      This means that when doing full sanitization, we must make sure there
      are no duplicate records in any of the collections.
      3716950c
    • Oran Agra's avatar
      Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed · c31055db
      Oran Agra authored
      The test creates keys with various encodings, DUMP them, corrupt the payload
      and RESTORES it.
      It utilizes the recently added use-exit-on-panic config to distinguish between
       asserts and segfaults.
      If the restore succeeds, it runs random commands on the key to attempt to
      trigger a crash.
      
      It runs in two modes, one with deep sanitation enabled and one without.
      In the first one we don't expect any assertions or segfaults, in the second one
      we expect assertions, but no segfaults.
      We also check for leaks and invalid reads using valgrind, and if we find them
      we print the commands that lead to that issue.
      
      Changes in the code (other than the test):
      - Replace a few NPD (null pointer deference) flows and division by zero with an
        assertion, so that it doesn't fail the test. (since we set the server to use
        `exit` rather than `abort` on assertion).
      - Fix quite a lot of flows in rdb.c that could have lead to memory leaks in
        RESTORE command (since it now responds with an error rather than panic)
      - Add a DEBUG flag for SET-SKIP-CHECKSUM-VALIDATION so that the test don't need
        to bother with faking a valid checksum
      - Remove a pile of code in serverLogObjectDebugInfo which is actually unsafe to
        run in the crash report (see comments in the code)
      - fix a missing boundary check in lzf_decompress
      
      test suite infra improvements:
      - be able to run valgrind checks before the process terminates
      - rotate log files when restarting servers
      c31055db
    • Oran Agra's avatar
      Sanitize dump payload: ziplist, listpack, zipmap, intset, stream · ca1c1825
      Oran Agra authored
      When loading an encoded payload we will at least do a shallow validation to
      check that the size that's encoded in the payload matches the size of the
      allocation.
      This let's us later use this encoded size to make sure the various offsets
      inside encoded payload don't reach outside the allocation, if they do, we'll
      assert/panic, but at least we won't segfault or smear memory.
      
      We can also do 'deep' validation which runs on all the records of the encoded
      payload and validates that they don't contain invalid offsets. This lets us
      detect corruptions early and reject a RESTORE command rather than accepting
      it and asserting (crashing) later when accessing that payload via some command.
      
      configuration:
      - adding ACL flag skip-sanitize-payload
      - adding config sanitize-dump-payload [yes/no/clients]
      
      For now, we don't have a good way to ensure MIGRATE in cluster resharding isn't
      being slowed down by these sanitation, so i'm setting the default value to `no`,
      but later on it should be set to `clients` by default.
      
      changes:
      - changing rdbReportError not to `exit` in RESTORE command
      - adding a new stat to be able to later check if cluster MIGRATE isn't being
        slowed down by sanitation.
      ca1c1825
  2. 16 Oct, 2018 1 commit
  3. 20 Jun, 2016 1 commit
  4. 18 May, 2016 1 commit
  5. 11 Feb, 2015 1 commit
    • antirez's avatar
      SPOP: reimplemented for speed and better distribution. · 9feee428
      antirez authored
      The old version of SPOP with "count" argument used an API call of dict.c
      which was actually designed for a different goal, and was not capable of
      good distribution. We follow a different three-cases approach optimized
      for different ratiion between sets and requested number of elements.
      
      The implementation is simpler and allowed the removal of a large amount
      of code.
      9feee428
  6. 23 Dec, 2014 1 commit
    • Matt Stancliff's avatar
      Allow all code tests to run using Redis args · 8febcffd
      Matt Stancliff authored
      Previously, many files had individual main() functions for testing,
      but each required being compiled with their own testing flags.
      That gets difficult when you have 8 different flags you need
      to set just to run all tests (plus, some test files required
      other files to be compiled aaginst them, and it seems some didn't
      build at all without including the rest of Redis).
      
      Now all individual test main() funcions are renamed to a test
      function for the file itself and one global REDIS_TEST define enables
      testing across the entire codebase.
      
      Tests can now be run with:
        - `./redis-server test <test>`
      
        e.g. ./redis-server test ziplist
      
      If REDIS_TEST is not defined, then no tests get included and no
      tests are included in the final redis-server binary.
      8febcffd
  7. 21 Dec, 2014 1 commit
    • Alon Diamant's avatar
      Following @mattsta's friendly review: · d74a5a08
      Alon Diamant authored
        1. memory leak in t_set.c has been fixed
        2. end-of-line spaces has been removed (from all over the place)
        3. for loops have been ordered up to match existing Redis style (less weird)
        4. comments format has been fixed (added * in the beggining of every comment line)
      d74a5a08
  8. 14 Dec, 2014 1 commit
    • Alon Diamant's avatar
      Added <count> parameter to SPOP: · 28802887
      Alon Diamant authored
      spopCommand() now runs spopWithCountCommand() in case the <count> param is found.
      Added intsetRandomMembers() to Intset: Copies N random members from the set into inputted 'values' array. Uses either the Knuth or Floyd sample algos depending on ratio count/size.
      Added setTypeRandomElements() to SET type: Returns a number of random elements from a non empty set. This is a version of setTypeRandomElement() that is modified in order to return multiple entries, using dictGetRandomKeys() and intsetRandomMembers().
      Added tests for SPOP with <count>: unit/type/set, unit/scripting, integration/aof
      --
      Cleaned up code a bit to match with required Redis coding style
      28802887
  9. 07 Aug, 2014 1 commit
  10. 08 Nov, 2012 1 commit
  11. 14 Feb, 2012 2 commits
  12. 09 Mar, 2011 1 commit
  13. 28 Feb, 2011 1 commit
  14. 09 Dec, 2010 1 commit
  15. 26 Aug, 2010 3 commits