1. 20 Jun, 2018 1 commit
    • Guy Benoish's avatar
      Enhance RESTORE with RDBv9 new features · b5197f1f
      Guy Benoish authored
      RESTORE now supports:
      1. Setting LRU/LFU
      2. Absolute-time TTL
      
      Other related changes:
      1. RDB loading will not override LRU bits when RDB file
         does not contain the LRU opcode.
      2. RDB loading will not set LRU/LFU bits if the server's
         maxmemory-policy does not match.
      b5197f1f
  2. 12 Jun, 2018 3 commits
  3. 29 May, 2018 2 commits
    • antirez's avatar
      Don't expire keys while loading RDB from AOF preamble. · 49147f36
      antirez authored
      The AOF tail of a combined RDB+AOF is based on the premise of applying
      the AOF commands to the exact state that there was in the server while
      the RDB was persisted. By expiring keys while loading the RDB file, we
      change the state, so applying the AOF tail later may change the state.
      
      Test case:
      
      * Time1: SET a 10
      * Time2: EXPIREAT a $time5
      * Time3: INCR a
      * Time4: PERSIT A. Start bgrewiteaof with RDB preamble. The value of a is 11 without expire time.
      * Time5: Restart redis from the RDB+AOF: consistency violation.
      
      Thanks to @soloestoy for providing the patch.
      Thanks to @trevor211 for the original issue report and the initial fix.
      
      Check issue #4950 for more info.
      49147f36
    • WuYunlong's avatar
      Fix rdb save by allowing dumping of expire keys, so that when · 2a887bd5
      WuYunlong authored
      we add a new slave, and do a failover, eighter by manual or
      not, other local slaves will delete the expired keys properly.
      2a887bd5
  4. 09 May, 2018 2 commits
  5. 08 May, 2018 1 commit
  6. 22 Apr, 2018 1 commit
  7. 16 Mar, 2018 1 commit
  8. 15 Mar, 2018 11 commits
  9. 27 Feb, 2018 1 commit
  10. 29 Dec, 2017 1 commit
    • Oran Agra's avatar
      fix processing of large bulks (above 2GB) · 60a4f12f
      Oran Agra authored
      - protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
        readQueryFromClient potential overflow
      - rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
      - several places in sds.c that used int for string length or index.
      - bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
      - RM_SaveStringBuffer was limitted to 32bit length
      60a4f12f
  11. 04 Dec, 2017 1 commit
    • antirez's avatar
      Refactoring: improve luaCreateFunction() API. · 60d26acf
      antirez authored
      The function in its initial form, and after the fixes for the PSYNC2
      bugs, required code duplication in multiple spots. This commit modifies
      it in order to always compute the script name independently, and to
      return the SDS of the SHA of the body: this way it can be used in all
      the places, including for SCRIPT LOAD, without duplicating the code to
      create the Lua function name. Note that this requires to re-compute the
      body SHA1 in the case of EVAL seeing a script for the first time, but
      this should not change scripting performance in any way because new
      scripts definition is a rare event happening the first time a script is
      seen, and the SHA1 computation is anyway not a very slow process against
      the typical Redis script and compared to the actua Lua byte compiling of
      the body.
      
      Note that the function used to assert() if a duplicated script was
      loaded, however actually now two times over three, we want the function
      to handle duplicated scripts just fine: this happens in SCRIPT LOAD and
      in RDB AUX "lua" loading. Moreover the assert was not defending against
      some obvious failure mode, so now the function always tests against
      already defined functions at start.
      60d26acf
  12. 01 Dec, 2017 5 commits
    • antirez's avatar
      Fix loading of RDB files lua AUX fields when the script is defined. · 65a9740f
      antirez authored
      In the case of slaves loading the RDB from master, or in other similar
      cases, the script is already defined, and the function registering the
      script should not fail in the assert() call.
      65a9740f
    • antirez's avatar
      Streams: delta encode IDs based on key. Add count + deleted fields. · f24d3a7d
      antirez authored
      We used to have the master ID stored at the start of the listpack,
      however using the key directly makes more sense in order to create a
      space efficient representation: anyway the key at the radix tree is very
      unlikely to change because of how the stream is implemented. Moreover on
      nodes merging, to rewrite the merged listpacks is anyway the most
      sensible operation, and we can use the iterator and the append-to-stream
      function in order to avoid re-implementing the code needed for merging.
      
      This commit also adds two items at the start of the listpack: the
      number of valid items inside the listpack, and the number of items
      marked as deleted. This means that there is no need to scan a listpack
      in order to understand if it's a good candidate for garbage collection,
      if the ration between valid/deleted items triggers the GC.
      f24d3a7d
    • antirez's avatar
      Streams: Save stream->length in RDB. · 98d184db
      antirez authored
      98d184db
    • antirez's avatar
      Streams: RDB loading. RDB saving modified. · edd70c19
      antirez authored
      After a few attempts it looked quite saner to just add the last item ID
      at the end of the serialized listpacks, instead of scanning the last
      listpack loaded from head to tail just to fetch it. It's a disk space VS
      CPU-and-simplicity tradeoff basically.
      edd70c19
    • antirez's avatar
      Streams: RDB saving. · 485014cc
      antirez authored
      485014cc
  13. 30 Nov, 2017 2 commits
    • antirez's avatar
      PSYNC2: just store script bodies into RDB. · 452ad2e9
      antirez authored
      Related to #4483. As suggested by @soloestoy, we can retrieve the SHA1
      from the body. Given that in the new implementation using AUX fields we
      ended copying around a lot to create new objects and strings, extremize
      such concept and trade CPU for space inside the RDB file.
      452ad2e9
    • antirez's avatar
      PSYNC2: Save Lua scripts state into RDB file. · f11a7585
      antirez authored
      This is currently needed in order to fix #4483, but this can be
      useful in other contexts, so maybe later we may want to remove the
      conditionals and always save/load scripts.
      
      Note that we are using the "lua" AUX field here, in order to guarantee
      backward compatibility of the RDB file. The unknown AUX fields must be
      discarded by past versions of Redis.
      f11a7585
  14. 24 Nov, 2017 1 commit
  15. 22 Nov, 2017 2 commits
  16. 06 Nov, 2017 1 commit
    • antirez's avatar
      Fix saving of zero-length lists. · a1944c3e
      antirez authored
      Normally in modern Redis you can't create zero-len lists, however it's
      possible to load them from old RDB files generated, for instance, using
      Redis 2.8 (see issue #4409). The "Right Thing" would be not loading such
      lists at all, but this requires to hook in rdb.c random places in a not
      great way, for a problem that is at this point, at best, minor.
      
      Here in this commit instead I just fix the fact that zero length lists,
      materialized as quicklists with the first node set to NULL, were
      iterated in the wrong way while they are saved, leading to a crash.
      
      The other parts of the list implementation are apparently able to deal
      with empty lists correctly, even if they are no longer a thing.
      a1944c3e
  17. 02 Nov, 2017 1 commit
  18. 01 Nov, 2017 1 commit
  19. 20 Sep, 2017 2 commits
    • antirez's avatar
      PSYNC2: More refinements related to #4316. · bb3b5ddd
      antirez authored
      bb3b5ddd
    • zhaozhao.zz's avatar
      PSYNC2: make persisiting replication info more solid · b541ccef
      zhaozhao.zz authored
      This commit is a reinforcement of commit c1c99e9f.
      
      1. Replication information can be stored when the RDB file is
      generated by a mater using server.slaveseldb when server.repl_backlog
      is not NULL, or set repl_stream_db be -1. That's safe, because
      NULL server.repl_backlog will trigger full synchronization,
      then master will send SELECT command to replicaiton stream.
      2. Only do rdbSave* when rsiptr is not NULL,
      if we do rdbSave* without rdbSaveInfo, slave will miss repl-stream-db.
      3. Save the replication informations also in the case of
      SAVE command, FLUSHALL command and DEBUG reload.
      b541ccef