1. 17 Jul, 2019 1 commit
    • Oran Agra's avatar
      Module API for Forking · 56258c6b
      Oran Agra authored
      * create module API for forking child processes.
      * refactor duplicate code around creating and tracking forks by AOF and RDB.
      * child processes listen to SIGUSR1 and dies exitFromChild in order to
        eliminate a valgrind warning of unhandled signal.
      * note that BGSAVE error reply has changed.
      
      valgrind error is:
        Process terminating with default action of signal 10 (SIGUSR1)
      56258c6b
  2. 08 Jul, 2019 1 commit
    • Oran Agra's avatar
      diskless replication on slave side (don't store rdb to file), plus some other related fixes · 2de544cf
      Oran Agra authored
      The implementation of the diskless replication was currently diskless only on the master side.
      The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.
      
      This commit adds two modes to load rdb directly from socket:
      1) when-empty
      2) using "swapdb"
      the third mode of using diskless slave by flushdb is risky and currently not included.
      
      other changes:
      --------------
      distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
      succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
      also a CONFIG GET and INFO during rdb loading would have lied
      
      When loading rdb from the network, don't kill the server on short read (that can be a network error)
      
      Fix rdb check when performed on preamble AOF
      
      tests:
      run replication tests for diskless slave too
      make replication test a bit more aggressive
      Add test for diskless load swapdb
      2de544cf
  3. 15 Mar, 2019 1 commit
  4. 04 Mar, 2019 1 commit
  5. 02 Mar, 2019 1 commit
    • antirez's avatar
      Use the RDB info fields to provide info to users. · b9b140e2
      antirez authored
      Fix #5790 and 5878.
      
      Maybe a better option was to have such fields named with the first
      byte '%' as those are info fields for specification, however now to
      break it in a backward incompatible way is not an option, so let's use
      the fields actively to provide info when sensible, otherwise ignore
      when they are not really helpful.
      b9b140e2
  6. 21 Jan, 2019 1 commit
  7. 28 Nov, 2018 1 commit
  8. 03 Jul, 2018 1 commit
  9. 21 Jun, 2018 1 commit
  10. 20 Jun, 2018 1 commit
    • Guy Benoish's avatar
      Enhance RESTORE with RDBv9 new features · b5197f1f
      Guy Benoish authored
      RESTORE now supports:
      1. Setting LRU/LFU
      2. Absolute-time TTL
      
      Other related changes:
      1. RDB loading will not override LRU bits when RDB file
         does not contain the LRU opcode.
      2. RDB loading will not set LRU/LFU bits if the server's
         maxmemory-policy does not match.
      b5197f1f
  11. 19 Jun, 2018 1 commit
  12. 12 Jun, 2018 3 commits
  13. 29 May, 2018 2 commits
    • antirez's avatar
      Don't expire keys while loading RDB from AOF preamble. · 49147f36
      antirez authored
      The AOF tail of a combined RDB+AOF is based on the premise of applying
      the AOF commands to the exact state that there was in the server while
      the RDB was persisted. By expiring keys while loading the RDB file, we
      change the state, so applying the AOF tail later may change the state.
      
      Test case:
      
      * Time1: SET a 10
      * Time2: EXPIREAT a $time5
      * Time3: INCR a
      * Time4: PERSIT A. Start bgrewiteaof with RDB preamble. The value of a is 11 without expire time.
      * Time5: Restart redis from the RDB+AOF: consistency violation.
      
      Thanks to @soloestoy for providing the patch.
      Thanks to @trevor211 for the original issue report and the initial fix.
      
      Check issue #4950 for more info.
      49147f36
    • WuYunlong's avatar
      Fix rdb save by allowing dumping of expire keys, so that when · 2a887bd5
      WuYunlong authored
      we add a new slave, and do a failover, eighter by manual or
      not, other local slaves will delete the expired keys properly.
      2a887bd5
  14. 09 May, 2018 2 commits
  15. 08 May, 2018 1 commit
  16. 22 Apr, 2018 1 commit
  17. 16 Mar, 2018 1 commit
  18. 15 Mar, 2018 11 commits
  19. 27 Feb, 2018 1 commit
  20. 29 Dec, 2017 1 commit
    • Oran Agra's avatar
      fix processing of large bulks (above 2GB) · 60a4f12f
      Oran Agra authored
      - protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
        readQueryFromClient potential overflow
      - rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
      - several places in sds.c that used int for string length or index.
      - bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
      - RM_SaveStringBuffer was limitted to 32bit length
      60a4f12f
  21. 04 Dec, 2017 1 commit
    • antirez's avatar
      Refactoring: improve luaCreateFunction() API. · 60d26acf
      antirez authored
      The function in its initial form, and after the fixes for the PSYNC2
      bugs, required code duplication in multiple spots. This commit modifies
      it in order to always compute the script name independently, and to
      return the SDS of the SHA of the body: this way it can be used in all
      the places, including for SCRIPT LOAD, without duplicating the code to
      create the Lua function name. Note that this requires to re-compute the
      body SHA1 in the case of EVAL seeing a script for the first time, but
      this should not change scripting performance in any way because new
      scripts definition is a rare event happening the first time a script is
      seen, and the SHA1 computation is anyway not a very slow process against
      the typical Redis script and compared to the actua Lua byte compiling of
      the body.
      
      Note that the function used to assert() if a duplicated script was
      loaded, however actually now two times over three, we want the function
      to handle duplicated scripts just fine: this happens in SCRIPT LOAD and
      in RDB AUX "lua" loading. Moreover the assert was not defending against
      some obvious failure mode, so now the function always tests against
      already defined functions at start.
      60d26acf
  22. 01 Dec, 2017 5 commits
    • antirez's avatar
      Fix loading of RDB files lua AUX fields when the script is defined. · 65a9740f
      antirez authored
      In the case of slaves loading the RDB from master, or in other similar
      cases, the script is already defined, and the function registering the
      script should not fail in the assert() call.
      65a9740f
    • antirez's avatar
      Streams: delta encode IDs based on key. Add count + deleted fields. · f24d3a7d
      antirez authored
      We used to have the master ID stored at the start of the listpack,
      however using the key directly makes more sense in order to create a
      space efficient representation: anyway the key at the radix tree is very
      unlikely to change because of how the stream is implemented. Moreover on
      nodes merging, to rewrite the merged listpacks is anyway the most
      sensible operation, and we can use the iterator and the append-to-stream
      function in order to avoid re-implementing the code needed for merging.
      
      This commit also adds two items at the start of the listpack: the
      number of valid items inside the listpack, and the number of items
      marked as deleted. This means that there is no need to scan a listpack
      in order to understand if it's a good candidate for garbage collection,
      if the ration between valid/deleted items triggers the GC.
      f24d3a7d
    • antirez's avatar
      Streams: Save stream->length in RDB. · 98d184db
      antirez authored
      98d184db
    • antirez's avatar
      Streams: RDB loading. RDB saving modified. · edd70c19
      antirez authored
      After a few attempts it looked quite saner to just add the last item ID
      at the end of the serialized listpacks, instead of scanning the last
      listpack loaded from head to tail just to fetch it. It's a disk space VS
      CPU-and-simplicity tradeoff basically.
      edd70c19
    • antirez's avatar
      Streams: RDB saving. · 485014cc
      antirez authored
      485014cc