1. 02 Oct, 2023 1 commit
    • YaacovHazan's avatar
      Stabilization and improvements around aof tests (#12626) · 2e0f6724
      YaacovHazan authored
      In some tests, the code manually searches for a log message, and it
      uses tail -1 with a delay of 1 second, which can miss the expected line.
      
      Also, because the aof tests use start_server_aof and not start_server,
      the test name doesn't log into the server log.
      
      To fix the above, I made the following changes:
      - Change the start_server_aof to wrap the start_server.
        This will add the created aof server to the servers list, and make
        srv() and wait_for_log_messages() available for the tests.
      
      - Introduce a new option for start_server.
        'wait_ready' - an option to let the caller start the test code without
        waiting for the server to be ready. useful for tests on a server that
        is expected to exit on startup.
      
      - Create a new start_server_aof_ex.
        The new proc also accept options as argument and make use of the
        new 'short_life' option for tests that are expected to exit on startup
        because of some error in the aof file(s).
      
      Because of the above, I had to change many lines and replace every
      local srv variable (a server config) usage with the srv().
      2e0f6724
  2. 12 Apr, 2023 1 commit
    • Oran Agra's avatar
      Attempt to solve MacOS CI issues in GH Actions (#12013) · 997fa41e
      Oran Agra authored
      The MacOS CI in github actions often hangs without any logs. GH argues that
      it's due to resource utilization, either running out of disk space, memory, or CPU
      starvation, and thus the runner is terminated.
      
      This PR contains multiple attempts to resolve this:
      1. introducing pause_process instead of SIGSTOP, which waits for the process
        to stop before resuming the test, possibly resolving race conditions in some tests,
        this was a suspect since there was one test that could result in an infinite loop in that
       case, in practice this didn't help, but still a good idea to keep.
      2. disable the `save` config in many tests that don't need it, specifically ones that use
        heavy writes and could create large files.
      3. change the `populate` proc to use short pipeline rather than an infinite one.
      4. use `--clients 1` in the macos CI so that we don't risk running multiple resource
        demanding tests in parallel.
      5. enable `--verbose` to be repeated to elevate verbosity and print more info to stdout
        when a test or a server starts.
      997fa41e
  3. 26 Apr, 2022 1 commit
    • chenyang8094's avatar
      Fix bug when AOF enabled after startup. put the new incr file in the manifest... · 46ec6ad9
      chenyang8094 authored
      Fix bug when AOF enabled after startup. put the new incr file in the manifest only when AOFRW is done. (#10616)
      
      Changes:
      
      - When AOF is enabled **after** startup, the data accumulated during `AOF_WAIT_REWRITE`
        will only be stored in a temp INCR AOF file. Only after the first AOFRW is successful, we will
        add it to manifest file.
        Before this fix, the manifest referred to the temp file which could cause a restart during that
        time to load it without it's base.
      - Add `aof_rewrites_consecutive_failures` info field for  aofrw limiting implementation.
      
      Now we can guarantee that these behaviors of MP-AOF are the same as before (past redis releases):
      - When AOF is enabled after startup, the data accumulated during `AOF_WAIT_REWRITE` will only
        be stored in a visible place. Only after the first AOFRW is successful, we will add it to manifest file.
      - When disable AOF, we did not delete the AOF file in the past so there's no need to change that
        behavior now (yet).
      - When toggling AOF off and then on (could be as part of a full-sync), a crash or restart before the
        first rewrite is completed, would result with the previous version being loaded (might not be right thing,
        but that's what we always had).
      46ec6ad9
  4. 22 Feb, 2022 1 commit
    • YaacovHazan's avatar
      fix return value of loadAppendOnlyFiles (#10295) · 65e4bce0
      YaacovHazan authored
      
      
      Make sure the status return from loading multiple AOF files reflects the overall
      result, not just the one of the last file.
      
      When one of the AOF files succeeded to load, but the last AOF file
      was empty, the loadAppendOnlyFiles will return AOF_EMPTY.
      This commit changes this behavior, and return AOF_OK in that case.
      
      This can happen for example, when loading old AOF file, and no more commands processed,
      the manifest file will include base AOF file with data, and empty incr AOF file.
      Co-authored-by: default avatarchenyang8094 <chenyang8094@users.noreply.github.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      65e4bce0
  5. 17 Feb, 2022 1 commit
    • chenyang8094's avatar
      Adapt redis-check-aof tool for Multi Part Aof (#10061) · a50aa29b
      chenyang8094 authored
      Modifications of this PR:
      1. Support the verification of `Multi Part AOF`, while still maintaining support for the
        old-style `AOF/RDB-preamble`. `redis-check-aof` will automatically choose which
        mode to use according to the incoming file format.
         
      `Usage: redis-check-aof [--fix|--truncate-to-timestamp $timestamp] <AOF/manifest>`
       
      2. Refactor part of the code to make it easier to understand
      3. Currently only supports truncate  (`--fix` or `--truncate-to-timestamp`) the last AOF
        file (may be `BASE` or `INCR`)
      
      The reasons for 3 above:
      - for `--fix`: Only the last AOF may be truncated, this is guaranteed by redis
      - for `--truncate-to-timestamp`:  Normally, we only have `BASE` + `INCR` files
        at most, and `BASE` cannot be truncated(It only contains a timestamp annotation
        at the beginning of the file), so only `INCR` can be truncated. If we have a
        `BASE+INCR1+INCR2` file (meaning we have an interrupted AOFRW), Only `INCR2`
        files can be truncated at this time. If we still insist on truncate `INCR1`, we need to
        manually delete `INCR2` and update the manifest file, then re-run `redis-check-aof`
      - If we want to support truncate any file, we need to add very complicated code to support
        the atomic modification of multiple file deletion and update manifest, I think this is unnecessary
      a50aa29b
  6. 24 Jan, 2022 1 commit
  7. 18 Jan, 2022 1 commit
    • Yossi Gottlieb's avatar
      Fix additional AOF filename issues. (#10110) · 25e6d4d4
      Yossi Gottlieb authored
      This extends the previous fix (#10049) to address any form of
      non-printable or whitespace character (including newlines, quotes,
      non-printables, etc.)
      
      Also, removes the limitation on appenddirname, to align with the way
      filenames are handled elsewhere in Redis.
      25e6d4d4
  8. 13 Jan, 2022 1 commit
    • chenyang8094's avatar
      Always create base AOF file when redis start from empty. (#10102) · e9bff797
      chenyang8094 authored
      
      
      Force create a BASE file (use a foreground `rewriteAppendOnlyFile`) when redis starts from an
      empty data set and  `appendonly` is  yes.
      
      The reasoning is that normally, after redis is running for some time, and the AOF has gone though
      a few rewrites, there's always a base rdb file. and the scenario where the base file is missing, is
      kinda rare (happens only at empty startup), so this change normalizes it.
      But more importantly, there are or could be some complex modules that are started with some
      configuration, when they create persistence they write that configuration to RDB AUX fields, so
      that can can always know with which configuration the persistence file they're loading was
      created (could be critical). there is (was) one scenario in which they could load their persisted data,
      and that configuration was missing, and this change fixes it.
      
      Add a new module event: REDISMODULE_SUBEVENT_PERSISTENCE_SYNC_AOF_START, similar to
      REDISMODULE_SUBEVENT_PERSISTENCE_AOF_START which is async.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      e9bff797
  9. 10 Jan, 2022 1 commit
  10. 03 Jan, 2022 1 commit
    • chenyang8094's avatar
      Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) · 87789fae
      chenyang8094 authored
      
      
      Implement Multi-Part AOF mechanism to avoid overheads during AOFRW.
      Introducing a folder with multiple AOF files tracked by a manifest file.
      
      The main issues with the the original AOFRW mechanism are:
      * buffering of commands that are processed during rewrite (consuming a lot of RAM)
      * freezes of the main process when the AOFRW completes to drain the remaining part of the buffer and fsync it.
      * double disk IO for the data that arrives during AOFRW (had to be written to both the old and new AOF files)
      
      The main modifications of this PR:
      1. Remove the AOF rewrite buffer and related code.
      2. Divide the AOF into multiple files, they are classified as two types, one is the the `BASE` type,
        it represents the full amount of data (Maybe AOF or RDB format) after each AOFRW, there is only
        one `BASE` file at most. The second is `INCR` type, may have more than one. They represent the
        incremental commands since the last AOFRW.
      3. Use a AOF manifest file to record and manage these AOF files mentioned above.
      4. The original configuration of `appendfilename` will be the base part of the new file name, for example:
        `appendonly.aof.1.base.rdb` and `appendonly.aof.2.incr.aof`
      5. Add manifest-related TCL tests, and modified some existing tests that depend on the `appendfilename`
      6. Remove the `aof_rewrite_buffer_length` field in info.
      7. Add `aof-disable-auto-gc` configuration. By default we're automatically deleting HISTORY type AOFs.
        It also gives users the opportunity to preserve the history AOFs. just for testing use now.
      8. Add AOFRW limiting measure. When the AOFRW failures reaches the threshold (3 times now),
        we will delay the execution of the next AOFRW by 1 minute. If the next AOFRW also fails, it will be
        delayed by 2 minutes. The next is 4, 8, 16, the maximum delay is 60 minutes (1 hour). During the limit
        period, we can still use the 'bgrewriteaof' command to execute AOFRW immediately.
      9. Support upgrade (load) data from old version redis.
      10. Add `appenddirname` configuration, as the directory name of the append only files. All AOF files and
        manifest file will be placed in this directory.
      11. Only the last AOF file (BASE or INCR) can be truncated. Otherwise redis will exit even if
        `aof-load-truncated` is enabled.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      87789fae