1. 20 Jan, 2022 1 commit
  2. 05 Jan, 2022 1 commit
    • sundb's avatar
      Show the elapsed time of single test and speed up some tests (#10058) · 4d3c4cfa
      sundb authored
      Following #10038.
      
      This PR introduces two changes.
      1. Show the elapsed time of a single test in the test output, in order to have a more
      detailed understanding of the changes in test run time.
      
      2. Speedup two tests related to `key-load-delay` configuration.
      other tests do not seem to be affected by #10003.
      4d3c4cfa
  3. 03 Jan, 2022 2 commits
    • chenyang8094's avatar
      Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) · 87789fae
      chenyang8094 authored
      
      
      Implement Multi-Part AOF mechanism to avoid overheads during AOFRW.
      Introducing a folder with multiple AOF files tracked by a manifest file.
      
      The main issues with the the original AOFRW mechanism are:
      * buffering of commands that are processed during rewrite (consuming a lot of RAM)
      * freezes of the main process when the AOFRW completes to drain the remaining part of the buffer and fsync it.
      * double disk IO for the data that arrives during AOFRW (had to be written to both the old and new AOF files)
      
      The main modifications of this PR:
      1. Remove the AOF rewrite buffer and related code.
      2. Divide the AOF into multiple files, they are classified as two types, one is the the `BASE` type,
        it represents the full amount of data (Maybe AOF or RDB format) after each AOFRW, there is only
        one `BASE` file at most. The second is `INCR` type, may have more than one. They represent the
        incremental commands since the last AOFRW.
      3. Use a AOF manifest file to record and manage these AOF files mentioned above.
      4. The original configuration of `appendfilename` will be the base part of the new file name, for example:
        `appendonly.aof.1.base.rdb` and `appendonly.aof.2.incr.aof`
      5. Add manifest-related TCL tests, and modified some existing tests that depend on the `appendfilename`
      6. Remove the `aof_rewrite_buffer_length` field in info.
      7. Add `aof-disable-auto-gc` configuration. By default we're automatically deleting HISTORY type AOFs.
        It also gives users the opportunity to preserve the history AOFs. just for testing use now.
      8. Add AOFRW limiting measure. When the AOFRW failures reaches the threshold (3 times now),
        we will delay the execution of the next AOFRW by 1 minute. If the next AOFRW also fails, it will be
        delayed by 2 minutes. The next is 4, 8, 16, the maximum delay is 60 minutes (1 hour). During the limit
        period, we can still use the 'bgrewriteaof' command to execute AOFRW immediately.
      9. Support upgrade (load) data from old version redis.
      10. Add `appenddirname` configuration, as the directory name of the append only files. All AOF files and
        manifest file will be placed in this directory.
      11. Only the last AOF file (BASE or INCR) can be truncated. Otherwise redis will exit even if
        `aof-load-truncated` is enabled.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      87789fae
    • Harkrishn Patro's avatar
      Sharded pubsub implementation (#8621) · 9f888576
      Harkrishn Patro authored
      
      
      This commit implements a sharded pubsub implementation based off of shard channels.
      Co-authored-by: default avatarHarkrishn Patro <harkrisp@amazon.com>
      Co-authored-by: default avatarMadelyn Olson <madelyneolson@gmail.com>
      9f888576
  4. 02 Jan, 2022 1 commit
    • Viktor Söderqvist's avatar
      Wait for replicas when shutting down (#9872) · 45a155bd
      Viktor Söderqvist authored
      
      
      To avoid data loss, this commit adds a grace period for lagging replicas to
      catch up the replication offset.
      
      Done:
      
      * Wait for replicas when shutdown is triggered by SIGTERM and SIGINT.
      
      * Wait for replicas when shutdown is triggered by the SHUTDOWN command. A new
        blocked client type BLOCKED_SHUTDOWN is introduced, allowing multiple clients
        to call SHUTDOWN in parallel.
        Note that they don't expect a response unless an error happens and shutdown is aborted.
      
      * Log warning for each replica lagging behind when finishing shutdown.
      
      * CLIENT_PAUSE_WRITE while waiting for replicas.
      
      * Configurable grace period 'shutdown-timeout' in seconds (default 10).
      
      * New flags for the SHUTDOWN command:
      
          - NOW disables the grace period for lagging replicas.
      
          - FORCE ignores errors writing the RDB or AOF files which would normally
            prevent a shutdown.
      
          - ABORT cancels ongoing shutdown. Can't be combined with other flags.
      
      * New field in the output of the INFO command: 'shutdown_in_milliseconds'. The
        value is the remaining maximum time to wait for lagging replicas before
        finishing the shutdown. This field is present in the Server section **only**
        during shutdown.
      
      Not directly related:
      
      * When shutting down, if there is an AOF saving child, it is killed **even** if AOF
        is disabled. This can happen if BGREWRITEAOF is used when AOF is off.
      
      * Client pause now has end time and type (WRITE or ALL) per purpose. The
        different pause purposes are *CLIENT PAUSE command*, *failover* and
        *shutdown*. If clients are unpaused for one purpose, it doesn't affect client
        pause for other purposes. For example, the CLIENT UNPAUSE command doesn't
        affect client pause initiated by the failover or shutdown procedures. A completed
        failover or a failed shutdown doesn't unpause clients paused by the CLIENT
        PAUSE command.
      
      Notes:
      
      * DEBUG RESTART doesn't wait for replicas.
      
      * We already have a warning logged when a replica disconnects. This means that
        if any replica connection is lost during the shutdown, it is either logged as
        disconnected or as lagging at the time of exit.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      45a155bd
  5. 16 Dec, 2021 1 commit
    • Meir Shpilraien (Spielrein)'s avatar
      Add FUNCTION FLUSH command to flush all functions (#9936) · 687210f1
      Meir Shpilraien (Spielrein) authored
      Added `FUNCTION FLUSH` command. The new sub-command allows delete all the functions.
      An optional `[SYNC|ASYNC]` argument can be given to control whether or not to flush the
      functions synchronously or asynchronously. if not given the default flush mode is chosen by
      `lazyfree-lazy-user-flush` configuration values.
      
      Add the missing `functions.tcl` test to the list of tests that are executed in test_helper.tcl,
      and call FUNCTION FLUSH in between servers in external mode
      687210f1
  6. 13 Dec, 2021 1 commit
    • yoav-steinberg's avatar
      Fix possible int overflow when hashing an sds. (#9916) · c7dc17fc
      yoav-steinberg authored
      This caused a crash when adding elements larger than 2GB to a set (same goes for hash keys). See #8455.
      
      Details:
      * The fix makes the dict hash functions receive a `size_t` instead of an `int`. In practice the dict hash functions
        call siphash which receives a `size_t` and the callers to the hash function pass a `size_t` to it so the fix is trivial.
      * The issue was recreated by attempting to add a >2gb value to a set. Appropriate tests were added where I create
        a set with large elements and check basic functionality on it (SADD, SCARD, SPOP, etc...).
      * When I added the tests I also refactored a bit all the tests code which is run under the `--large-memory` flag.
        This removed code duplication for the test framework's `write_big_bulk` and `write_big_bulk` code and also takes
        care of not allocating the test frameworks helper huge string used by these tests when not run under `--large-memory`.
      * I also added the _violoations.tcl_ unit tests to be part of the entire test suite and leaned up non relevant list related
        tests that were in there. This was done in this PR because most of the _violations_ tests are "large memory" tests.
      c7dc17fc
  7. 10 Nov, 2021 1 commit
    • Oran Agra's avatar
      Increase test timeout in valgrind runs (#9767) · 978eadba
      Oran Agra authored
      We saw some tests sporadically time out on valgrind (namely the ones
      from #9323).
      
      Increasing valgrind timeout from 20 mins to 40 mins in CI.
      And fixing an outdated help message.
      978eadba
  8. 03 Nov, 2021 1 commit
    • perryitay's avatar
      Add support for list type to store elements larger than 4GB (#9357) · f27083a4
      perryitay authored
      
      
      Redis lists are stored in quicklist, which is currently a linked list of ziplists.
      Ziplists are limited to storing elements no larger than 4GB, so when bigger
      items are added they're getting truncated.
      This PR changes quicklists so that they're capable of storing large items
      in quicklist nodes that are plain string buffers rather than ziplist.
      
      As part of the PR there were few other changes in redis: 
      1. new DEBUG sub-commands: 
         - QUICKLIST-PACKED-THRESHOLD - set the threshold of for the node type to
           be plan or ziplist. default (1GB)
         - QUICKLIST <key> - Shows low level info about the quicklist encoding of <key>
      2. rdb format change:
         - A new type was added - RDB_TYPE_LIST_QUICKLIST_2 . 
         - container type (packed / plain) was added to the beginning of the rdb object
           (before the actual node list).
      3. testing:
         - Tests that requires over 100MB will be by default skipped. a new flag was
           added to 'runtest' to run the large memory tests (not used by default)
      Co-authored-by: default avatarsundb <sundbcn@gmail.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      f27083a4
  9. 25 Oct, 2021 1 commit
    • Wang Yuan's avatar
      Replication backlog and replicas use one global shared replication buffer (#9166) · c1718f9d
      Wang Yuan authored
      ## Background
      For redis master, one replica uses one copy of replication buffer, that is a big waste of memory,
      more replicas more waste, and allocate/free memory for every reply list also cost much.
      If we set client-output-buffer-limit small and write traffic is heavy, master may disconnect with
      replicas and can't finish synchronization with replica. If we set  client-output-buffer-limit big,
      master may be OOM when there are many replicas that separately keep much memory.
      Because replication buffers of different replica client are the same, one simple idea is that
      all replicas only use one replication buffer, that will effectively save memory.
      
      Since replication backlog content is the same as replicas' output buffer, now we
      can discard replication backlog memory and use global shared replication buffer
      to implement replication backlog mechanism.
      
      ## Implementation
      I create one global "replication buffer" which contains content of replication stream.
      The structure of "replication buffer" is similar to the reply list that exists in every client.
      But the node of list is `replBufBlock`, which has `id, repl_offset, refcount` fields.
      ```c
      /* Replication buffer blocks is the list of replBufBlock.
       *
       * +--------------+       +--------------+       +--------------+
       * | refcount = 1 |  ...  | refcount = 0 |  ...  | refcount = 2 |
       * +--------------+       +--------------+       +--------------+
       *      |                                            /       \
       *      |                                           /         \
       *      |                                          /           \
       *  Repl Backlog                               Replia_A      Replia_B
       * 
       * Each replica or replication backlog increments only the refcount of the
       * 'ref_repl_buf_node' which it points to. So when replica walks to the next
       * node, it should first increase the next node's refcount, and when we trim
       * the replication buffer nodes, we remove node always from the head node which
       * refcount is 0. If the refcount of the head node is not 0, we must stop
       * trimming and never iterate the next node. */
      
      /* Similar with 'clientReplyBlock', it is used for shared buffers between
       * all replica clients and replication backlog. */
      typedef struct replBufBlock {
          int refcount;           /* Number of replicas or repl backlog using. */
          long long id;           /* The unique incremental number. */
          long long repl_offset;  /* Start replication offset of the block. */
          size_t size, used;
          char buf[];
      } replBufBlock;
      ```
      So now when we feed replication stream into replication backlog and all replicas, we only need
      to feed stream into replication buffer `feedReplicationBuffer`. In this function, we set some fields of
      replication backlog and replicas to references of the global replication buffer blocks. And we also
      need to check replicas' output buffer limit to free if exceeding `client-output-buffer-limit`, and trim
      replication backlog if exceeding `repl-backlog-size`.
      
      When sending reply to replicas, we also need to iterate replication buffer blocks and send its
      content, when totally sending one block for replica, we decrease current node count and
      increase the next current node count, and then free the block which reference is 0 from the
      head of replication buffer blocks.
      
      Since now we use linked list to manage replication backlog, it may cost much time for iterating
      all linked list nodes to find corresponding replication buffer node. So we create a rax tree to
      store some nodes  for index, but to avoid rax tree occupying too much memory, i record
      one per 64 nodes for index.
      
      Currently, to make partial resynchronization as possible as much, we always let replication
      backlog as the last reference of replication buffer blocks, backlog size may exceeds our setting
      if slow replicas that reference vast replication buffer blocks, and this method doesn't increase
      memory usage since they share replication buffer. To avoid freezing server for freeing unreferenced
      replication buffer blocks when we need to trim backlog for exceeding backlog size setting,
      we trim backlog incrementally (free 64 blocks per call now), and make it faster in
      `beforeSleep` (free 640 blocks).
      
      ### Other changes
      - `mem_total_replication_buffers`: we add this field in INFO command, it means the total
        memory of replication buffers used.
      - `mem_clients_slaves`:  now even replica is slow to replicate, and its output buffer memory
        is not 0, but it still may be 0, since replication backlog and replicas share one global replication
        buffer, only if replication buffer memory is more than the repl backlog setting size, we consider
        the excess as replicas' memory. Otherwise, we think replication buffer memory is the consumption
        of repl backlog.
      - Key eviction
        Since all replicas and replication backlog share global replication buffer, we think only the
        part of exceeding backlog size the extra separate consumption of replicas.
        Because we trim backlog incrementally in the background, backlog size may exceeds our
        setting if slow replicas that reference vast replication buffer blocks disconnect.
        To avoid massive eviction loop, we don't count the delayed freed replication backlog into
        used memory even if there are no replicas, i.e. we also regard this memory as replicas's memory.
      - `client-output-buffer-limit` check for replica clients
        It doesn't make sense to set the replica clients output buffer limit lower than the repl-backlog-size
        config (partial sync will succeed and then replica will get disconnected). Such a configuration is
        ignored (the size of repl-backlog-size will be used). This doesn't have memory consumption
        implications since the replica client will share the backlog buffers memory.
      - Drop replication backlog after loading data if needed
        We always create replication backlog if server is a master, we need it because we put DELs in
        it when loading expired keys in RDB, but if RDB doesn't have replication info or there is no rdb,
        it is not possible to support partial resynchronization, to avoid extra memory of replication backlog,
        we drop it.
      - Multi IO threads
       Since all replicas and replication backlog use global replication buffer,  if I/O threads are enabled,
        to guarantee data accessing thread safe, we must let main thread handle sending the output buffer
        to all replicas. But before, other IO threads could handle sending output buffer of all replicas.
      
      ## Other optimizations
      This solution resolve some other problem:
      - When replicas disconnect with master since of out of output buffer limit, releasing the output
        buffer of replicas may freeze server if we set big `client-output-buffer-limit` for replicas, but now,
        it doesn't cause freezing.
      - This implementation may mitigate reply list copy cost time(also freezes server) when one replication
        has huge reply buffer and another replica can copy buffer for full synchronization. now, we just copy
        reference info, it is very light.
      - If we set replication backlog size big, it also may cost much time to copy replication backlog into
        replica's output buffer. But this commit eliminates this problem.
      - Resizing replication backlog size doesn't empty current replication backlog content.
      c1718f9d
  10. 19 Oct, 2021 1 commit
    • qetu3790's avatar
      Release clients blocked on module commands in cluster resharding and down state (#9483) · 4962c552
      qetu3790 authored
      
      
      Prevent clients from being blocked forever in cluster when they block with their own module command
      and the hash slot is migrated to another master at the same time.
      These will get a redirection message when unblocked.
      Also, release clients blocked on module commands when cluster is down (same as other blocked clients)
      
      This commit adds basic tests for the main (non-cluster) redis test infra that test the cluster.
      This was done because the cluster test infra can't handle some common test features,
      but most importantly we only build the test modules with the non-cluster test suite.
      
      note that rather than really supporting cluster operations by the test infra, it was added (as dup code)
      in two files, one for module tests and one for non-modules tests, maybe in the future we'll refactor that.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      4962c552
  11. 26 Sep, 2021 1 commit
  12. 23 Sep, 2021 1 commit
    • yoav-steinberg's avatar
      Client eviction (#8687) · 2753429c
      yoav-steinberg authored
      
      
      ### Description
      A mechanism for disconnecting clients when the sum of all connected clients is above a
      configured limit. This prevents eviction or OOM caused by accumulated used memory
      between all clients. It's a complimentary mechanism to the `client-output-buffer-limit`
      mechanism which takes into account not only a single client and not only output buffers
      but rather all memory used by all clients.
      
      #### Design
      The general design is as following:
      * We track memory usage of each client, taking into account all memory used by the
        client (query buffer, output buffer, parsed arguments, etc...). This is kept up to date
        after reading from the socket, after processing commands and after writing to the socket.
      * Based on the used memory we sort all clients into buckets. Each bucket contains all
        clients using up up to x2 memory of the clients in the bucket below it. For example up
        to 1m clients, up to 2m clients, up to 4m clients, ...
      * Before processing a command and before sleep we check if we're over the configured
        limit. If we are we start disconnecting clients from larger buckets downwards until we're
        under the limit.
      
      #### Config
      `maxmemory-clients` max memory all clients are allowed to consume, above this threshold
      we disconnect clients.
      This config can either be set to 0 (meaning no limit), a size in bytes (possibly with MB/GB
      suffix), or as a percentage of `maxmemory` by using the `%` suffix (e.g. setting it to `10%`
      would mean 10% of `maxmemory`).
      
      #### Important code changes
      * During the development I encountered yet more situations where our io-threads access
        global vars. And needed to fix them. I also had to handle keeps the clients sorted into the
        memory buckets (which are global) while their memory usage changes in the io-thread.
        To achieve this I decided to simplify how we check if we're in an io-thread and make it
        much more explicit. I removed the `CLIENT_PENDING_READ` flag used for checking
        if the client is in an io-thread (it wasn't used for anything else) and just used the global
        `io_threads_op` variable the same way to check during writes.
      * I optimized the cleanup of the client from the `clients_pending_read` list on client freeing.
        We now store a pointer in the `client` struct to this list so we don't need to search in it
        (`pending_read_list_node`).
      * Added `evicted_clients` stat to `INFO` command.
      * Added `CLIENT NO-EVICT ON|OFF` sub command to exclude a specific client from the
        client eviction mechanism. Added corrosponding 'e' flag in the client info string.
      * Added `multi-mem` field in the client info string to show how much memory is used up
        by buffered multi commands.
      * Client `tot-mem` now accounts for buffered multi-commands, pubsub patterns and
        channels (partially), tracking prefixes (partially).
      * CLIENT_CLOSE_ASAP flag is now handled in a new `beforeNextClient()` function so
        clients will be disconnected between processing different clients and not only before sleep.
        This new function can be used in the future for work we want to do outside the command
        processing loop but don't want to wait for all clients to be processed before we get to it.
        Specifically I wanted to handle output-buffer-limit related closing before we process client
        eviction in case the two race with each other.
      * Added a `DEBUG CLIENT-EVICTION` command to print out info about the client eviction
        buckets.
      * Each client now holds a pointer to the client eviction memory usage bucket it belongs to
        and listNode to itself in that bucket for quick removal.
      * Global `io_threads_op` variable now can contain a `IO_THREADS_OP_IDLE` value
        indicating no io-threading is currently being executed.
      * In order to track memory used by each clients in real-time we can't rely on updating
        these stats in `clientsCron()` alone anymore. So now I call `updateClientMemUsage()`
        (used to be `clientsCronTrackClientsMemUsage()`) after command processing, after
        writing data to pubsub clients, after writing the output buffer and after reading from the
        socket (and maybe other places too). The function is written to be fast.
      * Clients are evicted if needed (with appropriate log line) in `beforeSleep()` and before
        processing a command (before performing oom-checks and key-eviction).
      * All clients memory usage buckets are grouped as follows:
        * All clients using less than 64k.
        * 64K..128K
        * 128K..256K
        * ...
        * 2G..4G
        * All clients using 4g and up.
      * Added client-eviction.tcl with a bunch of tests for the new mechanism.
      * Extended maxmemory.tcl to test the interaction between maxmemory and
        maxmemory-clients settings.
      * Added an option to flag a numeric configuration variable as a "percent", this means that
        if we encounter a '%' after the number in the config file (or config set command) we
        consider it as valid. Such a number is store internally as a negative value. This way an
        integer value can be interpreted as either a percent (negative) or absolute value (positive).
        This is useful for example if some numeric configuration can optionally be set to a percentage
        of something else.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      2753429c
  13. 13 Sep, 2021 1 commit
    • zhaozhao.zz's avatar
      PSYNC2: make partial sync possible after master reboot (#8015) · 794442b1
      zhaozhao.zz authored
      The main idea is how to allow a master to load replication info from RDB file when rebooting, if master can load replication info it means that replicas may have the chance to psync with master, it can save much traffic.
      
      The key point is we need guarantee safety and consistency, so there
      are two differences between master and replica:
      
      1. master would load the replication info as secondary ID and
         offset, in case other masters have the same replid.
      2. when master loading RDB, it would propagate expired keys as DEL
         command to replication backlog, then replica can receive these
         commands to delete stale keys.
         p.s. the expired keys when RDB loading is useful for users, so
         we show it as `rdb_last_load_keys_expired` and `rdb_last_load_keys_loaded` in info persistence.
      
      Moreover, after load replication info, master should update
      `no_replica_time` in case loading RDB cost too long time.
      794442b1
  14. 09 Sep, 2021 1 commit
    • sundb's avatar
      Replace all usage of ziplist with listpack for t_zset (#9366) · 3ca6972e
      sundb authored
      Part two of implementing #8702 (zset), after #8887.
      
      ## Description of the feature
      Replaced all uses of ziplist with listpack in t_zset, and optimized some of the code to optimize performance.
      
      ## Rdb format changes
      New `RDB_TYPE_ZSET_LISTPACK` rdb type.
      
      ## Rdb loading improvements:
      1) Pre-expansion of dict for validation of duplicate data for listpack and ziplist.
      2) Simplifying the release of empty key objects when RDB loading.
      3) Unify ziplist and listpack data verify methods for zset and hash, and move code to rdb.c.
      
      ## Interface changes
      1) New `zset-max-listpack-entries` config is an alias for `zset-max-ziplist-entries` (same with `zset-max-listpack-value`).
      2) OBJECT ENCODING will return listpack instead of ziplist.
      
      ## Listpack improvements:
      1) Add `lpDeleteRange` and `lpDeleteRangeWithEntry` functions to delete a range of entries from listpack.
      2) Improve the performance of `lpCompare`, converting from string to integer is faster than converting from integer to string.
      3) Replace `snprintf` with `ll2string` to improve performance in converting numbers to strings in `lpGet()`.
      
      ## Zset improvements:
      1) Improve the performance of `zzlFind` method, use `lpFind` instead of `lpCompare` in a loop.
      2) Use `lpDeleteRangeWithEntry` instead of `lpDelete` twice to delete a element of zset.
      
      ## Tests
      1) Add some unittests for `lpDeleteRange` and `lpDeleteRangeWithEntry` function.
      2) Add zset RDB loading test.
      3) Add benchmark test for `lpCompare` and `ziplsitCompare`.
      4) Add empty listpack zset corrupt dump test.
      3ca6972e
  15. 10 Aug, 2021 2 commits
    • yoav-steinberg's avatar
      19bc8371
    • sundb's avatar
      Replace all usage of ziplist with listpack for t_hash (#8887) · 02fd76b9
      sundb authored
      
      
      Part one of implementing #8702 (taking hashes first before other types)
      
      ## Description of the feature
      1. Change ziplist encoded hash objects to listpack encoding.
      2. Convert existing ziplists on RDB loading time. an O(n) operation.
      
      ## Rdb format changes
      1. Add RDB_TYPE_HASH_LISTPACK rdb type.
      2. Bump RDB_VERSION to 10
      
      ## Interface changes
      1. New `hash-max-listpack-entries` config is an alias for `hash-max-ziplist-entries` (same with `hash-max-listpack-value`)
      2. OBJECT ENCODING will return `listpack` instead of `ziplist`
      
      ## Listpack improvements:
      1. Support direct insert, replace integer element (rather than convert back and forth from string)
      3. Add more listpack capabilities to match the ziplist ones (like `lpFind`, `lpRandomPairs` and such)
      4. Optimize element length fetching, avoid multiple calculations
      5. Use inline to avoid function call overhead.
      
      ## Tests
      1. Add a new test to the RDB load time conversion
      2. Adding the listpack unit tests. (based on the one in ziplist.c)
      3. Add a few "corrupt payload: fuzzer findings" tests, and slightly modify existing ones.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      02fd76b9
  16. 04 Aug, 2021 1 commit
    • Wang Yuan's avatar
      Use madvise(MADV_DONTNEED) to release memory to reduce COW (#8974) · d4bca53c
      Wang Yuan authored
      
      
      ## Backgroud
      As we know, after `fork`, one process will copy pages when writing data to these
      pages(CoW), and another process still keep old pages, they totally cost more memory.
      For redis, we suffered that redis consumed much memory when the fork child is serializing
      key/values, even that maybe cause OOM.
      
      But actually we find, in redis fork child process, the child process don't need to keep some
      memory and parent process may write or update that, for example, child process will never
      access the key-value that is serialized but users may update it in parent process.
      So we think it may reduce COW if the child process release memory that it is not needed.
      
      ## Implementation
      For releasing key value in child process, we may think we call `decrRefCount` to free memory,
      but i find the fork child process still use much memory when we don't write any data to redis,
      and it costs much more time that slows down bgsave. Maybe because memory allocator doesn't
      really release memory to OS, and it may modify some inner data for this free operation, especially
      when we free small objects.
      
      Moreover, CoW is based on  pages, so it is a easy way that we only free the memory bulk that is
      not less than kernel page size. madvise(MADV_DONTNEED) can quickly release specified region
      pages to OS bypassing memory allocator, and allocator still consider that this memory still is used
      and don't change its inner data.
      
      There are some buffers we can release in the fork child process:
      - **Serialized key-values**
        the fork child process never access serialized key-values, so we try to free them.
        Because we only can release big bulk memory, and it is time consumed to iterate all
        items/members/fields/entries of complex data type. So we decide to iterate them and
        try to release them only when their average size of item/member/field/entry is more
        than page size of OS.
      - **Replication backlog**
        Because replication backlog is a cycle buffer, it will be changed quickly if redis has heavy
        write traffic, but in fork child process, we don't need to access that.
      - **Client buffers**
        If clients have requests during having the fork child process, clients' buffer also be changed
        frequently. The memory includes client query buffer, output buffer, and client struct used memory.
      
      To get child process peak private dirty memory, we need to count peak memory instead
      of last used memory, because the child process may continue to release memory (since
      COW used to only grow till now, the last was equivalent to the peak).
      Also we're adding a new `current_cow_peak` info variable (to complement the existing
      `current_cow_size`)
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      d4bca53c
  17. 30 Jun, 2021 1 commit
  18. 15 Jun, 2021 1 commit
    • sundb's avatar
      Fix the wrong reisze of querybuf (#9003) · e5d8a5eb
      sundb authored
      The initialize memory of `querybuf` is `PROTO_IOBUF_LEN(1024*16) * 2` (due to sdsMakeRoomFor being greedy), under `jemalloc`, the allocated memory will be 40k.
      This will most likely result in the `querybuf` being resized when call `clientsCronResizeQueryBuffer` unless the client requests it fast enough.
      
      Note that this bug existed even before #7875, since the condition for resizing includes the sds headers (32k+6).
      
      ## Changes
      1. Use non-greedy sdsMakeRoomFor when allocating the initial query buffer (of 16k).
      1. Also use non-greedy allocation when working with BIG_ARG (we won't use that extra space anyway)
      2. in case we did use a greedy allocation, read as much as we can into the buffer we got (including internal frag), to reduce system calls.
      3. introduce a dedicated constant for the shrinking (same value as before)
      3. Add test for querybuf.
      4. improve a maxmemory test by ignoring the effect of replica query buffers (can accumulate many ACKs on slow env)
      5. improve a maxmemory by disabling slowlog (it will cause slight memory growth on slow env).
      e5d8a5eb
  19. 09 Jun, 2021 1 commit
    • Yossi Gottlieb's avatar
      Improve test suite to handle external servers better. (#9033) · 8a86bca5
      Yossi Gottlieb authored
      This commit revives the improves the ability to run the test suite against
      external servers, instead of launching and managing `redis-server` processes as
      part of the test fixture.
      
      This capability existed in the past, using the `--host` and `--port` options.
      However, it was quite limited and mostly useful when running a specific tests.
      Attempting to run larger chunks of the test suite experienced many issues:
      
      * Many tests depend on being able to start and control `redis-server` themselves,
      and there's no clear distinction between external server compatible and other
      tests.
      * Cluster mode is not supported (resulting with `CROSSSLOT` errors).
      
      This PR cleans up many things and makes it possible to run the entire test suite
      against an external server. It also provides more fine grained controls to
      handle cases where the external server supports a subset of the Redis commands,
      limited number of databases, cluster mode, etc.
      
      The tests directory now contains a `README.md` file that describes how this
      works.
      
      This commit also includes additional cleanups and fixes:
      
      * Tests can now be tagged.
      * Tag-based selection is now unified across `start_server`, `tags` and `test`.
      * More information is provided about skipped or ignored tests.
      * Repeated patterns in tests have been extracted to common procedures, both at a
        global level and on a per-test file basis.
      * Cleaned up some cases where test setup was based on a previous test executing
        (a major anti-pattern that repeats itself in many places).
      * Cleaned up some cases where test teardown was not part of a test (in the
        future we should have dedicated teardown code that executes even when tests
        fail).
      * Fixed some tests that were flaky running on external servers.
      8a86bca5
  20. 06 Jun, 2021 1 commit
    • Oran Agra's avatar
      tests: prevent name clash in variables leading to wrong test name (#8995) · 3e39ea0b
      Oran Agra authored
      running the "geo" unit would have shown that it completed a unit named
      "north". this was because the variable `$name` was overwritten.
      This commit isn't perfect, but it slightly reduces the chance for
      variable name clash.
      
      ```
      $ ./runtest --single unit/geo
      .......
      Testing unit/geo
      .......
      [1/1 done]: north (15 seconds)
      ```
      3e39ea0b
  21. 01 Mar, 2021 1 commit
    • YaacovHazan's avatar
      Make port, tls-port and bind configurations modifiable (#8510) · a031d268
      YaacovHazan authored
      Add ability to modify port, tls-port and bind configurations by CONFIG SET command.
      
      To simplify the code and make it cleaner, a new structure
      added, socketFds, which contains the file descriptors array and its counter,
      and used for TCP, TLS and Cluster sockets file descriptors.
      a031d268
  22. 07 Feb, 2021 1 commit
  23. 28 Jan, 2021 1 commit
  24. 25 Jan, 2021 1 commit
    • Oran Agra's avatar
      Fix rare test failures due to repl-ping-replica-period (#8393) · 437e2583
      Oran Agra authored
      some tests use attach_to_replication_stream to watch what's propagated
      to replicas, but in some cases the periodic ping may slip in and fail
      the test.
      we disable that ping by setting the period to once an hour (tests should
      not run for that long).
      
      other change is so that the next time this oom-score-adj test fails,
      we'll see the value (assert_equals prints it)
      437e2583
  25. 31 Dec, 2020 1 commit
    • filipe oliveira's avatar
      Add errorstats info section, Add failed_calls and rejected_calls to commandstats (#8217) · 90b9f08e
      filipe oliveira authored
      This Commit pushes forward the observability on overall error statistics and command statistics within redis-server:
      
      It extends INFO COMMANDSTATS to have
      - failed_calls in - so we can keep track of errors that happen from the command itself, broken by command.
      - rejected_calls - so we can keep track of errors that were triggered outside the commmand processing per se
      
      Adds a new section to INFO, named ERRORSTATS that enables keeping track of the different errors that
      occur within redis ( within processCommand and call ) based on the reply Error Prefix ( The first word
      after the "-", up to the first space ).
      
      This commit also fixes RM_ReplyWithError so that it can be correctly identified as an error reply.
      90b9f08e
  26. 11 Dec, 2020 1 commit
    • Yossi Gottlieb's avatar
      TLS: Add different client cert support. (#8076) · 8c291b97
      Yossi Gottlieb authored
      This adds a new `tls-client-cert-file` and `tls-client-key-file`
      configuration directives which make it possible to use different
      certificates for the TLS-server and TLS-client functions of Redis.
      
      This is an optional directive. If it is not specified the `tls-cert-file`
      and `tls-key-file` directives are used for TLS client functions as well.
      
      Also, `utils/gen-test-certs.sh` now creates additional server-only and client-only certs and will skip intensive operations if target files already exist.
      8c291b97
  27. 06 Dec, 2020 3 commits
    • Oran Agra's avatar
      testsuite: fix fd leak, prevent port clashing when using --baseport · 5b446313
      Oran Agra authored
      when using --baseport to run two tests suite in parallel (different
      folders), we need to also make sure the port used by the testsuite to
      communicate with it's workers is unique. otherwise the attept to find a
      free port connects to the other test suite and messes it.
      
      maybe one day we need to attempt to bind, instead of connect when tring
      to find a free port.
      5b446313
    • Oran Agra's avatar
      Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed · c31055db
      Oran Agra authored
      The test creates keys with various encodings, DUMP them, corrupt the payload
      and RESTORES it.
      It utilizes the recently added use-exit-on-panic config to distinguish between
       asserts and segfaults.
      If the restore succeeds, it runs random commands on the key to attempt to
      trigger a crash.
      
      It runs in two modes, one with deep sanitation enabled and one without.
      In the first one we don't expect any assertions or segfaults, in the second one
      we expect assertions, but no segfaults.
      We also check for leaks and invalid reads using valgrind, and if we find them
      we print the commands that lead to that issue.
      
      Changes in the code (other than the test):
      - Replace a few NPD (null pointer deference) flows and division by zero with an
        assertion, so that it doesn't fail the test. (since we set the server to use
        `exit` rather than `abort` on assertion).
      - Fix quite a lot of flows in rdb.c that could have lead to memory leaks in
        RESTORE command (since it now responds with an error rather than panic)
      - Add a DEBUG flag for SET-SKIP-CHECKSUM-VALIDATION so that the test don't need
        to bother with faking a valid checksum
      - Remove a pile of code in serverLogObjectDebugInfo which is actually unsafe to
        run in the crash report (see comments in the code)
      - fix a missing boundary check in lzf_decompress
      
      test suite infra improvements:
      - be able to run valgrind checks before the process terminates
      - rotate log files when restarting servers
      c31055db
    • Oran Agra's avatar
      Sanitize dump payload: ziplist, listpack, zipmap, intset, stream · ca1c1825
      Oran Agra authored
      When loading an encoded payload we will at least do a shallow validation to
      check that the size that's encoded in the payload matches the size of the
      allocation.
      This let's us later use this encoded size to make sure the various offsets
      inside encoded payload don't reach outside the allocation, if they do, we'll
      assert/panic, but at least we won't segfault or smear memory.
      
      We can also do 'deep' validation which runs on all the records of the encoded
      payload and validates that they don't contain invalid offsets. This lets us
      detect corruptions early and reject a RESTORE command rather than accepting
      it and asserting (crashing) later when accessing that payload via some command.
      
      configuration:
      - adding ACL flag skip-sanitize-payload
      - adding config sanitize-dump-payload [yes/no/clients]
      
      For now, we don't have a good way to ensure MIGRATE in cluster resharding isn't
      being slowed down by these sanitation, so i'm setting the default value to `no`,
      but later on it should be set to `clients` by default.
      
      changes:
      - changing rdbReportError not to `exit` in RESTORE command
      - adding a new stat to be able to later check if cluster MIGRATE isn't being
        slowed down by sanitation.
      ca1c1825
  28. 04 Nov, 2020 1 commit
  29. 26 Oct, 2020 1 commit
  30. 22 Oct, 2020 1 commit
  31. 17 Sep, 2020 1 commit
    • Wang Yuan's avatar
      Remove tmp rdb file in background thread (#7762) · b002d2b4
      Wang Yuan authored
      We're already using bg_unlink in several places to delete the rdb file in the background,
      and avoid paying the cost of the deletion from our main thread.
      This commit uses bg_unlink to remove the temporary rdb file in the background too.
      
      However, in case we delete that rdb file just before exiting, we don't actually wait for the
      background thread or the main thread to delete it, and just let the OS clean up after us.
      i.e. we open the file, unlink it and exit with the fd still open.
      
      Furthermore, rdbRemoveTempFile can be called from a thread and was using snprintf which is
      not async-signal-safe, we now use ll2string instead.
      b002d2b4
  32. 10 Sep, 2020 1 commit
    • Oran Agra's avatar
      Squash merging 125 typo/grammar/comment/doc PRs (#7773) · 1c710385
      Oran Agra authored
      
      
      List of squashed commits or PRs
      ===============================
      
      commit 66801ea
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Jan 13 00:54:31 2020 -0500
      
          typo fix in acl.c
      
      commit 46f55db
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun Sep 6 18:24:11 2020 +0300
      
          Updates a couple of comments
      
          Specifically:
      
          * RM_AutoMemory completed instead of pointing to docs
          * Updated link to custom type doc
      
      commit 61a2aa0
      Author: xindoo <xindoo@qq.com>
      Date:   Tue Sep 1 19:24:59 2020 +0800
      
          Correct errors in code comments
      
      commit a5871d1
      Author: yz1509 <pro-756@qq.com>
      Date:   Tue Sep 1 18:36:06 2020 +0800
      
          fix typos in module.c
      
      commit 41eede7
      Author: bookug <bookug@qq.com>
      Date:   Sat Aug 15 01:11:33 2020 +0800
      
          docs: fix typos in comments
      
      commit c303c84
      Author: lazy-snail <ws.niu@outlook.com>
      Date:   Fri Aug 7 11:15:44 2020 +0800
      
          fix spelling in redis.conf
      
      commit 1eb76bf
      Author: zhujian <zhujianxyz@gmail.com>
      Date:   Thu Aug 6 15:22:10 2020 +0800
      
          add a missing 'n' in comment
      
      commit 1530ec2
      Author: Daniel Dai <764122422@qq.com>
      Date:   Mon Jul 27 00:46:35 2020 -0400
      
          fix spelling in tracking.c
      
      commit e517b31
      Author: Hunter-Chen <huntcool001@gmail.com>
      Date:   Fri Jul 17 22:33:32 2020 +0800
      
          Update redis.conf
      Co-authored-by: default avatarItamar Haber <itamar@redislabs.com>
      
      commit c300eff
      Author: Hunter-Chen <huntcool001@gmail.com>
      Date:   Fri Jul 17 22:33:23 2020 +0800
      
          Update redis.conf
      Co-authored-by: default avatarItamar Haber <itamar@redislabs.com>
      
      commit 4c058a8
      Author: 陈浩鹏 <chenhaopeng@heytea.com>
      Date:   Thu Jun 25 19:00:56 2020 +0800
      
          Grammar fix and clarification
      
      commit 5fcaa81
      Author: bodong.ybd <bodong.ybd@alibaba-inc.com>
      Date:   Fri Jun 19 10:09:00 2020 +0800
      
          Fix typos
      
      commit 4caca9a
      Author: Pruthvi P <pruthvi@ixigo.com>
      Date:   Fri May 22 00:33:22 2020 +0530
      
          Fix typo eviciton => eviction
      
      commit b2a25f6
      Author: Brad Dunbar <dunbarb2@gmail.com>
      Date:   Sun May 17 12:39:59 2020 -0400
      
          Fix a typo.
      
      commit 12842ae
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Sun May 3 17:16:59 2020 -0400
      
          fix spelling in redis conf
      
      commit ddba07c
      Author: Chris Lamb <chris@chris-lamb.co.uk>
      Date:   Sat May 2 23:25:34 2020 +0100
      
          Correct a "conflicts" spelling error.
      
      commit 8fc7bf2
      Author: Nao YONASHIRO <yonashiro@r.recruit.co.jp>
      Date:   Thu Apr 30 10:25:27 2020 +0900
      
          docs: fix EXPIRE_FAST_CYCLE_DURATION to ACTIVE_EXPIRE_CYCLE_FAST_DURATION
      
      commit 9b2b67a
      Author: Brad Dunbar <dunbarb2@gmail.com>
      Date:   Fri Apr 24 11:46:22 2020 -0400
      
          Fix a typo.
      
      commit 0746f10
      Author: devilinrust <63737265+devilinrust@users.noreply.github.com>
      Date:   Thu Apr 16 00:17:53 2020 +0200
      
          Fix typos in server.c
      
      commit 92b588d
      Author: benjessop12 <56115861+benjessop12@users.noreply.github.com>
      Date:   Mon Apr 13 13:43:55 2020 +0100
      
          Fix spelling mistake in lazyfree.c
      
      commit 1da37aa
      Merge: 2d4ba28 af347a8c
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Thu Mar 5 22:41:31 2020 -0500
      
          Merge remote-tracking branch 'upstream/unstable' into expiretypofix
      
      commit 2d4ba28
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Mar 2 00:09:40 2020 -0500
      
          fix typo in expire.c
      
      commit 1a746f7
      Author: SennoYuki <minakami1yuki@gmail.com>
      Date:   Thu Feb 27 16:54:32 2020 +0800
      
          fix typo
      
      commit 8599b1a
      Author: dongheejeong <donghee950403@gmail.com>
      Date:   Sun Feb 16 20:31:43 2020 +0000
      
          Fix typo in server.c
      
      commit f38d4e8
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Sun Feb 2 22:58:38 2020 -0500
      
          fix typo in evict.c
      
      commit fe143fc
      Author: Leo Murillo <leonardo.murillo@gmail.com>
      Date:   Sun Feb 2 01:57:22 2020 -0600
      
          Fix a few typos in redis.conf
      
      commit 1ab4d21
      Author: viraja1 <anchan.viraj@gmail.com>
      Date:   Fri Dec 27 17:15:58 2019 +0530
      
          Fix typo in Latency API docstring
      
      commit ca1f70e
      Author: gosth <danxuedexing@qq.com>
      Date:   Wed Dec 18 15:18:02 2019 +0800
      
          fix typo in sort.c
      
      commit a57c06b
      Author: ZYunH <zyunhjob@163.com>
      Date:   Mon Dec 16 22:28:46 2019 +0800
      
          fix-zset-typo
      
      commit b8c92b5
      Author: git-hulk <hulk.website@gmail.com>
      Date:   Mon Dec 16 15:51:42 2019 +0800
      
          FIX: typo in cluster.c, onformation->information
      
      commit 9dd981c
      Author: wujm2007 <jim.wujm@gmail.com>
      Date:   Mon Dec 16 09:37:52 2019 +0800
      
          Fix typo
      
      commit e132d7a
      Author: Sebastien Williams-Wynn <s.williamswynn.mail@gmail.com>
      Date:   Fri Nov 15 00:14:07 2019 +0000
      
          Minor typo change
      
      commit 47f44d5
      Author: happynote3966 <01ssrmikururudevice01@gmail.com>
      Date:   Mon Nov 11 22:08:48 2019 +0900
      
          fix comment typo in redis-cli.c
      
      commit b8bdb0d
      Author: fulei <fulei@kuaishou.com>
      Date:   Wed Oct 16 18:00:17 2019 +0800
      
          Fix a spelling mistake of comments  in defragDictBucketCallback
      
      commit 0def46a
      Author: fulei <fulei@kuaishou.com>
      Date:   Wed Oct 16 13:09:27 2019 +0800
      
          fix some spelling mistakes of comments in defrag.c
      
      commit f3596fd
      Author: Phil Rajchgot <tophil@outlook.com>
      Date:   Sun Oct 13 02:02:32 2019 -0400
      
          Typo and grammar fixes
      
          Redis and its documentation are great -- just wanted to submit a few corrections in the spirit of Hacktoberfest. Thanks for all your work on this project. I use it all the time and it works beautifully.
      
      commit 2b928cd
      Author: KangZhiDong <worldkzd@gmail.com>
      Date:   Sun Sep 1 07:03:11 2019 +0800
      
          fix typos
      
      commit 33aea14
      Author: Axlgrep <axlgrep@gmail.com>
      Date:   Tue Aug 27 11:02:18 2019 +0800
      
          Fixed eviction spelling issues
      
      commit e282a80
      Author: Simen Flatby <simen@oms.no>
      Date:   Tue Aug 20 15:25:51 2019 +0200
      
          Update comments to reflect prop name
      
          In the comments the prop is referenced as replica-validity-factor,
          but it is really named cluster-replica-validity-factor.
      
      commit 74d1f9a
      Author: Jim Green <jimgreen2013@qq.com>
      Date:   Tue Aug 20 20:00:31 2019 +0800
      
          fix comment error, the code is ok
      
      commit eea1407
      Author: Liao Tonglang <liaotonglang@gmail.com>
      Date:   Fri May 31 10:16:18 2019 +0800
      
          typo fix
      
          fix cna't to can't
      
      commit 0da553c
      Author: KAWACHI Takashi <tkawachi@gmail.com>
      Date:   Wed Jul 17 00:38:16 2019 +0900
      
          Fix typo
      
      commit 7fc8fb6
      Author: Michael Prokop <mika@grml.org>
      Date:   Tue May 28 17:58:42 2019 +0200
      
          Typo fixes
      
          s/familar/familiar/
          s/compatiblity/compatibility/
          s/ ot / to /
          s/itsef/itself/
      
      commit 5f46c9d
      Author: zhumoing <34539422+zhumoing@users.noreply.github.com>
      Date:   Tue May 21 21:16:50 2019 +0800
      
          typo-fixes
      
          typo-fixes
      
      commit 321dfe1
      Author: wxisme <850885154@qq.com>
      Date:   Sat Mar 16 15:10:55 2019 +0800
      
          typo fix
      
      commit b4fb131
      Merge: 267e0e6 3df1eb85
      Author: Nikitas Bastas <nikitasbst@gmail.com>
      Date:   Fri Feb 8 22:55:45 2019 +0200
      
          Merge branch 'unstable' of antirez/redis into unstable
      
      commit 267e0e6
      Author: Nikitas Bastas <nikitasbst@gmail.com>
      Date:   Wed Jan 30 21:26:04 2019 +0200
      
          Minor typo fix
      
      commit 30544e7
      Author: inshal96 <39904558+inshal96@users.noreply.github.com>
      Date:   Fri Jan 4 16:54:50 2019 +0500
      
          remove an extra 'a' in the comments
      
      commit 337969d
      Author: BrotherGao <yangdongheng11@gmail.com>
      Date:   Sat Dec 29 12:37:29 2018 +0800
      
          fix typo in redis.conf
      
      commit 9f4b121
      Merge: 423a030 e504583b
      Author: BrotherGao <yangdongheng@xiaomi.com>
      Date:   Sat Dec 29 11:41:12 2018 +0800
      
          Merge branch 'unstable' of antirez/redis into unstable
      
      commit 423a030
      Merge: 42b02b7 46a51cdc
      Author: 杨东衡 <yangdongheng@xiaomi.com>
      Date:   Tue Dec 4 23:56:11 2018 +0800
      
          Merge branch 'unstable' of antirez/redis into unstable
      
      commit 42b02b7
      Merge: 68c0e6e3 b8febe60
      
      
      Author: Dongheng Yang <yangdongheng11@gmail.com>
      Date:   Sun Oct 28 15:54:23 2018 +0800
      
          Merge pull request #1 from antirez/unstable
      
          update local data
      
      commit 714b589
      Author: Christian <crifei93@gmail.com>
      Date:   Fri Dec 28 01:17:26 2018 +0100
      
          fix typo "resulution"
      
      commit e23259d
      Author: garenchan <1412950785@qq.com>
      Date:   Wed Dec 26 09:58:35 2018 +0800
      
          fix typo: segfauls -> segfault
      
      commit a9359f8
      Author: xjp <jianping_xie@aliyun.com>
      Date:   Tue Dec 18 17:31:44 2018 +0800
      
          Fixed REDISMODULE_H spell bug
      
      commit a12c3e4
      Author: jdiaz <jrd.palacios@gmail.com>
      Date:   Sat Dec 15 23:39:52 2018 -0600
      
          Fixes hyperloglog hash function comment block description
      
      commit 770eb11
      Author: 林上耀 <1210tom@163.com>
      Date:   Sun Nov 25 17:16:10 2018 +0800
      
          fix typo
      
      commit fd97fbb
      Author: Chris Lamb <chris@chris-lamb.co.uk>
      Date:   Fri Nov 23 17:14:01 2018 +0100
      
          Correct "unsupported" typo.
      
      commit a85522d
      Author: Jungnam Lee <jungnam.lee@oracle.com>
      Date:   Thu Nov 8 23:01:29 2018 +0900
      
          fix typo in test comments
      
      commit ade8007
      Author: Arun Kumar <palerdot@users.noreply.github.com>
      Date:   Tue Oct 23 16:56:35 2018 +0530
      
          Fixed grammatical typo
      
          Fixed typo for word 'dictionary'
      
      commit 869ee39
      Author: Hamid Alaei <hamid.a85@gmail.com>
      Date:   Sun Aug 12 16:40:02 2018 +0430
      
          fix documentations: (ThreadSafeContextStart/Stop -> ThreadSafeContextLock/Unlock), minor typo
      
      commit f89d158
      Author: Mayank Jain <mayankjain255@gmail.com>
      Date:   Tue Jul 31 23:01:21 2018 +0530
      
          Updated README.md with some spelling corrections.
      
          Made correction in spelling of some misspelled words.
      
      commit 892198e
      Author: dsomeshwar <someshwar.dhayalan@gmail.com>
      Date:   Sat Jul 21 23:23:04 2018 +0530
      
          typo fix
      
      commit 8a4d780
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Mon Apr 30 02:06:52 2018 +0300
      
          Fixes some typos
      
      commit e3acef6
      Author: Noah Rosamilia <ivoahivoah@gmail.com>
      Date:   Sat Mar 3 23:41:21 2018 -0500
      
          Fix typo in /deps/README.md
      
      commit 04442fb
      Author: WuYunlong <xzsyeb@126.com>
      Date:   Sat Mar 3 10:32:42 2018 +0800
      
          Fix typo in readSyncBulkPayload() comment.
      
      commit 9f36880
      Author: WuYunlong <xzsyeb@126.com>
      Date:   Sat Mar 3 10:20:37 2018 +0800
      
          replication.c comment: run_id -> replid.
      
      commit f866b4a
      Author: Francesco 'makevoid' Canessa <makevoid@gmail.com>
      Date:   Thu Feb 22 22:01:56 2018 +0000
      
          fix comment typo in server.c
      
      commit 0ebc69b
      Author: 줍 <jubee0124@gmail.com>
      Date:   Mon Feb 12 16:38:48 2018 +0900
      
          Fix typo in redis.conf
      
          Fix `five behaviors` to `eight behaviors` in [this sentence ](antirez/redis@unstable/redis.conf#L564)
      
      commit b50a620
      Author: martinbroadhurst <martinbroadhurst@users.noreply.github.com>
      Date:   Thu Dec 28 12:07:30 2017 +0000
      
          Fix typo in valgrind.sup
      
      commit 7d8f349
      Author: Peter Boughton <peter@sorcerersisle.com>
      Date:   Mon Nov 27 19:52:19 2017 +0000
      
          Update CONTRIBUTING; refer doc updates to redis-doc repo.
      
      commit 02dec7e
      Author: Klauswk <klauswk1@hotmail.com>
      Date:   Tue Oct 24 16:18:38 2017 -0200
      
          Fix typo in comment
      
      commit e1efbc8
      Author: chenshi <baiwfg2@gmail.com>
      Date:   Tue Oct 3 18:26:30 2017 +0800
      
          Correct two spelling errors of comments
      
      commit 93327d8
      Author: spacewander <spacewanderlzx@gmail.com>
      Date:   Wed Sep 13 16:47:24 2017 +0800
      
          Update the comment for OBJ_ENCODING_EMBSTR_SIZE_LIMIT's value
      
          The value of OBJ_ENCODING_EMBSTR_SIZE_LIMIT is 44 now instead of 39.
      
      commit 63d361f
      Author: spacewander <spacewanderlzx@gmail.com>
      Date:   Tue Sep 12 15:06:42 2017 +0800
      
          Fix <prevlen> related doc in ziplist.c
      
          According to the definition of ZIP_BIG_PREVLEN and other related code,
          the guard of single byte <prevlen> should be 254 instead of 255.
      
      commit ebe228d
      Author: hanael80 <hanael80@gmail.com>
      Date:   Tue Aug 15 09:09:40 2017 +0900
      
          Fix typo
      
      commit 6b696e6
      Author: Matt Robenolt <matt@ydekproductions.com>
      Date:   Mon Aug 14 14:50:47 2017 -0700
      
          Fix typo in LATENCY DOCTOR output
      
      commit a2ec6ae
      Author: caosiyang <caosiyang@qiyi.com>
      Date:   Tue Aug 15 14:15:16 2017 +0800
      
          Fix a typo: form => from
      
      commit 3ab7699
      Author: caosiyang <caosiyang@qiyi.com>
      Date:   Thu Aug 10 18:40:33 2017 +0800
      
          Fix a typo: replicationFeedSlavesFromMaster() => replicationFeedSlavesFromMasterStream()
      
      commit 72d43ef
      Author: caosiyang <caosiyang@qiyi.com>
      Date:   Tue Aug 8 15:57:25 2017 +0800
      
          fix a typo: servewr => server
      
      commit 707c958
      Author: Bo Cai <charpty@gmail.com>
      Date:   Wed Jul 26 21:49:42 2017 +0800
      
          redis-cli.c typo: conut -> count.
      Signed-off-by: default avatarBo Cai <charpty@gmail.com>
      
      commit b9385b2
      Author: JackDrogon <jack.xsuperman@gmail.com>
      Date:   Fri Jun 30 14:22:31 2017 +0800
      
          Fix some spell problems
      
      commit 20d9230
      Author: akosel <aaronjkosel@gmail.com>
      Date:   Sun Jun 4 19:35:13 2017 -0500
      
          Fix typo
      
      commit b167bfc
      Author: Krzysiek Witkowicz <krzysiekwitkowicz@gmail.com>
      Date:   Mon May 22 21:32:27 2017 +0100
      
          Fix #4008 small typo in comment
      
      commit 2b78ac8
      Author: Jake Clarkson <jacobwclarkson@gmail.com>
      Date:   Wed Apr 26 15:49:50 2017 +0100
      
          Correct typo in tests/unit/hyperloglog.tcl
      
      commit b0f1cdb
      Author: Qi Luo <qiluo-msft@users.noreply.github.com>
      Date:   Wed Apr 19 14:25:18 2017 -0700
      
          Fix typo
      
      commit a90b0f9
      Author: charsyam <charsyam@naver.com>
      Date:   Thu Mar 16 18:19:53 2017 +0900
      
          fix typos
      
          fix typos
      
          fix typos
      
      commit 8430a79
      Author: Richard Hart <richardhart92@gmail.com>
      Date:   Mon Mar 13 22:17:41 2017 -0400
      
          Fixed log message typo in listenToPort.
      
      commit 481a1c2
      Author: Vinod Kumar <kumar003vinod@gmail.com>
      Date:   Sun Jan 15 23:04:51 2017 +0530
      
          src/db.c: Correct "save" -> "safe" typo
      
      commit 586b4d3
      Author: wangshaonan <wshn13@gmail.com>
      Date:   Wed Dec 21 20:28:27 2016 +0800
      
          Fix typo they->the in helloworld.c
      
      commit c1c4b5e
      Author: Jenner <hypxm@qq.com>
      Date:   Mon Dec 19 16:39:46 2016 +0800
      
          typo error
      
      commit 1ee1a3f
      Author: tielei <43289893@qq.com>
      Date:   Mon Jul 18 13:52:25 2016 +0800
      
          fix some comments
      
      commit 11a41fb
      Author: Otto Kekäläinen <otto@seravo.fi>
      Date:   Sun Jul 3 10:23:55 2016 +0100
      
          Fix spelling in documentation and comments
      
      commit 5fb5d82
      Author: francischan <f1ancis621@gmail.com>
      Date:   Tue Jun 28 00:19:33 2016 +0800
      
          Fix outdated comments about redis.c file.
          It should now refer to server.c file.
      
      commit 6b254bc
      Author: lmatt-bit <lmatt123n@gmail.com>
      Date:   Thu Apr 21 21:45:58 2016 +0800
      
          Refine the comment of dictRehashMilliseconds func
      
      SLAVECONF->REPLCONF in comment - by andyli029
      
      commit ee9869f
      Author: clark.kang <charsyam@naver.com>
      Date:   Tue Mar 22 11:09:51 2016 +0900
      
          fix typos
      
      commit f7b3b11
      Author: Harisankar H <harisankarh@gmail.com>
      Date:   Wed Mar 9 11:49:42 2016 +0530
      
          Typo correction: "faield" --> "failed"
      
          Typo correction: "faield" --> "failed"
      
      commit 3fd40fc
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Thu Feb 25 10:31:51 2016 +0200
      
          Fixes a typo in comments
      
      commit 621c160
      Author: Prayag Verma <prayag.verma@gmail.com>
      Date:   Mon Feb 1 12:36:20 2016 +0530
      
          Fix typo in Readme.md
      
          Spelling mistakes -
          `eviciton` > `eviction`
          `familar` > `familiar`
      
      commit d7d07d6
      Author: WonCheol Lee <toctoc21c@gmail.com>
      Date:   Wed Dec 30 15:11:34 2015 +0900
      
          Typo fixed
      
      commit a4dade7
      Author: Felix Bünemann <buenemann@louis.info>
      Date:   Mon Dec 28 11:02:55 2015 +0100
      
          [ci skip] Improve supervised upstart config docs
      
          This mentions that "expect stop" is required for supervised upstart
          to work correctly. See http://upstart.ubuntu.com/cookbook/#expect-stop
      
      
          for an explanation.
      
      commit d9caba9
      Author: daurnimator <quae@daurnimator.com>
      Date:   Mon Dec 21 18:30:03 2015 +1100
      
          README: Remove trailing whitespace
      
      commit 72d42e5
      Author: daurnimator <quae@daurnimator.com>
      Date:   Mon Dec 21 18:29:32 2015 +1100
      
          README: Fix typo. th => the
      
      commit dd6e957
      Author: daurnimator <quae@daurnimator.com>
      Date:   Mon Dec 21 18:29:20 2015 +1100
      
          README: Fix typo. familar => familiar
      
      commit 3a12b23
      Author: daurnimator <quae@daurnimator.com>
      Date:   Mon Dec 21 18:28:54 2015 +1100
      
          README: Fix typo. eviciton => eviction
      
      commit 2d1d03b
      Author: daurnimator <quae@daurnimator.com>
      Date:   Mon Dec 21 18:21:45 2015 +1100
      
          README: Fix typo. sever => server
      
      commit 3973b06
      Author: Itamar Haber <itamar@garantiadata.com>
      Date:   Sat Dec 19 17:01:20 2015 +0200
      
          Typo fix
      
      commit 4f2e460
      Author: Steve Gao <fu@2token.com>
      Date:   Fri Dec 4 10:22:05 2015 +0800
      
          Update README - fix typos
      
      commit b21667c
      Author: binyan <binbin.yan@nokia.com>
      Date:   Wed Dec 2 22:48:37 2015 +0800
      
          delete redundancy color judge in sdscatcolor
      
      commit 88894c7
      Author: binyan <binbin.yan@nokia.com>
      Date:   Wed Dec 2 22:14:42 2015 +0800
      
          the example output shoule be HelloWorld
      
      commit 2763470
      Author: binyan <binbin.yan@nokia.com>
      Date:   Wed Dec 2 17:41:39 2015 +0800
      
          modify error word keyevente
      Signed-off-by: default avatarbinyan <binbin.yan@nokia.com>
      
      commit 0847b3d
      Author: Bruno Martins <bscmartins@gmail.com>
      Date:   Wed Nov 4 11:37:01 2015 +0000
      
          typo
      
      commit bbb9e9e
      Author: dawedawe <dawedawe@gmx.de>
      Date:   Fri Mar 27 00:46:41 2015 +0100
      
          typo: zimap -> zipmap
      
      commit 5ed297e
      Author: Axel Advento <badwolf.bloodseeker.rev@gmail.com>
      Date:   Tue Mar 3 15:58:29 2015 +0800
      
          Fix 'salve' typos to 'slave'
      
      commit edec9d6
      Author: LudwikJaniuk <ludvig.janiuk@gmail.com>
      Date:   Wed Jun 12 14:12:47 2019 +0200
      
          Update README.md
      Co-Authored-By: default avatarQix <Qix-@users.noreply.github.com>
      
      commit 692a7af
      Author: LudwikJaniuk <ludvig.janiuk@gmail.com>
      Date:   Tue May 28 14:32:04 2019 +0200
      
          grammar
      
      commit d962b0a
      Author: Nick Frost <nickfrostatx@gmail.com>
      Date:   Wed Jul 20 15:17:12 2016 -0700
      
          Minor grammar fix
      
      commit 24fff01aaccaf5956973ada8c50ceb1462e211c6 (typos)
      Author: Chad Miller <chadm@squareup.com>
      Date:   Tue Sep 8 13:46:11 2020 -0400
      
          Fix faulty comment about operation of unlink()
      
      commit 3cd5c1f3326c52aa552ada7ec797c6bb16452355
      Author: Kevin <kevin.xgr@gmail.com>
      Date:   Wed Nov 20 00:13:50 2019 +0800
      
          Fix typo in server.c.
      
      From a83af59 Mon Sep 17 00:00:00 2001
      From: wuwo <wuwo@wacai.com>
      Date: Fri, 17 Mar 2017 20:37:45 +0800
      Subject: [PATCH] falure to failure
      
      From c961896 Mon Sep 17 00:00:00 2001
      From: =?UTF-8?q?=E5=B7=A6=E6=87=B6?= <veficos@gmail.com>
      Date: Sat, 27 May 2017 15:33:04 +0800
      Subject: [PATCH] fix typo
      
      From e600ef2 Mon Sep 17 00:00:00 2001
      From: "rui.zou" <rui.zou@yunify.com>
      Date: Sat, 30 Sep 2017 12:38:15 +0800
      Subject: [PATCH] fix a typo
      
      From c7d07fa Mon Sep 17 00:00:00 2001
      From: Alexandre Perrin <alex@kaworu.ch>
      Date: Thu, 16 Aug 2018 10:35:31 +0200
      Subject: [PATCH] deps README.md typo
      
      From b25cb67 Mon Sep 17 00:00:00 2001
      From: Guy Korland <gkorland@gmail.com>
      Date: Wed, 26 Sep 2018 10:55:37 +0300
      Subject: [PATCH 1/2] fix typos in header
      
      From ad28ca6 Mon Sep 17 00:00:00 2001
      From: Guy Korland <gkorland@gmail.com>
      Date: Wed, 26 Sep 2018 11:02:36 +0300
      Subject: [PATCH 2/2] fix typos
      
      commit 34924cdedd8552466fc22c1168d49236cb7ee915
      Author: Adrian Lynch <adi_ady_ade@hotmail.com>
      Date:   Sat Apr 4 21:59:15 2015 +0100
      
          Typos fixed
      
      commit fd2a1e7
      Author: Jan <jsteemann@users.noreply.github.com>
      Date:   Sat Oct 27 19:13:01 2018 +0200
      
          Fix typos
      
          Fix typos
      
      commit e14e47c1a234b53b0e103c5f6a1c61481cbcbb02
      Author: Andy Lester <andy@petdance.com>
      Date:   Fri Aug 2 22:30:07 2019 -0500
      
          Fix multiple misspellings of "following"
      
      commit 79b948ce2dac6b453fe80995abbcaac04c213d5a
      Author: Andy Lester <andy@petdance.com>
      Date:   Fri Aug 2 22:24:28 2019 -0500
      
          Fix misspelling of create-cluster
      
      commit 1fffde52666dc99ab35efbd31071a4c008cb5a71
      Author: Andy Lester <andy@petdance.com>
      Date:   Wed Jul 31 17:57:56 2019 -0500
      
          Fix typos
      
      commit 204c9ba9651e9e05fd73936b452b9a30be456cfe
      Author: Xiaobo Zhu <xiaobo.zhu@shopee.com>
      Date:   Tue Aug 13 22:19:25 2019 +0800
      
          fix typos
      
      Squashed commit of the following:
      
      commit 1d9aaf8
      Author: danmedani <danmedani@gmail.com>
      Date:   Sun Aug 2 11:40:26 2015 -0700
      
      README typo fix.
      
      Squashed commit of the following:
      
      commit 32bfa7c
      Author: Erik Dubbelboer <erik@dubbelboer.com>
      Date:   Mon Jul 6 21:15:08 2015 +0200
      
      Fixed grammer
      
      Squashed commit of the following:
      
      commit b24f69c
      Author: Sisir Koppaka <sisir.koppaka@gmail.com>
      Date:   Mon Mar 2 22:38:45 2015 -0500
      
      utils/hashtable/rehashing.c: Fix typos
      
      Squashed commit of the following:
      
      commit 4e04082
      Author: Erik Dubbelboer <erik@dubbelboer.com>
      Date:   Mon Mar 23 08:22:21 2015 +0000
      
      Small config file documentation improvements
      
      Squashed commit of the following:
      
      commit acb8773
      Author: ctd1500 <ctd1500@gmail.com>
      Date:   Fri May 8 01:52:48 2015 -0700
      
      Typo and grammar fixes in readme
      
      commit 2eb75b6
      Author: ctd1500 <ctd1500@gmail.com>
      Date:   Fri May 8 01:36:18 2015 -0700
      
      fixed redis.conf comment
      
      Squashed commit of the following:
      
      commit a8249a2
      Author: Masahiko Sawada <sawada.mshk@gmail.com>
      Date:   Fri Dec 11 11:39:52 2015 +0530
      
      Revise correction of typos.
      
      Squashed commit of the following:
      
      commit 3c02028
      Author: zhaojun11 <zhaojun11@jd.com>
      Date:   Wed Jan 17 19:05:28 2018 +0800
      
      Fix typos include two code typos in cluster.c and latency.c
      
      Squashed commit of the following:
      
      commit 9dba47c
      Author: q191201771 <191201771@qq.com>
      Date:   Sat Jan 4 11:31:04 2020 +0800
      
      fix function listCreate comment in adlist.c
      
      Update src/server.c
      
      commit 2c7c2cb536e78dd211b1ac6f7bda00f0f54faaeb
      Author: charpty <charpty@gmail.com>
      Date:   Tue May 1 23:16:59 2018 +0800
      
          server.c typo: modules system dictionary type comment
      Signed-off-by: default avatarcharpty <charpty@gmail.com>
      
      commit a8395323fb63cb59cb3591cb0f0c8edb7c29a680
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun May 6 00:25:18 2018 +0300
      
          Updates test_helper.tcl's help with undocumented options
      
          Specifically:
      
          * Host
          * Port
          * Client
      
      commit bde6f9ced15755cd6407b4af7d601b030f36d60b
      Author: wxisme <850885154@qq.com>
      Date:   Wed Aug 8 15:19:19 2018 +0800
      
          fix comments in deps files
      
      commit 3172474ba991532ab799ee1873439f3402412331
      Author: wxisme <850885154@qq.com>
      Date:   Wed Aug 8 14:33:49 2018 +0800
      
          fix some comments
      
      commit 01b6f2b6858b5cf2ce4ad5092d2c746e755f53f0
      Author: Thor Juhasz <thor@juhasz.pro>
      Date:   Sun Nov 18 14:37:41 2018 +0100
      
          Minor fixes to comments
      
          Found some parts a little unclear on a first read, which prompted me to have a better look at the file and fix some minor things I noticed.
          Fixing minor typos and grammar. There are no changes to configuration options.
          These changes are only meant to help the user better understand the explanations to the various configuration options
      1c710385
  33. 06 Sep, 2020 2 commits
    • Oran Agra's avatar
      test infra - add durable mode to work around test suite crashing · fe5da2e6
      Oran Agra authored
      in some cases a command that returns an error possibly due to a timing
      issue causes the tcl code to crash and thus prevents the rest of the
      tests from running. this adds an option to make the test proceed despite
      the crash.
      maybe it should be the default mode some day.
      fe5da2e6
    • Oran Agra's avatar
      test infra - improve test skipping ability · 677d14c2
      Oran Agra authored
      - skip full units
      - skip a single test (not just a list of tests)
      - when skipping tag, skip spinning up servers, not just the tests
      - skip tags when running against an external server too
      - allow using multiple tags (split them)
      677d14c2
  34. 12 Aug, 2020 1 commit
    • Yossi Gottlieb's avatar
      Add oom-score-adj configuration option to control Linux OOM killer. (#1690) · 2530dc0e
      Yossi Gottlieb authored
      Add Linux kernel OOM killer control option.
      
      This adds the ability to control the Linux OOM killer oom_score_adj
      parameter for all Redis processes, depending on the process role (i.e.
      master, replica, background child).
      
      A oom-score-adj global boolean flag control this feature. In addition,
      specific values can be configured using oom-score-adj-values if
      additional tuning is required.
      2530dc0e
  35. 13 Jul, 2020 1 commit
    • Oran Agra's avatar
      runtest --stop pause stops before terminating the redis server (#7513) · 02ef355f
      Oran Agra authored
      in the majority of the cases (on this rarely used feature) we want to
      stop and be able to connect to the shard with redis-cli.
      since these are two different processes interracting with the tty we
      need to stop both, and we'll have to hit enter twice, but it's not that
      bad considering it is rarely used.
      02ef355f