1. 05 Jul, 2021 4 commits
    • Binbin's avatar
      hrandfield and zrandmember with count should return emptyarray when key does not exist. (#9178) · a418a2d3
      Binbin authored
      
      
      due to a copy-paste bug, it used to reply with null response rather than empty array.
      this commit includes new tests that are looking at the RESP response directly in
      order to be able to tell the difference between them.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      a418a2d3
    • Oran Agra's avatar
      Use accept4 on linux instead of fcntl to make a client socket non-blocking (#9177) · 5a3de819
      Oran Agra authored
      
      
      This reduces system calls on linux when a new connection is made / accepted.
      
      Changes:
      * Add the SOCK_CLOEXEC option to the accept4() call
        This  ensure that a fork/exec call does not leak a file descriptor.
      * Move anetCloexec and connNonBlock info anetGenericAccept
      * Moving connNonBlock from accept handlers to anetGenericAccept
      
      Moving connNonBlock from createClient, is safe because createClient is
      used in the following ways:
      1. without a connection (fake client)
      2. on an accepted connection (see above)
      3. creating the master client by using connConnect (see below)
      
      The third case, can either use anetTcpNonBlockConnect, or connTLSConnect
      which is by default non-blocking.
      Co-authored-by: default avatarRajiv Kurian <geetasen@gmail.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarYoav Steinberg <yoav@redislabs.com>
      5a3de819
    • Oran Agra's avatar
      Query buffer shrinking improvements (#5013) · ec582cc7
      Oran Agra authored
      
      
      when tracking the peak, don't reset the peak to 0, reset it to the
      maximum of the current used, and the planned to be used by the current
      arg.
      
      when shrining, split the two separate conditions.
      the idle time shrinking will remove all free space.
      but the peak based shrinking will keep room for the current arg.
      
      when we resize due to a peak (rahter than idle time), don't trim all
      unused space, let the qbuf keep a size that's sufficient for the
      currently process bulklen, and the current peak.
      Co-authored-by: default avatarsundb <sundbcn@gmail.com>
      Co-authored-by: default avataryoav-steinberg <yoav@monfort.co.il>
      ec582cc7
    • zhaozhao.zz's avatar
      resize query buffer more accurately · 2248eaac
      zhaozhao.zz authored
      1. querybuf_peak has not been updated correctly in readQueryFromClient.
      2. qbuf shrinking uses sdsalloc instead of sdsAllocSize
      
      see more details in issue #4983
      2248eaac
  2. 03 Jul, 2021 1 commit
  3. 01 Jul, 2021 3 commits
    • Yossi Gottlieb's avatar
      Fix CLIENT UNBLOCK crashing modules. (#9167) · aa139e2f
      Yossi Gottlieb authored
      Modules that use background threads with thread safe contexts are likely
      to use RM_BlockClient() without a timeout function, because they do not
      set up a timeout.
      
      Before this commit, `CLIENT UNBLOCK` would result with a crash as the
      `NULL` timeout callback is called. Beyond just crashing, this is also
      logically wrong as it may throw the module into an unexpected client
      state.
      
      This commits makes `CLIENT UNBLOCK` on such clients behave the same as
      any other client that is not in a blocked state and therefore cannot be
      unblocked.
      aa139e2f
    • Oran Agra's avatar
      Fix bug in sdscatfmt when % is the last format char (#9173) · de9bae21
      Oran Agra authored
      
      
      For the sdscatfmt function in sds.c, when the parameter fmt ended up with '%',
      the behavior is undefined. This commit fix this bug.
      Co-authored-by: default avatarstafuc <stafuc@gmail.com>
      de9bae21
    • Wang Yuan's avatar
      Don't start in sentinel mode if only the folder name contains redis-sentinel (#9176) · 16e04ed9
      Wang Yuan authored
      Before this commit, redis-server starts in sentinel mode if the first startup
      argument has the string redis-sentinel, so redis also starts in sentinel mode
      if the directory it was started from contains the string redis-sentinel.
      Now we check the executable name instead of directory.
      
      Some examples:
      1. Execute ./redis-sentinel/redis/src/redis-sentinel, starts in sentinel mode.
      2. Execute ./redis-sentinel/redis/src/redis-server, starts in server mode,
         but before, redis will start in sentinel mode.
      3. Execute ./redis-sentinel/redis/src/redis-server --sentinel, of course, like
         before, starts in sentinel mode.
      16e04ed9
  4. 30 Jun, 2021 6 commits
  5. 29 Jun, 2021 4 commits
  6. 28 Jun, 2021 1 commit
    • Wang Yuan's avatar
      Remove unnecessary replication backlog memory copy (#9157) · 4fa3e230
      Wang Yuan authored
      in the past, the reply list was a list of sds objects, so this didn't have any overhead,
      but now addReplySds just copies the data from the sds and frees it, so there's no
      need to make a copy of the buffer before copying again.
      this reduces an excessive allocation and free and a memcpy.
      4fa3e230
  7. 27 Jun, 2021 1 commit
  8. 24 Jun, 2021 3 commits
    • Yossi Gottlieb's avatar
      Add bind-source-addr configuration argument. (#9142) · f233c4c5
      Yossi Gottlieb authored
      In the past, the first bind address that was explicitly specified was
      also used to bind outgoing connections. This could result with some
      problems. For example: on some systems using `bind 127.0.0.1` would
      result with outgoing connections also binding to `127.0.0.1` and failing
      to connect to remote addresses.
      
      With the recent change to the way `bind` is handled, this presented
      other issues:
      
      * The default first bind address is '*' which is not a valid address.
      * We make no distinction between user-supplied config that is identical
      to the default, and the default config.
      
      This commit addresses both these issues by introducing an explicit
      configuration parameter to control the bind address on outgoing
      connections.
      f233c4c5
    • Huang Zhw's avatar
      Clean redis-benchmark Throughput output. (#9139) · d1a21e02
      Huang Zhw authored
      some leftovers from print are visible when the new line is printed.
      d1a21e02
    • ZhaolongLi's avatar
      change streamAppendItem to use raxEOF instead of raxNext (#9138) · 89ae3537
      ZhaolongLi authored
      The call to raxNext didn't really progress in the rax, since we were already on the last item.
      instead, all it does is check that it is indeed a valid item, so the new code clearer.
      89ae3537
  9. 22 Jun, 2021 6 commits
    • Oran Agra's avatar
      Adjustments to recent RM_StringTruncate fix (#3718) (#9125) · ae418eca
      Oran Agra authored
      - Introduce a new sdssubstr api as a building block for sdsrange.
        The API of sdsrange is many times hard to work with and also has
        corner case that cause bugs. sdsrange is easy to work with and also
        simplifies the implementation of sdsrange.
      - Revert the fix to RM_StringTruncate and just use sdssubstr instead of
        sdsrange.
      - Solve valgrind warnings from the new tests introduced by the previous
        PR.
      ae418eca
    • Yossi Gottlieb's avatar
      Improve bind and protected-mode config handling. (#9034) · 07b0d144
      Yossi Gottlieb authored
      * Specifying an empty `bind ""` configuration prevents Redis from listening on any TCP port. Before this commit, such configuration was not accepted.
      * Using `CONFIG GET bind` will always return an explicit configuration value. Before this commit, if a bind address was not specified the returned value was empty (which was an anomaly).
      
      Another behavior change is that modifying the `bind` configuration to a non-default value will NO LONGER DISABLE protected-mode implicitly.
      07b0d144
    • Evan's avatar
      modules: Add newlen == 0 handling to RM_StringTruncate (#3717) (#3718) · 1ccf2ca2
      Evan authored
      Previously, passing 0 for newlen would not truncate the string at all.
      This adds handling of this case, freeing the old string and creating a new empty string.
      
      Other changes:
      - Move `src/modules/testmodule.c` to `tests/modules/basics.c`
      - Introduce that basic test into the test suite
      - Add tests to cover StringTruncate
      - Add `test-modules` build target for the main makefile
      - Extend `distclean` build target to clean modules too
      1ccf2ca2
    • Binbin's avatar
      Remove extra semicolon (#9117) · bf92000e
      Binbin authored
      Remove extra semicolon.
      bf92000e
    • Wen Hui's avatar
      add tilt mode since in sentinel info (#9000) · 81d5f05b
      Wen Hui authored
      Sentinel shows tilt mode boolean in info, now it'll show an indication of how long ago it started.
      81d5f05b
    • Oran Agra's avatar
      Fix race in client side tracking (#9116) · 9b564b52
      Oran Agra authored
      The `Tracking gets notification of expired keys` test in tracking.tcl
      used to hung in valgrind CI quite a lot.
      
      It turns out the reason is that with valgrind and a busy machine, the
      server cron active expire cycle could easily run in the same event loop
      as the command that created `mykey`, so that when they key got expired,
      there were two change events to broadcast, one that set the key and one
      that expired it, but since we used raxTryInsert, the client that was
      associated with the "last" change was the one that created the key, so
      the NOLOOP filtered that event.
      
      This commit adds a test that reproduces the problem by using lazy expire
      in a multi-exec which makes sure the key expires in the same event loop
      as the one that added it.
      9b564b52
  10. 21 Jun, 2021 2 commits
  11. 20 Jun, 2021 4 commits
  12. 17 Jun, 2021 3 commits
    • gourav's avatar
    • sundb's avatar
      Make readQueryFromClient more aggressive when reading big arg again (Followup for #9003) (#9100) · 1a22445d
      sundb authored
      Due to the change in #9003, a long-standing bug was raised under `valgrind`.
      This bug can cause the master-slave sync to take a very long time, causing the `pendingquerybuf.tcl` test to fail.
      This problem does not only occur in master-slave sync, it is triggered when the big arg is greater than 32k.
      step:
      ```sh
      dd if=/dev/zero of=bigfile bs=1M count=32
      ./src/redis-cli -x hset a a < bigfile
      ```
      
      1) Make room for querybuf in processMultibulkBuffer, now the alloc of querybuf will be more than 32k.
      2) If this happens to trigger the `clientsCronResizeQueryBuffer`, querybuf will be resized to 0.
      3) Finally, in readQueryFromClient, we expand the querybuf non-greedily, from 0 to 32k.
          Old code, make room for querybuf is greedy, so it only needs 11 times to expand to 32M(16k*(2^11)),
          but now we need 2048(32*1024/16) times to reach it, due to the slow allocation under valgrind that exposed the problem.
      
      The fix for the excessive shrinking of the query buf to 0, will be handled in #5013 (that other change on it's own can fix failing test too), but the fix in this PR will also fix the failing test.
      
      The fix in this PR will makes the reading in `readQueryFromClient` more aggressive when working on a big arg (so that it is in par with the same code in `processMultibulkBuffer` (i.e. the two calls to `sdsMakeRoomForNonGreedy` should both use the bulk size).
      In the code before this fix the one in readQueryFromClient always has `readlen = PROTO_IOBUF_LEN`
      1a22445d
    • DarrenJiang13's avatar
      Improve objectComputeSize() with allocator fragmentaiton. (#9095) · ada60d80
      DarrenJiang13 authored
      This commit improve MEMORY USAGE command to include internal fragmentation overheads of:
      1. EMBSTR encoded strings
      2. ziplist encoded zsets and hashes
      3. List type nodes
      ada60d80
  13. 16 Jun, 2021 2 commits
    • Sam Bortman's avatar
      Support glob pattern matching for config include files (#8980) · c2b93ff8
      Sam Bortman authored
      This will allow distros to use an "include conf.d/*.conf" statement in the default configuration file
      which will facilitate customization across upgrades/downgrades.
      
      The change itself is trivial: instead of opening an individual file, the glob call creates a vector of files to open, and each file is opened in turn, and its content is added to the configuration.
      c2b93ff8
    • yoav-steinberg's avatar
      Remove gopher protocol support. (#9057) · 362786c5
      yoav-steinberg authored
      Gopher support was added mainly because it was simple (trivial to add).
      But apparently even something that was trivial at the time, does cause complications
      down the line when adding more features.
      We recently ran into a few issues with io-threads conflicting with the gopher support.
      We had to either complicate the code further in order to solve them, or drop gopher.
      AFAIK it's completely unused, so we wanna chuck it, rather than keep supporting it.
      362786c5