1. 05 Jul, 2023 1 commit
  2. 03 May, 2023 1 commit
    • Madelyn Olson's avatar
      Remove prototypes with empty declarations (#12020) · 5e3be1be
      Madelyn Olson authored
      Technically declaring a prototype with an empty declaration has been deprecated since the early days of C, but we never got a warning for it. C2x will apparently be introducing a breaking change if you are using this type of declarator, so Clang 15 has started issuing a warning with -pedantic. Although not apparently a problem for any of the compiler we build on, if feels like the right thing is to properly adhere to the C standard and use (void).
      5e3be1be
  3. 02 May, 2023 1 commit
    • Hassaan Khan's avatar
      [redis-benchmark] Adding --seed option to seed the RNG (#11945) · 8163e816
      Hassaan Khan authored
      Adds ability to set the random seed so that more consistent repeatable benchmarks are possible.
      
      Example usage:
      
      Adding 2 hash items
      ```
      src/redis-benchmark -r 100 -n 2 --seed 250 hset myhash:__rand_int__ age __rand_int__
      ```
      
      Monitor:
      
      1st benchmark invocation:
      ```
      1679332814.824357 [0 127.0.0.1:36686] "hset" "myhash:000000000022" "age" "000000000069"
      1679332814.824404 [0 127.0.0.1:36690] "hset" "myhash:000000000007" "age" "000000000043"
      ```
      
      2nd benchmark invocation:
      ```
      1679332814.824357 [0 127.0.0.1:36686] "hset" "myhash:000000000022" "age" "000000000069"
      1679332814.824404 [0 127.0.0.1:36690] "hset" "myhash:000000000007" "age" "000000000043"
      ```
      8163e816
  4. 18 Apr, 2023 1 commit
    • sundb's avatar
      Fix some compile warnings and errors when building with gcc-12 or clang (#12035) · 42c8c618
      sundb authored
      This PR is to fix the compilation warnings and errors generated by the latest
      complier toolchain, and to add a new runner of the latest toolchain for daily CI.
      
      ## Fix various compilation warnings and errors
      
      1) jemalloc.c
      
      COMPILER: clang-14 with FORTIFY_SOURCE
      
      WARNING:
      ```
      src/jemalloc.c:1028:7: warning: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Wstring-concatenation]
                          "/etc/malloc.conf",
                          ^
      src/jemalloc.c:1027:3: note: place parentheses around the string literal to silence warning
                      "\"name\" of the file referenced by the symbolic link named "
                      ^
      ```
      
      REASON:  the compiler to alert developers to potential issues with string concatenation
      that may miss a comma,
      just like #9534 which misses a comma.
      
      SOLUTION: use `()` to tell the compiler that these two line strings are continuous.
      
      2) config.h
      
      COMPILER: clang-14 with FORTIFY_SOURCE
      
      WARNING:
      ```
      In file included from quicklist.c:36:
      ./config.h:319:76: warning: attribute declaration must precede definition [-Wignored-attributes]
      char *strcat(char *restrict dest, const char *restrict src) __attribute__((deprecated("please avoid use of unsafe C functions. prefer use of redis_strlcat instead")));
      ```
      
      REASON: Enabling _FORTIFY_SOURCE will cause the compiler to use `strcpy()` with check,
      it results in a deprecated attribute declaration after including <features.h>.
      
      SOLUTION: move the deprecated attribute declaration from config.h to fmacro.h before "#include <features.h>".
      
      3) networking.c
      
      COMPILER: GCC-12
      
      WARNING: 
      ```
      networking.c: In function ‘addReplyDouble.part.0’:
      networking.c:876:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
        876 |         dbuf[start] = '$';
            |                     ^
      networking.c:868:14: note: at offset -5 into destination object ‘dbuf’ of size 5152
        868 |         char dbuf[MAX_LONG_DOUBLE_CHARS+32];
            |              ^
      networking.c:876:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
        876 |         dbuf[start] = '$';
            |                     ^
      networking.c:868:14: note: at offset -6 into destination object ‘dbuf’ of size 5152
        868 |         char dbuf[MAX_LONG_DOUBLE_CHARS+32];
      ```
      
      REASON: GCC-12 predicts that digits10() may return 9 or 10 through `return 9 + (v >= 1000000000UL)`.
      
      SOLUTION: add an assert to let the compiler know the possible length;
      
      4) redis-cli.c & redis-benchmark.c
      
      COMPILER: clang-14 with FORTIFY_SOURCE
      
      WARNING:
      ```
      redis-benchmark.c:1621:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive] #ifdef USE_OPENSSL
      redis-cli.c:3015:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive] #ifdef USE_OPENSSL
      ```
      
      REASON: when _FORTIFY_SOURCE is enabled, the compiler will use the print() with
      check, which is a macro. this may result in the use of directives within the macro, which
      is undefined behavior.
      
      SOLUTION: move the directives-related code out of `print()`.
      
      5) server.c
      
      COMPILER: gcc-13 with FORTIFY_SOURCE
      
      WARNING:
      ```
      In function 'lookupCommandLogic',
          inlined from 'lookupCommandBySdsLogic' at server.c:3139:32:
      server.c:3102:66: error: '*(robj **)argv' may be used uninitialized [-Werror=maybe-uninitialized]
       3102 |     struct redisCommand *base_cmd = dictFetchValue(commands, argv[0]->ptr);
            |                                                              ~~~~^~~
      ```
      
      REASON: The compiler thinks that the `argc` returned by `sdssplitlen()` could be 0,
      resulting in an empty array of size 0 being passed to lookupCommandLogic.
      this should be a false positive, `argc` can't be 0 when strings are not NULL.
      
      SOLUTION: add an assert to let the compiler know that `argc` is positive.
      
      6) sha1.c
      
      COMPILER: gcc-12
      
      WARNING:
      ```
      In function ‘SHA1Update’,
          inlined from ‘SHA1Final’ at sha1.c:195:5:
      sha1.c:152:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread]
        152 |             SHA1Transform(context->state, &data[i]);
            |             ^
      sha1.c:152:13: note: referencing argument 2 of type ‘const unsigned char[64]’
      sha1.c: In function ‘SHA1Final’:
      sha1.c:56:6: note: in a call to function ‘SHA1Transform’
         56 | void SHA1Transform(uint32_t state[5], const unsigned char buffer[64])
            |      ^
      In function ‘SHA1Update’,
          inlined from ‘SHA1Final’ at sha1.c:198:9:
      sha1.c:152:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread]
        152 |             SHA1Transform(context->state, &data[i]);
            |             ^
      sha1.c:152:13: note: referencing argument 2 of type ‘const unsigned char[64]’
      sha1.c: In function ‘SHA1Final’:
      sha1.c:56:6: note: in a call to function ‘SHA1Transform’
         56 | void SHA1Transform(uint32_t state[5], const unsigned char buffer[64])
      ```
      
      REASON: due to the bug[https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80922], when
      enable LTO, gcc-12 will not see `diagnostic ignored "-Wstringop-overread"`, resulting in a warning.
      
      SOLUTION: temporarily set SHA1Update to noinline to avoid compiler warnings due
      to LTO being enabled until the above gcc bug is fixed.
      
      7) zmalloc.h
      
      COMPILER: GCC-12
      
      WARNING: 
      ```
      In function ‘memset’,
          inlined from ‘moduleCreateContext’ at module.c:877:5,
          inlined from ‘RM_GetDetachedThreadSafeContext’ at module.c:8410:5:
      /usr/include/x86_64-linux-gnu/bits/string_fortified.h:59:10: warning: ‘__builtin_memset’ writing 104 bytes into a region of size 0 overflows the destination [-Wstringop-overflow=]
         59 |   return __builtin___memset_chk (__dest, __ch, __len,
      ```
      
      REASON: due to the GCC-12 bug [https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503],
      GCC-12 cannot see alloc_size, which causes GCC to think that the actual size of memory
      is 0 when checking with __glibc_objsize0().
      
      SOLUTION: temporarily set malloc-related interfaces to `noinline` to avoid compiler warnings
      due to LTO being enabled until the above gcc bug is fixed.
      
      ## Other changes
      1) Fixed `ps -p [pid]`  doesn't output `<defunct>` when using procps 4.x causing `replication
        child dies when parent is killed - diskless` test to fail.
      2) Add a new fortify CI with GCC-13 and ubuntu-lunar docker image.
      42c8c618
  5. 02 Feb, 2023 1 commit
  6. 28 Nov, 2022 1 commit
  7. 09 Oct, 2022 1 commit
  8. 11 Aug, 2022 1 commit
    • Ozan Tezcan's avatar
      Fix overflow in redis-benchmark (#11102) · 99ebbee2
      Ozan Tezcan authored
      Fix overflow in redis-benchmark affecting latency measurements on 32bit builds.
      
      If `long` is 4 bytes (typical on 32 bit systems), multiplication overflows.
      Using `long long` will fix the issue as it is guaranteed to be at least 8 bytes. 
      
      Also, I've added a change to reuse `ustime()` for `mstime()`. 
      99ebbee2
  9. 13 Jul, 2022 1 commit
  10. 22 Jun, 2022 1 commit
    • judeng's avatar
      fix redis-benchmark's bug: check if clients are created successfully in idle mode (#10891) · 49876158
      judeng authored
      my maxclients config:
      ```
      redis-cli config get maxclients
      1) "maxclients"
      2) "4064"
      ```
      
      Before this bug was fixed, creating 4065 clients appeared to be successful, but only 4064 were actually created```
      ```
      ./redis-benchmark -c 4065 -I
      Creating 4065 idle connections and waiting forever (Ctrl+C when done)
      cients: 4065
      ```
      
      now :
      ```
      ./redis-benchmark -c 4065 -I
      Creating 4065 idle connections and waiting forever (Ctrl+C when done)
      Error from server: ERR max number of clients reached
      
      ./redis-benchmark -c 4064 -I
      Creating 4064 idle connections and waiting forever (Ctrl+C when done)
      clients: 4064
      
      ```
      49876158
  11. 08 Mar, 2022 1 commit
  12. 27 Feb, 2022 1 commit
  13. 07 Feb, 2022 1 commit
    • ivanstosic-janea's avatar
      Fix protocol error caused by redis-benchmark (#10236) · bb875603
      ivanstosic-janea authored
      The protocol error was caused by the buggy `writeHandler` in `redis-benchmark.c`,
      which didn't handle one of the cases, thereby repeating data, leading to protocol errors
      when the values being sent are very long.
      
      This PR fixes #10233, issue introduced by #7959
      bb875603
  14. 03 Oct, 2021 1 commit
    • Binbin's avatar
      Cleanup typos, incorrect comments, and fixed small memory leak in redis-cli (#9153) · dd3ac97f
      Binbin authored
      1. Remove forward declarations from header files to functions that do not exist:
      hmsetCommand and rdbSaveTime.
      2. Minor phrasing fixes in #9519
      3. Add missing sdsfree(title) and fix typo in redis-benchmark.
      4. Modify some error comments in some zset commands.
      5. Fix copy-paste bug comment in syncWithMaster about `ip-address`.
      dd3ac97f
  15. 24 Sep, 2021 1 commit
  16. 20 Sep, 2021 1 commit
  17. 14 Sep, 2021 1 commit
  18. 29 Aug, 2021 1 commit
    • Viktor Söderqvist's avatar
      redis-benchmark: improved help and warnings (#9419) · 97dcf95c
      Viktor Söderqvist authored
      1. The output of --help:
      
        * On the Usage line, just write [OPTIONS] [COMMAND ARGS...] instead listing
          only a few arbitrary options and no command.
        * For --cluster, describe that if the command is supplied on the command line,
          the key must contain "{tag}". Otherwise, the command will not be sent to the
          right cluster node.
        * For -r, add a note that if -r is omitted, all commands in a benchmark will
          use the same key. Also align the description.
        * For -t, describe that -t is ignored if a command is supplied on the command
          line.
      
      2. Print a warning if -t is present when a specific command is supplied.
      
      3. Print all warnings and errors to stderr.
      
      4. Remove -e from calls in redis-benchmark test suite.
      97dcf95c
  19. 25 Aug, 2021 1 commit
  20. 05 Aug, 2021 1 commit
    • yoav-steinberg's avatar
      dict struct memory optimizations (#9228) · 5e908a29
      yoav-steinberg authored
      Reduce dict struct memory overhead
      on 64bit dict size goes down from jemalloc's 96 byte bin to its 56 byte bin.
      
      summary of changes:
      - Remove `privdata` from callbacks and dict creation. (this affects many files, see "Interface change" below).
      - Meld `dictht` struct into the `dict` struct to eliminate struct padding. (this affects just dict.c and defrag.c)
      - Eliminate the `sizemask` field, can be calculated from size when needed.
      - Convert the `size` field into `size_exp` (exponent), utilizes one byte instead of 8.
      
      Interface change: pass dict pointer to dict type call back functions.
      This is instead of passing the removed privdata field. In the future if
      we'd like to have private data in the callbacks we can extract it from
      the dict type. We can extend dictType to include a custom dict struct
      allocator and use it to allocate more data at the end of the dict
      struct. This data can then be used to store private data later acccessed
      by the callbacks.
      5e908a29
  21. 03 Aug, 2021 1 commit
  22. 24 Jun, 2021 1 commit
  23. 10 Jun, 2021 1 commit
    • Binbin's avatar
      Fixed some typos, add a spell check ci and others minor fix (#8890) · 0bfccc55
      Binbin authored
      This PR adds a spell checker CI action that will fail future PRs if they introduce typos and spelling mistakes.
      This spell checker is based on blacklist of common spelling mistakes, so it will not catch everything,
      but at least it is also unlikely to cause false positives.
      
      Besides that, the PR also fixes many spelling mistakes and types, not all are a result of the spell checker we use.
      
      Here's a summary of other changes:
      1. Scanned the entire source code and fixes all sorts of typos and spelling mistakes (including missing or extra spaces).
      2. Outdated function / variable / argument names in comments
      3. Fix outdated keyspace masks error log when we check `config.notify-keyspace-events` in loadServerConfigFromString.
      4. Trim the white space at the end of line in `module.c`. Check: https://github.com/redis/redis/pull/7751
      5. Some outdated https link URLs.
      6. Fix some outdated comment. Such as:
          - In README: about the rdb, we used to said create a `thread`, change to `process`
          - dbRandomKey function coment (about the dictGetRandomKey, change to dictGetFairRandomKey)
          - notifyKeyspaceEvent fucntion comment (add type arg)
          - Some others minor fix in comment (Most of them are incorrectly quoted by variable names)
      7. Modified the error log so that users can easily distinguish between TCP and TLS in `changeBindAddr`
      0bfccc55
  24. 09 May, 2021 1 commit
  25. 06 May, 2021 1 commit
    • Oran Agra's avatar
      fix redis-benchmark to ignore unsupported configs (#8916) · 4d1094e8
      Oran Agra authored
      Redis Enterprise supports the CONFIG GET command, but it replies with am
      empty array since the save and appendonly configs are not supported.
      before this fix redis-benchmark would segfault for trying to access the
      error string on an array type reply.
      see #8869
      4d1094e8
  26. 28 Apr, 2021 2 commits
    • Binbin's avatar
      redis-benchmark: Add zfree(data) and fix lrange size / text mismatch (#8872) · 1eff8564
      Binbin authored
      missing zfree(data) in redis-benchmark.
      
      And also correct the wrong size in lrange.
      the text mentioned 500, but size was 450, changed to 500
      1eff8564
    • filipe oliveira's avatar
      redis-benchmark: Error/Warning handling updates. (#8869) · ef6f9023
      filipe oliveira authored
      - Immediately exit on errors that are not related to topology updates.
      - Deprecates the `-e` option ( retro compatible ) and warns that we now
        exit immediately on errors that are not related to topology updates.
      - Fixed wrongfully failing on config fetch error (warning only). This only affects RE.
      
      Bottom line:
      - MOVED and ASK errors will not show any warning (unlike the throttled error with `-e` before).
      - CLUSTERDOWN still prints an error unconditionally and sleeps for 1 second.
      - other errors are fatal.
      ef6f9023
  27. 26 Apr, 2021 1 commit
    • Andy Pan's avatar
      Fail fast when systemic error occurs in poll (#8749) · a8b6596d
      Andy Pan authored
      Most of the ae.c backends didn't explicitly handle errors, and instead
      ignored all errors and did an implicit retry.
      This is desired for EAGAIN and EINTER, but in case of other systematic
      errors, we prefer to fail and log the error we got rather than get into a busy loop.
      a8b6596d
  28. 25 Apr, 2021 1 commit
  29. 09 Mar, 2021 1 commit
  30. 05 Mar, 2021 1 commit
  31. 14 Feb, 2021 1 commit
  32. 07 Feb, 2021 1 commit
  33. 23 Dec, 2020 1 commit
    • Greg Femec's avatar
      Fix random element selection for large hash tables. (#8133) · 266949c7
      Greg Femec authored
      
      
      When a database on a 64 bit build grows past 2^31 keys, the underlying hash table expands to 2^32 buckets. After this point, the algorithms for selecting random elements only return elements from half of the available buckets because they use random() which has a range of 0 to 2^31 - 1. This causes problems for eviction policies which use dictGetSomeKeys or dictGetRandomKey. Over time they cause the hash table to become unbalanced because, while new keys are spread out evenly across all buckets, evictions come from only half of the available buckets. Eventually this half of the table starts to run out of keys and it takes longer and longer to find candidates for eviction. This continues until no more evictions can happen.
      
      This solution addresses this by using a 64 bit PRNG instead of libc random().
      Co-authored-by: default avatarGreg Femec <gfemec@google.com>
      266949c7
  34. 17 Dec, 2020 1 commit
    • Wang Yuan's avatar
      [Redis-benchmark] Use IP from CLUSTER NODE reply for first node too (#8154) · 6413e5f8
      Wang Yuan authored
      If we only has one node in cluster or before 8fdc857a, we don't know myself ip, so we should use config.hostip for myself.
      However, we should use the IP from the command response to update node->ip if it exists and is different from config.hostip
      
      otherwise, when there's more than one node in cluster, if we use -h with virtual IP or DNS, benchmark doesn't show node real ip and port of myself even though it could get right IP and port by CLUSTER NODES command.
      6413e5f8
  35. 13 Dec, 2020 1 commit
    • gourav's avatar
      Randomize the random number generator's seed used in redis-benchmark (#8174) · ddd43b6b
      gourav authored
      The pid of the benchmark process is used to randomize the random number generator's
      seed. This ensures that when multiple benchmark processes are started at the same time
      to generate load on a server, they use different seeds. This will ensure randomness in
      the keys generated by different benchmark processes.
      ddd43b6b
  36. 06 Dec, 2020 1 commit
    • Wang Yuan's avatar
      Limit the main db and expires dictionaries to expand (#7954) · 75f9dec6
      Wang Yuan authored
      As we know, redis may reject user's requests or evict some keys if
      used memory is over maxmemory. Dictionaries expanding may make
      things worse, some big dictionaries, such as main db and expires dict,
      may eat huge memory at once for allocating a new big hash table and be
      far more than maxmemory after expanding.
      There are related issues: #4213 #4583
      
      More details, when expand dict in redis, we will allocate a new big
      ht[1] that generally is double of ht[0], The size of ht[1] will be
      very big if ht[0] already is big. For db dict, if we have more than
      64 million keys, we need to cost 1GB for ht[1] when dict expands.
      
      If the sum of used memory and new hash table of dict needed exceeds
      maxmemory, we shouldn't allow the dict to expand. Because, if we
      enable keys eviction, we still couldn't add much more keys after
      eviction and rehashing, what's worse, redis will keep less keys when
      redis only remains a little memory for storing new hash table instead
      of users' data. Moreover users can't write data in redis if disable
      keys eviction.
      
      What this commit changed ?
      
      Add a new member function expandAllowed for dict type, it provide a way
      for caller to allow expand or not. We expose two parameters for this
      function: more memory needed for expanding and dict current load factor,
      users can implement a function to make a decision by them.
      For main db dict and expires dict type, these dictionaries may be very
      big and cost huge memory for expanding, so we implement a judgement
      function: we can stop dict to expand provisionally if used memory will
      be over maxmemory after dict expands, but to guarantee the performance
      of redis, we still allow dict to expand if dict load factor exceeds the
      safe load factor.
      Add test cases to verify we don't allow main db to expand when left
      memory is not enough, so that avoid keys eviction.
      
      Other changes:
      
      For new hash table size when expand. Before this commit, the size is
      that double used of dict and later _dictNextPower. Actually we aim to
      control a dict load factor between 0.5 and 1.0. Now we replace *2 with
      +1, since the first check is that used >= size, the outcome of before
      will usually be the same as _dictNextPower(used+1). The only case where
      it'll differ is when dict_can_resize is false during fork, so that later
      the _dictNextPower(used*2) will cause the dict to jump to *4 (i.e.
      _dictNextPower(1025*2) will return 4096).
      Fix rehash test cases due to changing algorithm of new hash table size
      when expand.
      75f9dec6
  37. 02 Dec, 2020 1 commit
    • luhuachao's avatar
      Modify help msg PING_BULK to PING_MBULK in benchmark (#8109) · 7885faf1
      luhuachao authored
      As described in redis-benchamrk help message 'The test names are the same as the ones produced as output.', In redis-benchmark output, we can only see PING_BULK, but the cmd `redis-benchmark -t ping_bulk` is not supported. We  have to run it with ping_mbulk which is not user friendly.
      7885faf1
  38. 04 Nov, 2020 1 commit
  39. 28 Oct, 2020 1 commit