1. 31 Oct, 2024 2 commits
  2. 13 Jul, 2023 1 commit
    • Chayim I. Kirshen's avatar
      Squashed 'deps/hiredis/' changes from b6a052fe0..60e5075d4 · 67ed3183
      Chayim I. Kirshen authored
      60e5075d4 Version 1.2.0 (#1202)
      adef139a7 Remove support in deprecated TLS versions 1.0 and 1.1 (#1205)
      d543baba6 Install major version symlink of shared objects.
      052f99ab2 Ensure functionality without _MSC_VER definition
      ded32c7d1 Add a test for the TCP_USER_TIMEOUT option. (#1192)
      5cbd1f296 Add -Werror as a default. (#1193)
      af1445638 CI: Update homebrew Redis version. (#1191)
      6de326e87 Fix a typo in b6a052f.
      
      git-subtree-dir: deps/hiredis
      git-subtree-split: 60e5075d4ac77424809f855ba3e398df7aacefe8
      67ed3183
  3. 27 Jun, 2023 1 commit
  4. 30 May, 2023 1 commit
    • Yossi Gottlieb's avatar
      Squashed 'deps/hiredis/' changes from f8de9a4bd..b6a052fe0 · fad6c713
      Yossi Gottlieb authored
      b6a052fe0 Helper for setting TCP_USER_TIMEOUT socket option (#1188)
      3fa9b6944 Add RedisModule adapter (#1182)
      d13c091e9 Fix wincrypt symbols conflict
      5d84c8cfd Add a test ensuring we don't clobber connection error.
      3f95fcdae Don't attempt to set a timeout if we are in an error state.
      aacb84b8d Fix typo in makefile.
      563b062e3 Accept -nan per the RESP3 spec recommendation.
      04c1b5b02 Fix colliding option values
      4ca8e73f6 Rework searching for openssl
      cd208812f Attempt to find the correct path for openssl.
      011f7093c Allow specifying the keepalive interval
      e9243d4f7 Cmake static or shared (#1160)
      1cbd5bc76 Write a version file for the CMake package (#1165)
      6f5bae8c6 fix typo
      acd09461d CMakeLists.txt: respect BUILD_SHARED_LIBS
      97fcf0fd1 Add sdevent adapter
      ccff093bc Bump dev version for the next release cycle.
      c14775b4e Prepare for v1.1.0 GA
      f0bdf8405 Add support for nan in RESP3 double (#1133)
      991b0b0b3 Add an example that calls redisCommandArgv (#1140)
      a36686f84 CI updates (#1139)
      8ad4985e9 fix flag reference
      7583ebb1b Make freeing a NULL redisAsyncContext a no op.
      2c53dea7f Update version in dev branch.
      f063370ed Prepare for v1.1.0-rc1
      2b069573a CI fixes in preparation of release
      e1e9eb40d Add author information to release-drafter template.
      afc29ee1a Update for mingw cross compile
      ceb8a8815 fixed cpp build error with adapters/libhv.h
      3b15a04b5 Fixup of PR734: Coverage of hiredis.c (#1124)
      c245df9fb CMake corrections for building on Windows (#1122)
      9c338a598 Fix PUSH handler tests for Redis >= 7.0.5
      6d5c3ee74 Install on windows fixes (#1117)
      68b29e1ad Add timeout support to libhv adapter. (#1109)
      722e3409c Additional include directory given by pkg-config (#1118)
      bd9ccb8c4 Use __attribute__ when building with clang on windows
      5392adc26 set default SSL certificate directory
      560e66486 Minor refactor
      d756f68a5 Add libhv example to our standard Makefile
      a66916719 Add adapters/libhv
      855b48a81 Fix pkgconfig for hiredis_ssl
      79ae5ffc6 Fix protocol error (#1106)
      61b5b299f Use a windows specific keepalive function. (#1104)
      fce8abc1c Introduce .close method for redisContextFuncs
      cfb6ca881 Add REDIS_OPT_PREFER_UNSPEC (#1101)
      cc7c35ce6 Update documentation to explain redisConnectWithOptions.
      bc8d837b7 fix heap-buffer-overflow (#957)
      ca4a0e850 uvadapter: reduce number of uv_poll_start calls
      35d398c90 Fix cmake config path on Linux. CMake config files were installed to `/usr/local/share/hiredis`, which is not recognizable by `find_package()`. I'm not sure why it was set that way. Given the commit introducing it is for Windows, I keep that behavior consistent there, but fix the rest.
      10c78c6e1 Add possibility to prefer IPv6, IPv4 or unspecified
      1abe0c828 fuzzer: No alloc in redisFormatCommand() when fail
      329eaf9ba Fix heap-buffer-overflow issue in redisvFormatCommad
      eaae7321c Polling adapter requires sockcompat.h
      0a5fa3dde Regression test for off-by-one parsing error
      9e174e8f7 Add do while(0) protection for macros
      4ad99c69a Rework asSleep to be a generic millisleep function.
      75cb6c1ea Do store command timeout in the context for redisSetTimeout (#593)
      c57cad658 CMake: remove dict.c form hiredis_sources
      8491a65a9 Add Github Actions CI workflow for hiredis: Arm, Arm64, 386, windows. (#943)
      77e4f09ea Merge pull request #964 from afcidk/fix-createDoubleObject
      9219f7e7c Merge pull request #901 from devnexen/illumos_test_fix
      810cc6104 Merge pull request #905 from sundb/master
      df8b74d69 Merge pull request #1091 from redis/ssl-error-ub-fix
      0ed6cdec3 Fix some undefined behaviour
      507a6dcaa Merge pull request #1090 from Nordix/subscribe-oom-error
      b044eaa6a Copy error to redisAsyncContext when finding subscribe cb
      e0200b797 Merge pull request #1087 from redis/const-and-non-const-callback
      6a3e96ad2 Maintain backward compatibiliy withour onConnect callback.
      e7afd998f Merge pull request #1079 from SukkaW/drop-macos-10.15-runner
      17c8fe079 Merge pull request #931 from kristjanvalur/pr2
      b808c0c20 Merge pull request #1083 from chayim/ck-drafter
      367a82bf0 Merge pull request #1085 from stanhu/ssl-improve-options-setting
      71119a71d Make it possible to set SSL verify mode
      dd7979ac1 Merge pull request #1084 from stanhu/sh-improve-ssl-docs
      c71116178 Improve example for SSL initialization in README.md
      5c9b6b571 Release drafter
      a606ccf2a CI: use recommended `vmactions/freebsd-vm@v0`
      0865c115b Merge pull request #1080 from Nordix/readme-corrections
      f6cee7142 Fix README typos
      06be7ff31 Merge pull request #1050 from smmir-cent/fix-cmake-version
      7dd833d54 CI: bump macos runner version
      f69fac769 Drop `const` on redisAsyncContext in redisConnectCallback Since the callback is now re-entrant, it can call apis such as redisAsyncDisconnect()
      005d7edeb Support calling redisAsyncDisconnect from the onConnected callback, by deferring context deletion
      6ed060920 Add async regression test for issue #931
      eaa2a7ee7 Merge pull request #932 from kristjanvalur/pr3
      2ccef30f3 Add regression test for issue #945
      4b901d44a Initial async tests
      31c91408e Polling adapter and example
      8a15f4d65 Merge pull request #1057 from orgads/static-name
      902dd047f Merge pull request #1054 from kristjanvalur/pr08
      c78d0926b Merge pull request #1074 from michael-grunder/kristjanvalur-pr4
      2b115d56c Whitespace
      1343988ce Fix typos
      47b57aa24 Add some documentation on connect/disconnect callbacks and command callbacks
      a890d9ce2 Merge pull request #1073 from michael-grunder/kristjanvalur-pr1
      f246ee433 Whitespace, style
      94c1985bd Use correct type for getsockopt()
      5e002bc21 Support failed async connects on windows.
      5d68ad2f4 Merge pull request #1072 from michael-grunder/fix-redis7-unit-tests
      f4b6ed289 Fix tests so they work for Redis 7.0
      95a0c1283 Merge pull request #1058 from orgads/win64
      eedb37a65 Fix warnings on Win64
      47c3ecefc Merge pull request #1062 from yossigo/fix-push-notification-order
      e23d91c97 Merge pull request #1061 from yossigo/update-redis-apt
      34211ad54 Merge pull request #1063 from redis/fix-windows-tests
      9957af7e3 Whitelist hiredis repo path in cygwin
      b455b3381 Handle push notifications before or after reply.
      aed9ce446 Use official repository for redis package.
      d7683f35a Merge pull request #1047 from Nordix/unsubscribe-handling
      7c44a9d7e Merge pull request #1045 from Nordix/sds-updates
      dd4bf9783 Use the same name for static and shared libraries
      ff57c18b9 Embed debug information in windows static lib, rather than create a .pdb file
      8310ad4f5 fix cmake version
      7123b87f6 Handle any pipelined unsubscribe in async
      b6fb548fc Ignore pubsub replies without a channel/pattern
      00b82683b Handle overflows as errors instead of asserting
      64062a1d4 Catch size_t overflows in sds.c
      066c6de79 Use size_t/long to avoid truncation
      c6657ef65 Merge branch 'redis:master' into master
      50cdcab49 Fix potential fault at createDoubleObject
      fd033e983 Remove semicolon after do-while in _EL_CLEANUP
      664c415e7 Illumos test fixes, error message difference fot bad hostname test.
      
      git-subtree-dir: deps/hiredis
      git-subtree-split: b6a052fe0959dae69e16b9d74449faeb1b70dbe1
      fad6c713
  5. 24 May, 2023 1 commit
  6. 25 Apr, 2023 1 commit
  7. 18 Apr, 2023 1 commit
    • sundb's avatar
      Fix some compile warnings and errors when building with gcc-12 or clang (#12035) · 42c8c618
      sundb authored
      This PR is to fix the compilation warnings and errors generated by the latest
      complier toolchain, and to add a new runner of the latest toolchain for daily CI.
      
      ## Fix various compilation warnings and errors
      
      1) jemalloc.c
      
      COMPILER: clang-14 with FORTIFY_SOURCE
      
      WARNING:
      ```
      src/jemalloc.c:1028:7: warning: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Wstring-concatenation]
                          "/etc/malloc.conf",
                          ^
      src/jemalloc.c:1027:3: note: place parentheses around the string literal to silence warning
                      "\"name\" of the file referenced by the symbolic link named "
                      ^
      ```
      
      REASON:  the compiler to alert developers to potential issues with string concatenation
      that may miss a comma,
      just like #9534 which misses a comma.
      
      SOLUTION: use `()` to tell the compiler that these two line strings are continuous.
      
      2) config.h
      
      COMPILER: clang-14 with FORTIFY_SOURCE
      
      WARNING:
      ```
      In file included from quicklist.c:36:
      ./config.h:319:76: warning: attribute declaration must precede definition [-Wignored-attributes]
      char *strcat(char *restrict dest, const char *restrict src) __attribute__((deprecated("please avoid use of unsafe C functions. prefer use of redis_strlcat instead")));
      ```
      
      REASON: Enabling _FORTIFY_SOURCE will cause the compiler to use `strcpy()` with check,
      it results in a deprecated attribute declaration after including <features.h>.
      
      SOLUTION: move the deprecated attribute declaration from config.h to fmacro.h before "#include <features.h>".
      
      3) networking.c
      
      COMPILER: GCC-12
      
      WARNING: 
      ```
      networking.c: In function ‘addReplyDouble.part.0’:
      networking.c:876:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
        876 |         dbuf[start] = '$';
            |                     ^
      networking.c:868:14: note: at offset -5 into destination object ‘dbuf’ of size 5152
        868 |         char dbuf[MAX_LONG_DOUBLE_CHARS+32];
            |              ^
      networking.c:876:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
        876 |         dbuf[start] = '$';
            |                     ^
      networking.c:868:14: note: at offset -6 into destination object ‘dbuf’ of size 5152
        868 |         char dbuf[MAX_LONG_DOUBLE_CHARS+32];
      ```
      
      REASON: GCC-12 predicts that digits10() may return 9 or 10 through `return 9 + (v >= 1000000000UL)`.
      
      SOLUTION: add an assert to let the compiler know the possible length;
      
      4) redis-cli.c & redis-benchmark.c
      
      COMPILER: clang-14 with FORTIFY_SOURCE
      
      WARNING:
      ```
      redis-benchmark.c:1621:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive] #ifdef USE_OPENSSL
      redis-cli.c:3015:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive] #ifdef USE_OPENSSL
      ```
      
      REASON: when _FORTIFY_SOURCE is enabled, the compiler will use the print() with
      check, which is a macro. this may result in the use of directives within the macro, which
      is undefined behavior.
      
      SOLUTION: move the directives-related code out of `print()`.
      
      5) server.c
      
      COMPILER: gcc-13 with FORTIFY_SOURCE
      
      WARNING:
      ```
      In function 'lookupCommandLogic',
          inlined from 'lookupCommandBySdsLogic' at server.c:3139:32:
      server.c:3102:66: error: '*(robj **)argv' may be used uninitialized [-Werror=maybe-uninitialized]
       3102 |     struct redisCommand *base_cmd = dictFetchValue(commands, argv[0]->ptr);
            |                                                              ~~~~^~~
      ```
      
      REASON: The compiler thinks that the `argc` returned by `sdssplitlen()` could be 0,
      resulting in an empty array of size 0 being passed to lookupCommandLogic.
      this should be a false positive, `argc` can't be 0 when strings are not NULL.
      
      SOLUTION: add an assert to let the compiler know that `argc` is positive.
      
      6) sha1.c
      
      COMPILER: gcc-12
      
      WARNING:
      ```
      In function ‘SHA1Update’,
          inlined from ‘SHA1Final’ at sha1.c:195:5:
      sha1.c:152:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread]
        152 |             SHA1Transform(context->state, &data[i]);
            |             ^
      sha1.c:152:13: note: referencing argument 2 of type ‘const unsigned char[64]’
      sha1.c: In function ‘SHA1Final’:
      sha1.c:56:6: note: in a call to function ‘SHA1Transform’
         56 | void SHA1Transform(uint32_t state[5], const unsigned char buffer[64])
            |      ^
      In function ‘SHA1Update’,
          inlined from ‘SHA1Final’ at sha1.c:198:9:
      sha1.c:152:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread]
        152 |             SHA1Transform(context->state, &data[i]);
            |             ^
      sha1.c:152:13: note: referencing argument 2 of type ‘const unsigned char[64]’
      sha1.c: In function ‘SHA1Final’:
      sha1.c:56:6: note: in a call to function ‘SHA1Transform’
         56 | void SHA1Transform(uint32_t state[5], const unsigned char buffer[64])
      ```
      
      REASON: due to the bug[https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80922], when
      enable LTO, gcc-12 will not see `diagnostic ignored "-Wstringop-overread"`, resulting in a warning.
      
      SOLUTION: temporarily set SHA1Update to noinline to avoid compiler warnings due
      to LTO being enabled until the above gcc bug is fixed.
      
      7) zmalloc.h
      
      COMPILER: GCC-12
      
      WARNING: 
      ```
      In function ‘memset’,
          inlined from ‘moduleCreateContext’ at module.c:877:5,
          inlined from ‘RM_GetDetachedThreadSafeContext’ at module.c:8410:5:
      /usr/include/x86_64-linux-gnu/bits/string_fortified.h:59:10: warning: ‘__builtin_memset’ writing 104 bytes into a region of size 0 overflows the destination [-Wstringop-overflow=]
         59 |   return __builtin___memset_chk (__dest, __ch, __len,
      ```
      
      REASON: due to the GCC-12 bug [https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503],
      GCC-12 cannot see alloc_size, which causes GCC to think that the actual size of memory
      is 0 when checking with __glibc_objsize0().
      
      SOLUTION: temporarily set malloc-related interfaces to `noinline` to avoid compiler warnings
      due to LTO being enabled until the above gcc bug is fixed.
      
      ## Other changes
      1) Fixed `ps -p [pid]`  doesn't output `<defunct>` when using procps 4.x causing `replication
        child dies when parent is killed - diskless` test to fail.
      2) Add a new fortify CI with GCC-13 and ubuntu-lunar docker image.
      42c8c618
  8. 12 Apr, 2023 2 commits
    • Binbin's avatar
      Don't pass --fail-commands-not-all-hit to validator if we don't run the full testsuite (#12023) · 810ea67b
      Binbin authored
      In daily.yml, if the input suggests we don't run the full testsuite,
      do not pass --fail-commands-not-all-hit to the validator.
      
      This fixes the first point in #11954. Credit goes to the comment
      on the open issue for GH actions: actions/runner#409
      
      Also improve prints to show the dispatch arguments in every job.
      810ea67b
    • Oran Agra's avatar
      Attempt to solve MacOS CI issues in GH Actions (#12013) · 997fa41e
      Oran Agra authored
      The MacOS CI in github actions often hangs without any logs. GH argues that
      it's due to resource utilization, either running out of disk space, memory, or CPU
      starvation, and thus the runner is terminated.
      
      This PR contains multiple attempts to resolve this:
      1. introducing pause_process instead of SIGSTOP, which waits for the process
        to stop before resuming the test, possibly resolving race conditions in some tests,
        this was a suspect since there was one test that could result in an infinite loop in that
       case, in practice this didn't help, but still a good idea to keep.
      2. disable the `save` config in many tests that don't need it, specifically ones that use
        heavy writes and could create large files.
      3. change the `populate` proc to use short pipeline rather than an infinite one.
      4. use `--clients 1` in the macos CI so that we don't risk running multiple resource
        demanding tests in parallel.
      5. enable `--verbose` to be repeated to elevate verbosity and print more info to stdout
        when a test or a server starts.
      997fa41e
  9. 05 Apr, 2023 1 commit
    • Oran Agra's avatar
      Increase threshold for flaky cache reclaim test (#12004) · f263b6da
      Oran Agra authored
      This test produces 1GB of data and moves it around, and was expecting less
      than 500kb to be present in the system page cache.
      It sometimes fails with up to some 6mb in the page cache (0 in the actual RDB files),
      increasing the threshold. It looks like some background tasks in the container are
      occupying the page cache.
      
      It is safe to ignore the above since we also explicitly check the pages of our dump.rdb
      are not cached (matching `vmtouch -v` to `0%`).
      An additional fix is to match ` 0%` (add space), so that we don't successfully match `10%`.
      
      details in https://github.com/redis/redis/pull/11818
      f263b6da
  10. 23 Mar, 2023 1 commit
  11. 11 Mar, 2023 1 commit
    • guybe7's avatar
      Add reply_schema to command json files (internal for now) (#10273) · 4ba47d2d
      guybe7 authored
      Work in progress towards implementing a reply schema as part of COMMAND DOCS, see #9845
      Since ironing the details of the reply schema of each and every command can take a long time, we
      would like to merge this PR when the infrastructure is ready, and let this mature in the unstable branch.
      Meanwhile the changes of this PR are internal, they are part of the repo, but do not affect the produced build.
      
      ### Background
      In #9656 we add a lot of information about Redis commands, but we are missing information about the replies
      
      ### Motivation
      1. Documentation. This is the primary goal.
      2. It should be possible, based on the output of COMMAND, to be able to generate client code in typed
        languages. In order to do that, we need Redis to tell us, in detail, what each reply looks like.
      3. We would like to build a fuzzer that verifies the reply structure (for now we use the existing
        testsuite, see the "Testing" section)
      
      ### Schema
      The idea is to supply some sort of schema for the various replies of each command.
      The schema will describe the conceptual structure of the reply (for generated clients), as defined in RESP3.
      Note that the reply structure itself may change, depending on the arguments (e.g. `XINFO STREAM`, with
      and without the `FULL` modifier)
      We decided to use the standard json-schema (see https://json-schema.org/) as the reply-schema.
      
      Example for `BZPOPMIN`:
      ```
      "reply_schema": {
          "oneOf": [
              {
                  "description": "Timeout reached and no elements were popped.",
                  "type": "null"
              },
              {
                  "description": "The keyname, popped member, and its score.",
                  "type": "array",
                  "minItems": 3,
                  "maxItems": 3,
                  "items": [
                      {
                          "description": "Keyname",
                          "type": "string"
                      },
                      {
                          "description": "Member",
                          "type": "string"
                      },
                      {
                          "description": "Score",
                          "type": "number"
                      }
                  ]
              }
          ]
      }
      ```
      
      #### Notes
      1.  It is ok that some commands' reply structure depends on the arguments and it's the caller's responsibility
        to know which is the relevant one. this comes after looking at other request-reply systems like OpenAPI,
        where the reply schema can also be oneOf and the caller is responsible to know which schema is the relevant one.
      2. The reply schemas will describe RESP3 replies only. even though RESP3 is structured, we want to use reply
        schema for documentation (and possibly to create a fuzzer that validates the replies)
      3. For documentation, the description field will include an explanation of the scenario in which the reply is sent,
        including any relation to arguments. for example, for `ZRANGE`'s two schemas we will need to state that one
        is with `WITHSCORES` and the other is without.
      4. For documentation, there will be another optional field "notes" in which we will add a short description of
        the representation in RESP2, in case it's not trivial (RESP3's `ZRANGE`'s nested array vs. RESP2's flat
        array, for example)
      
      Given the above:
      1. We can generate the "return" section of all commands in [redis-doc](https://redis.io/commands/)
        (given that "description" and "notes" are comprehensive enough)
      2. We can generate a client in a strongly typed language (but the return type could be a conceptual
        `union` and the caller needs to know which schema is relevant). see the section below for RESP2 support.
      3. We can create a fuzzer for RESP3.
      
      ### Limitations (because we are using the standard json-schema)
      The problem is that Redis' replies are more diverse than what the json format allows. This means that,
      when we convert the reply to a json (in order to validate the schema against it), we lose information (see
      the "Testing" section below).
      The other option would have been to extend the standard json-schema (and json format) to include stuff
      like sets, bulk-strings, error-string, etc. but that would mean also extending the schema-validator - and that
      seemed like too much work, so we decided to compromise.
      
      Examples:
      1. We cannot tell the difference between an "array" and a "set"
      2. We cannot tell the difference between simple-string and bulk-string
      3. we cannot verify true uniqueness of items in commands like ZRANGE: json-schema doesn't cover the
        case of two identical members with different scores (e.g. `[["m1",6],["m1",7]]`) because `uniqueItems`
        compares (member,score) tuples and not just the member name. 
      
      ### Testing
      This commit includes some changes inside Redis in order to verify the schemas (existing and future ones)
      are indeed correct (i.e. describe the actual response of Redis).
      To do that, we added a debugging feature to Redis that causes it to produce a log of all the commands
      it executed and their replies.
      For that, Redis needs to be compiled with `-DLOG_REQ_RES` and run with
      `--reg-res-logfile <file> --client-default-resp 3` (the testsuite already does that if you run it with
      `--log-req-res --force-resp3`)
      You should run the testsuite with the above args (and `--dont-clean`) in order to make Redis generate
      `.reqres` files (same dir as the `stdout` files) which contain request-response pairs.
      These files are later on processed by `./utils/req-res-log-validator.py` which does:
      1. Goes over req-res files, generated by redis-servers, spawned by the testsuite (see logreqres.c)
      2. For each request-response pair, it validates the response against the request's reply_schema
        (obtained from the extended COMMAND DOCS)
      5. In order to get good coverage of the Redis commands, and all their different replies, we chose to use
        the existing redis test suite, rather than attempt to write a fuzzer.
      
      #### Notes about RESP2
      1. We will not be able to use the testing tool to verify RESP2 replies (we are ok with that, it's time to
        accept RESP3 as the future RESP)
      2. Since the majority of the test suite is using RESP2, and we want the server to reply with RESP3
        so that we can validate it, we will need to know how to convert the actual reply to the one expected.
         - number and boolean are always strings in RESP2 so the conversion is easy
         - objects (maps) are always a flat array in RESP2
         - others (nested array in RESP3's `ZRANGE` and others) will need some special per-command
           handling (so the client will not be totally auto-generated)
      
      Example for ZRANGE:
      ```
      "reply_schema": {
          "anyOf": [
              {
                  "description": "A list of member elements",
                  "type": "array",
                  "uniqueItems": true,
                  "items": {
                      "type": "string"
                  }
              },
              {
                  "description": "Members and their scores. Returned in case `WITHSCORES` was used.",
                  "notes": "In RESP2 this is returned as a flat array",
                  "type": "array",
                  "uniqueItems": true,
                  "items": {
                      "type": "array",
                      "minItems": 2,
                      "maxItems": 2,
                      "items": [
                          {
                              "description": "Member",
                              "type": "string"
                          },
                          {
                              "description": "Score",
                              "type": "number"
                          }
                      ]
                  }
              }
          ]
      }
      ```
      
      ### Other changes
      1. Some tests that behave differently depending on the RESP are now being tested for both RESP,
        regardless of the special log-req-res mode ("Pub/Sub PING" for example)
      2. Update the history field of CLIENT LIST
      3. Added basic tests for commands that were not covered at all by the testsuite
      
      ### TODO
      
      - [x] (maybe a different PR) add a "condition" field to anyOf/oneOf schemas that refers to args. e.g.
        when `SET` return NULL, the condition is `arguments.get||arguments.condition`, for `OK` the condition
        is `!arguments.get`, and for `string` the condition is `arguments.get` - https://github.com/redis/redis/issues/11896
      - [x] (maybe a different PR) also run `runtest-cluster` in the req-res logging mode
      - [x] add the new tests to GH actions (i.e. compile with `-DLOG_REQ_RES`, run the tests, and run the validator)
      - [x] (maybe a different PR) figure out a way to warn about (sub)schemas that are uncovered by the output
        of the tests - https://github.com/redis/redis/issues/11897
      - [x] (probably a separate PR) add all missing schemas
      - [x] check why "SDOWN is triggered by misconfigured instance replying with errors" fails with --log-req-res
      - [x] move the response transformers to their own file (run both regular, cluster, and sentinel tests - need to
        fight with the tcl including mechanism a bit)
      - [x] issue: module API - https://github.com/redis/redis/issues/11898
      - [x] (probably a separate PR): improve schemas: add `required` to `object`s - https://github.com/redis/redis/issues/11899
      
      Co-authored-by: default avatarOzan Tezcan <ozantezcan@gmail.com>
      Co-authored-by: default avatarHanna Fadida <hanna.fadida@redislabs.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarShaya Potter <shaya@redislabs.com>
      4ba47d2d
  12. 19 Feb, 2023 1 commit
    • Oran Agra's avatar
      Stablize page reclaim CI test (#11818) · 3ac83577
      Oran Agra authored
      stabilize the test introduced in #11248
      * remove random aspect of the test by using DEBUG POPULATE instead of redis-benchmark
      * disable rdbcompression, so that the rdb file is always about 1GB.
      
      when fadvise was disabled, i get about 1GB in the page cace
      when enabled i get less than 200KB
      so for now, i'll keep the 500kb threshold.
      3ac83577
  13. 16 Feb, 2023 1 commit
  14. 12 Feb, 2023 1 commit
    • Tian's avatar
      Reclaim page cache of RDB file (#11248) · 7dae142a
      Tian authored
      # Background
      The RDB file is usually generated and used once and seldom used again, but the content would reside in page cache until OS evicts it. A potential problem is that once the free memory exhausts, the OS have to reclaim some memory from page cache or swap anonymous page out, which may result in a jitters to the Redis service.
      
      Supposing an exact scenario, a high-capacity machine hosts many redis instances, and we're upgrading the Redis together. The page cache in host machine increases as RDBs are generated. Once the free memory drop into low watermark(which is more likely to happen in older Linux kernel like 3.10, before [watermark_scale_factor](https://lore.kernel.org/lkml/1455813719-2395-1-git-send-email-hannes@cmpxchg.org/) is introduced, the `low watermark` is linear to `min watermark`, and there'is not too much buffer space for `kswapd` to be wake up to reclaim memory), a `direct reclaim` happens, which means the process would stall to wait for memory allocation.
      
      # What the PR does
      The PR introduces a capability to reclaim the cache when the RDB is operated. Generally there're two cases, read and write the RDB. For read it's a little messy to address the incremental reclaim, so the reclaim is done in one go in background after the load is finished to avoid blocking the work thread. For write, incremental reclaim amortizes the work of reclaim so no need to put it into background, and the peak watermark of cache can be reduced in this way.
      
      Two cases are addresses specially, replication and restart, for both of which the cache is leveraged to speed up the processing, so the reclaim is postponed to a right time. To do this, a flag is added to`rdbSave` and `rdbLoad` to control whether the cache need to be kept, with the default value false.
      
      # Something deserve noting
      1. Though `posix_fadvise` is the POSIX standard, but only few platform support it, e.g. Linux, FreeBSD 10.0.
      2. In Linux `posix_fadvise` only take effect on writeback-ed pages, so a `sync`(or `fsync`, `fdatasync`) is needed to flush the dirty page before `posix_fadvise` if we reclaim write cache.
      
      # About test
      A unit test is added to verify the effect of `posix_fadvise`.
      In integration test overall cache increase is checked, as well as the cache backed by RDB as a specific TCL test is executed in isolated Github action job.
      7dae142a
  15. 21 Nov, 2022 1 commit
    • Binbin's avatar
      Fix sentinel update loglevel tls test (#11528) · 0f857131
      Binbin authored
      Apparently we used to set `loglevel debug` for tls in spawn_instance.
      I.e. cluster and sentinel tests used to run with debug logging, only when tls mode was enabled.
      this was probably a leftover from when creating the tls mode tests.
      it cause a new test created for #11214 to fail in tls mode.
      
      At the same time, in order to better distinguish the tests, change the
      name of `test-centos7-tls` to `test-centos7-tls-module`, change the name
      of `test-centos7-tls-no-tls` to `test-centos7-tls-module-no-tls`.
      
      Note that in `test-centos7-tls-module`, we did not pass `--tls-module`
      in sentinel test because it is not supported, see 4faddf18
      
      , added in #9320.
      So only `test-ubuntu-tls` fails in daily CI.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      0f857131
  16. 04 Nov, 2022 1 commit
    • Binbin's avatar
      Bump vmactions/freebsd-vm to 0.3.0 to fix FreeBSD daily (#11476) · 5246bf45
      Binbin authored
      Our FreeBSD daily has been failing recently:
      ```
        Config file: freebsd-13.1.conf
        cd: /Users/runner/work/redis/redis: No such file or directory
        gmake: *** No targets specified and no makefile found.  Stop.
      ```
      
      Upgrade vmactions/freebsd-vm to the latest version (0.3.0) can work.
      I've tested it, but don't know why, but first let's fix it.
      5246bf45
  17. 31 Aug, 2022 1 commit
  18. 23 Aug, 2022 1 commit
    • Oran Agra's avatar
      Build TLS as a loadable module · 4faddf18
      Oran Agra authored
      
      
      * Support BUILD_TLS=module to be loaded as a module via config file or
        command line. e.g. redis-server --loadmodule redis-tls.so
      * Updates to redismodule.h to allow it to be used side by side with
        server.h by defining REDISMODULE_CORE_MODULE
      * Changes to server.h, redismodule.h and module.c to avoid repeated
        type declarations (gcc 4.8 doesn't like these)
      * Add a mechanism for non-ABI neutral modules (ones who include
        server.h) to refuse loading if they detect not being built together with
        redis (release.c)
      * Fix wrong signature of RedisModuleDefragFunc, this could break
        compilation of a module, but not the ABI
      * Move initialization of listeners in server.c to be after loading
        the modules
      * Config TLS after initialization of listeners
      * Init cluster after initialization of listeners
      * Add TLS module to CI
      * Fix a test suite race conditions:
        Now that the listeners are initialized later, it's not sufficient to
        wait for the PID message in the log, we need to wait for the "Server
        Initialized" message.
      * Fix issues with moduleconfigs test as a result from start_server
        waiting for "Server Initialized"
      * Fix issues with modules/infra test as a result of an additional module
        present
      
      Notes about Sentinel:
      Sentinel can't really rely on the tls module, since it uses hiredis to
      initiate connections and depends on OpenSSL (won't be able to use any
      other connection modules for that), so it was decided that when TLS is
      built as a module, sentinel does not support TLS at all.
      This means that it keeps using redis_tls_ctx and redis_tls_client_ctx directly.
      
      Example code of config in redis-tls.so(may be use in the future):
      RedisModuleString *tls_cfg = NULL;
      
      void tlsInfo(RedisModuleInfoCtx *ctx, int for_crash_report) {
          UNUSED(for_crash_report);
          RedisModule_InfoAddSection(ctx, "");
          RedisModule_InfoAddFieldLongLong(ctx, "var", 42);
      }
      
      int tlsCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
      {
          if (argc != 2) return RedisModule_WrongArity(ctx);
          return RedisModule_ReplyWithString(ctx, argv[1]);
      }
      
      RedisModuleString *getStringConfigCommand(const char *name, void *privdata) {
          REDISMODULE_NOT_USED(name);
          REDISMODULE_NOT_USED(privdata);
          return tls_cfg;
      }
      
      int setStringConfigCommand(const char *name, RedisModuleString *new, void *privdata, RedisModuleString **err) {
          REDISMODULE_NOT_USED(name);
          REDISMODULE_NOT_USED(err);
          REDISMODULE_NOT_USED(privdata);
          if (tls_cfg) RedisModule_FreeString(NULL, tls_cfg);
          RedisModule_RetainString(NULL, new);
          tls_cfg = new;
          return REDISMODULE_OK;
      }
      
      int RedisModule_OnLoad(void *ctx, RedisModuleString **argv, int argc)
      {
          ....
          if (RedisModule_CreateCommand(ctx,"tls",tlsCommand,"",0,0,0) == REDISMODULE_ERR)
              return REDISMODULE_ERR;
      
          if (RedisModule_RegisterStringConfig(ctx, "cfg", "", REDISMODULE_CONFIG_DEFAULT, getStringConfigCommand, setStringConfigCommand, NULL, NULL) == REDISMODULE_ERR)
              return REDISMODULE_ERR;
      
          if (RedisModule_LoadConfigs(ctx) == REDISMODULE_ERR) {
              if (tls_cfg) {
                  RedisModule_FreeString(ctx, tls_cfg);
                  tls_cfg = NULL;
              }
              return REDISMODULE_ERR;
          }
          ...
      }
      Co-authored-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      Signed-off-by: default avatarzhenwei pi <pizhenwei@bytedance.com>
      4faddf18
  19. 02 Aug, 2022 1 commit
  20. 19 Jul, 2022 1 commit
  21. 04 May, 2022 1 commit
  22. 12 Apr, 2022 1 commit
  23. 30 Mar, 2022 2 commits
  24. 29 Mar, 2022 1 commit
    • Oran Agra's avatar
      fix daily.yaml skip filters (#10490) · 16d206ee
      Oran Agra authored
      * missing parenthesis meant that the ubuntu and centos jobs were not
        skipped
      * the recently divided freebsd, macos, and valgrind jobs, which are now
        split into distict jobs for redis, modules, sentinel, cluster. were
        all executed, producing a build, but not running anything.
        now they're filtered at the job level
      * iothreads was missing from the skip list defaults, so was not skipped
      16d206ee
  25. 22 Mar, 2022 1 commit
  26. 09 Mar, 2022 1 commit
  27. 28 Feb, 2022 1 commit
  28. 14 Feb, 2022 1 commit
    • Yossi Gottlieb's avatar
      Squashed 'deps/hiredis/' changes from 00272d669..f8de9a4bd · 418de21d
      Yossi Gottlieb authored
      f8de9a4bd Merge pull request #1046 from redis/rockylinux-ci
      a41c9bc8b CentOS 8 is EOL, switch to RockyLinux
      be41ed60d Avoid incorrect call to the previous reply's callback (#1040)
      f2e8010d9 fix building on AIX and SunOS (#1031)
      e73ab2f23 Add timeout support for libuv adapter (#1016)
      f2ce5980e Allow sending commands after sending an unsubscribe (#1036)
      ff860e55d Correction for command timeout during pubsub (#1038)
      24d534493 CMakeLists.txt: allow building without a C++ compiler (#872)
      4ece9a02e Fix adapters/libevent.h compilation for 64-bit Windows (#937)
      799edfaad Don't link with crypto libs if USE_SSL isn't set.
      f74b08182 Makefile: move SSL options into a block and refine rules
      f347743b7 Update CMakeLists.txt for more portability (#1005)
      f2be74802 Fix integer overflow when format command larger than 4GB (#1030)
      58aacdac6 Handle array response in parallell with pubsub using RESP3 (#1014)
      d3384260e Support PING while subscribing (RESP2) (#1027)
      e3a479e40 FreeBSD build fixes + CI (#1026)
      da5a4ff36 Add asynchronous test for pubsub using RESP3 (#1012)
      b5716ee82 Valgrind returns error exit code when errors found (#1011)
      1aed21a8c Move to using make directly in Cygwin (#1020)
      a83f4b890 Correct CMake warning for libevent adapter example
      c4333203e Remove unused parameter warning in libev adapter
      7ad38dc4a Small tweaks of the async tests
      4021726a6 Add asynchronous test for pubsub using RESP2
      648763c36 Add build options for enabling async tests
      c98c6994d Correcting the build target `coverage` for enabled SSL (#1009)
      30ff8d850 Run SSL tests in CI
      4a126e8a9 Add valgrind and CMake to tests
      b73c2d410 Add Centos8
      e9f647384 We should run actions on PRs
      6ad4ccf3c Add Cygwin build test
      783a3789c Add Windows tests in GitHub actions
      0cac8dae1 Switch to GitHub actions
      fa900ef76 Fix unused variable warning.
      e489846b7 Minor refactor of CVE-2021-32765 fix.
      51c740824 Remove extra comma from cmake var. Or it'll be treated as part of the var name.
      632bf0718 Merge branch 'release/v1.0.2'
      b73128324 Prepare for v1.0.2 GA
      d4e6f109a Revert erroneous SONAME bump
      a39824a5d Merge branch 'release/v1.0.1'
      8d1bfac46  Prepare for v1.0.1 GA
      76a7b1000 Fix for integer/buffer overflow CVE-2021-32765
      9eca1f36f Allow to override OPENSSL_PREFIX in Linux
      2d9d77518 Don't leak memory if an invalid type is set (#906)
      f5f31ff9b Added REDIS_NO_AUTO_FREE_REPLIES flag (#962)
      5850a8ecd Ensure we curry any connect error to an async context.
      b6f86f38c Fix README.md
      667dbf536 Merge pull request #935 from kristjanvalur/pr5
      9bf6c250e Merge pull request #939 from zmartzone/improve_pr_896_ssl_leak
      959af9760 Merge pull request #949 from plan-do-break-fix/Typo-corrections
      0743f57bb fix(docs): corrects typos in project README
      5f4382247 improve SSL leak fix redis/hiredis#896
      e06ecf7e4 Ignore timeout callback from a successful connect
      dfa33e60b Change order independant push logic to not change behavior.
      6204182aa Handle the case where an invalidation is sent second.
      d6a0b192b Merge branch 'reader-updates'
      410c24d2a Fix off-by-one error in seekNewline
      bd7488d27 read: Validate line items prior to checking for object creation callbacks
      5f9242a1f read: Remove obsolete comment on nested multi bulk depth limitation
      83c145042 read: Add support for the RESP3 bignum type
      c6646cb19 read: Ensure no invalid '\r' or '\n' in simple status/error strings
      e43061156 read: Additional validation and test case for RESP3 double
      c8adea402 redisReply: Fix parent type assertions during double, nil, bool creation
      ff73f1f9e redisReply: Explicitly list nil and bool cases in freeReplyObject() switch.
      0f9251884 test: Add test case for RESP3 set
      33c06dd50 test: Add test case for RESP3 map
      397fe2630 read: Use memchr() in seekNewline() instead of looping over entire string
      81c48a982 test: Add test cases for RESP3 bool
      51e693f4f read: Add additional RESP3 bool validation
      790b4d3b4 test: Add test cases for RESP3 nil
      d8899fbc1 read: Add additional RESP3 nil validation
      96e8ea611 test: Add test cases for infinite and NaN doubles
      f913e9b99 read: Fix double validation and infinity parsing
      8039c7d26 test: Add test case for doubles
      49539fd1a redisReply: Fix - set len in double objects
      53a8144c8 Merge pull request #924 from cheese1/master
      9390de006 http -> https
      7d99b5635 Merge pull request #917 from Nordix/stack-alloc-dict-iter
      4bba72103 Handle OOM during async command callback registration
      920128a26 Stack allocate dict iterators
      297ecbecb Tiny formatting changes + suppress implicit memcpy warning
      f746a28e7 Removed 2 typecasts
      940a04f4d Added fuzzer
      e4a200040 Merge pull request #896 from ayeganov/bugfix/ssl_leak
      aefef8987 Free SSL object when redisSSLConnect fails
      e3f88ebcf Merge pull request #894 from jcohen02/fix/issue893
      308ffcab8 Updating SSL connection example
      297f6551d Merge pull request #889 from redis/wincert
      e7dda9785 Formatting
      f44945a0a Merge pull request #874 from masariello/position-independent-code
      74e78498c Merge pull request #888 from michael-grunder/nil-push-invalidation
      b9b9f446f Fix handling of NIL invalidation messages.
      acc917548 Merge pull request #885 from gkorland/patch-1
      b086f763e clean a warning, remvoe empty else block
      b47fae4e7 Merge pull request #881 from timgates42/bugfix_typo_terminated
      f989670e5 docs: Fix simple typo, termined -> terminated
      773d6ea8a Copy error to redisAsyncContext on timeout
      e35300a66 add pdb files to packages for MSVC builds
      dde6916b4 Add d suffix to debug libraries so that can packaged together with optimized builds (Release, RelWithDebInfo, etc)
      3b68b5018 Enable position-independent code
      6693863f4 Add support for system CA certificate store on Windows
      2a5a57b90 Remove whitespace
      1b40ec509 fixed issue with unit test linking on windows with SSL
      d7b1d21e8 Merge branch 'master' of github.com:redis/hiredis
      fb0e6c0dd Merge pull request #870 from michael-grunder/cmake-c99
      13a35bdb6 Explicitly set c99 in CMake
      bea137ca9 Merge pull request #868 from michael-grunder/fix-sockaddr-typo
      bd6f86eb6 Fix sockaddr typo
      48696e7e5 Don't use non-installed win32.h helper in examples (#863)
      faa1c4863 Merge tag 'v1.0.0'
      5003906d6 Define a no op assert if we detect NDEBUG (#861)
      ea063b7cc Use development specific versions in master
      04a27f480 We can run SSL tests everywhere except mingw/Windows (#859)
      8966a1fc2 Remove extra whitespace (#858)
      34b7f7a0f Keep libev's code style (#857)
      07c3618ff Add static library target and cpack support
      REVERT: 00272d669 Rename sds calls so they don't conflict in Redis.
      
      git-subtree-dir: deps/hiredis
      git-subtree-split: f8de9a4bd433791890572f7b9147e685653ddef9
      418de21d
  29. 13 Feb, 2022 1 commit
  30. 02 Feb, 2022 1 commit
  31. 29 Jan, 2022 1 commit
  32. 04 Jan, 2022 1 commit
  33. 19 Dec, 2021 2 commits
    • Oran Agra's avatar
      Add external test that runs without debug command (#9964) · 6add1b72
      Oran Agra authored
      - add needs:debug flag for some tests
      - disable "save" in external tests (speedup?)
      - use debug_digest proc instead of debug command directly so it can be skipped
      - use OBJECT ENCODING instead of DEBUG OBJECT to get encoding
      - add a proc for OBJECT REFCOUNT so it can be skipped
      - move a bunch of tests in latency_monitor tests to happen later so that latency monitor has some values in it
      - add missing close_replication_stream calls
      - make sure to close the temp client if DEBUG LOG fails
      6add1b72
    • YaacovHazan's avatar
      Protected configs and sensitive commands (#9920) · ae2f5b7b
      YaacovHazan authored
      Block sensitive configs and commands by default.
      
      * `enable-protected-configs` - block modification of configs with the new `PROTECTED_CONFIG` flag.
         Currently we add this flag to `dbfilename`, and `dir` configs,
         all of which are non-mutable configs that can set a file redis will write to.
      * `enable-debug-command` - block the `DEBUG` command
      * `enable-module-command` - block the `MODULE` command
      
      These have a default value set to `no`, so that these features are not
      exposed by default to client connections, and can only be set by modifying the config file.
      
      Users can change each of these to either `yes` (allow all access), or `local` (allow access from
      local TCP connections and unix domain connections)
      
      Note that this is a **breaking change** (specifically the part about MODULE command being disabled by default).
      I.e. we don't consider DEBUG command being blocked as an issue (people shouldn't have been using it),
      and the few configs we protected are unlikely to have been set at runtime anyway.
      On the other hand, it's likely to assume some users who use modules, load them from the config file anyway.
      Note that's the whole point of this PR, for redis to be more secure by default and reduce the attack surface on
      innocent users, so secure defaults will necessarily mean a breaking change.
      ae2f5b7b
  34. 29 Nov, 2021 1 commit
    • Binbin's avatar
      Add REDIS_CFLAGS='-Werror' to CI tests (#9828) · 980bb3ae
      Binbin authored
      Update CI so that warnings cause build failures.
      
      Also fix a warning in `test-sanitizer-address`:
      ```
      In function ‘strncpy’,
         inlined from ‘clusterUpdateMyselfIp’ at cluster.c:545:13:
      
      /usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10:
      error: ‘__builtin_strncpy’ specified bound 46 equals destination size [-Werror=stringop-truncation]
      
        106 |   return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      cc1: all warnings being treated as errors
      ```
      980bb3ae
  35. 28 Nov, 2021 1 commit
    • Meir Shpilraien (Spielrein)'s avatar
      Fix Lua C API violation on lua msgpack lib. (#9832) · a8c1253b
      Meir Shpilraien (Spielrein) authored
      msgpack lib missed using lua_checkstack and so on rare
      cases overflow the stack by at most 2 elements. This is a
      violation of the Lua C API. Notice that Lua allocates
      additional 5 more elements on top of lua->stack_last
      so Redis does not access an invalid memory. But it is an
      API violation and we should avoid it.
      
      This PR also added a new Lua compilation option. The new
      option can be enable using environment variable called
      LUA_DEBUG. If set to `yes` (by default `no`), Lua will be
      compiled without optimizations and with debug symbols (`-O0 -g`).
      In addition, in this new mode, Lua will be compiled with the
      `-DLUA_USE_APICHECK` flag that enables extended Lua C API
      validations.
      
      In addition, set LUA_DEBUG=yes on daily valgrind flow so we
      will be able to catch Lua C API violations in the future.
      a8c1253b
  36. 21 Nov, 2021 1 commit