- 15 Sep, 2024 1 commit
-
-
Filipe Oliveira (Personal) authored
Fixes #8825 We're using the fast_float library[1] in our (compiled-in) floating-point fast_float_strtod implementation for faster and more portable parsing of 64 decimal strings. The single file fast_float.h is an amalgamation of the entire library, which can be (re)generated with the amalgamate.py script (from the fast_float repository) via the command: ``` python3 ./script/amalgamate.py --license=MIT > $REDIS_SRC/deps/fast_float/fast_float.h ``` [1]: https://github.com/fastfloat/fast_float The used commit from fast_float library was the one from https://github.com/fastfloat/fast_float/releases/tag/v3.10.1 --------- Co-authored-by:
fcostaoliveira <filipe@redis.com>
-
- 10 Jul, 2023 1 commit
-
-
Oran Agra authored
* Fix integer overflows due to using wrong integer size. * Add assertions / panic when overflow still happens. * Deletion of dead code to avoid need to maintain it * Some changes are not because of bugs, but rather paranoia. * Improve cmsgpack and cjson test coverage. Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com>
-
- 18 Jun, 2023 1 commit
-
-
Oran Agra authored
Apparently for large size classes Jemalloc allocate some extra memory (can be up to 25% overhead for allocations of 16kb). see https://github.com/jemalloc/jemalloc/issues/1098#issuecomment-1589870476 p.s. from Redis's perspective that looks like external fragmentation, (i.e. allocated bytes will be low, and active pages bytes will be large) which can cause active-defrag to eat CPU cycles in vain. Some details about this mechanism we disable: --------------------------------------------------------------- Disabling this mechanism only affects large allocations (above 16kb) Not only that it isn't expected to cause any performance regressions, it's actually recommended, unless you have a specific workload pattern and hardware that benefit from this feature -- by default it's enabled and adds address randomization to all large buffers, by over allocating 1 page per large size class, and offsetting into that page to make the starting address of the user buffer randomized. Workloads such as scientific computation often handle multiple big matrixes at the same time, and the randomization makes sure that the cacheline level accesses don't suffer bad conflicts (when they all start from page-aligned addresses). However the downsize is also quite noticeable, like you observed that extra page per large size can cause memory overhead, plus the extra TLB entry. The other factor is, hardware in the last few years started doing the randomization at the hardware level, i.e. the address to cacheline mapping isn't a direct mapping anymore. So there's debate to disable the randomization by default, but we are still hesitant because when it matters, it could matter a lot, and having it enabled by default limits that worst case behavior, even though it means the majority of workloads suffers a regression. So in short, it's safe and offers better performance in most cases.
-
- 01 May, 2023 1 commit
-
-
Oran Agra authored
* Regenerate configure script sccording to deps/README * update iget_defrag_hint by following changes to arena_dalloc_no_tcache
-
- 02 Feb, 2023 1 commit
-
-
Florian Weimer authored
Previously, jemalloc was explicitly configured to build in `gnu99` mode. As a result, `<stdatomic.h>` was presumed to be unavailable and never used. This commit removes explicit build flags configuration and lets `autoconf` determine the supported build flags. In addition, we also no longer build C++ jemalloc code. Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com>
-
- 15 Oct, 2022 1 commit
-
-
filipe oliveira authored
optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (#10587) All commands / use cases that heavily rely on double to a string representation conversion, (e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ), could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures 100% coverage of conversion. This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries ( fmtlib ) that use the optimized double to string conversion underneath. The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). test suite changes: Despite being compatible, in some cases it produces a different result from printf, and some tests had to be adjusted. one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000` instead of 5e+9, which sounds like a bug? In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding issues (`expr 0.8 == 0.79999999999999999`)
-
- 23 Aug, 2022 1 commit
-
-
Oran Agra authored
* Support BUILD_TLS=module to be loaded as a module via config file or command line. e.g. redis-server --loadmodule redis-tls.so * Updates to redismodule.h to allow it to be used side by side with server.h by defining REDISMODULE_CORE_MODULE * Changes to server.h, redismodule.h and module.c to avoid repeated type declarations (gcc 4.8 doesn't like these) * Add a mechanism for non-ABI neutral modules (ones who include server.h) to refuse loading if they detect not being built together with redis (release.c) * Fix wrong signature of RedisModuleDefragFunc, this could break compilation of a module, but not the ABI * Move initialization of listeners in server.c to be after loading the modules * Config TLS after initialization of listeners * Init cluster after initialization of listeners * Add TLS module to CI * Fix a test suite race conditions: Now that the listeners are initialized later, it's not sufficient to wait for the PID message in the log, we need to wait for the "Server Initialized" message. * Fix issues with moduleconfigs test as a result from start_server waiting for "Server Initialized" * Fix issues with modules/infra test as a result of an additional module present Notes about Sentinel: Sentinel can't really rely on the tls module, since it uses hiredis to initiate connections and depends on OpenSSL (won't be able to use any other connection modules for that), so it was decided that when TLS is built as a module, sentinel does not support TLS at all. This means that it keeps using redis_tls_ctx and redis_tls_client_ctx directly. Example code of config in redis-tls.so(may be use in the future): RedisModuleString *tls_cfg = NULL; void tlsInfo(RedisModuleInfoCtx *ctx, int for_crash_report) { UNUSED(for_crash_report); RedisModule_InfoAddSection(ctx, ""); RedisModule_InfoAddFieldLongLong(ctx, "var", 42); } int tlsCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { if (argc != 2) return RedisModule_WrongArity(ctx); return RedisModule_ReplyWithString(ctx, argv[1]); } RedisModuleString *getStringConfigCommand(const char *name, void *privdata) { REDISMODULE_NOT_USED(name); REDISMODULE_NOT_USED(privdata); return tls_cfg; } int setStringConfigCommand(const char *name, RedisModuleString *new, void *privdata, RedisModuleString **err) { REDISMODULE_NOT_USED(name); REDISMODULE_NOT_USED(err); REDISMODULE_NOT_USED(privdata); if (tls_cfg) RedisModule_FreeString(NULL, tls_cfg); RedisModule_RetainString(NULL, new); tls_cfg = new; return REDISMODULE_OK; } int RedisModule_OnLoad(void *ctx, RedisModuleString **argv, int argc) { .... if (RedisModule_CreateCommand(ctx,"tls",tlsCommand,"",0,0,0) == REDISMODULE_ERR) return REDISMODULE_ERR; if (RedisModule_RegisterStringConfig(ctx, "cfg", "", REDISMODULE_CONFIG_DEFAULT, getStringConfigCommand, setStringConfigCommand, NULL, NULL) == REDISMODULE_ERR) return REDISMODULE_ERR; if (RedisModule_LoadConfigs(ctx) == REDISMODULE_ERR) { if (tls_cfg) { RedisModule_FreeString(ctx, tls_cfg); tls_cfg = NULL; } return REDISMODULE_ERR; } ... } Co-authored-by:
zhenwei pi <pizhenwei@bytedance.com> Signed-off-by:
zhenwei pi <pizhenwei@bytedance.com>
-
- 28 Nov, 2021 1 commit
-
-
Meir Shpilraien (Spielrein) authored
msgpack lib missed using lua_checkstack and so on rare cases overflow the stack by at most 2 elements. This is a violation of the Lua C API. Notice that Lua allocates additional 5 more elements on top of lua->stack_last so Redis does not access an invalid memory. But it is an API violation and we should avoid it. This PR also added a new Lua compilation option. The new option can be enable using environment variable called LUA_DEBUG. If set to `yes` (by default `no`), Lua will be compiled without optimizations and with debug symbols (`-O0 -g`). In addition, in this new mode, Lua will be compiled with the `-DLUA_USE_APICHECK` flag that enables extended Lua C API validations. In addition, set LUA_DEBUG=yes on daily valgrind flow so we will be able to catch Lua C API violations in the future.
-
- 24 Oct, 2021 1 commit
-
-
Yossi Gottlieb authored
-
- 10 Oct, 2021 1 commit
-
-
Yoav Steinberg authored
./autogen.sh --with-version=5.2.1-0-g0
-
- 29 Sep, 2020 1 commit
-
-
YoongHM authored
- The argument `u` in for `ar` is ignored (and generates warnings since `D` became the default. All it does is avoid updating unchanged objects (shouldn't have any impact on our build) - Enable `LUA_USE_MKSTEMP` to force the use of `mkstemp()` instead of `tmpname()` (which is dead code in redis anyway). - Remove unused variable `c` in `f_parser()` - Removed misleadingly indented space in `luaL_loadfile()` and ``addfield()` Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 21 Sep, 2020 1 commit
-
-
YoongHM authored
jemalloc configure shows this: configure: WARNING: unrecognized options: --enable-cc-silence The changelog of jemalloc 4.0 has: - Replace --enable-cc-silence with --disable-cc-silence to suppress spurious warnings by default.
-
- 25 Aug, 2020 1 commit
-
-
filipe oliveira authored
A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
-
- 07 Oct, 2019 1 commit
-
-
Yossi Gottlieb authored
* Introduce a connection abstraction layer for all socket operations and integrate it across the code base. * Provide an optional TLS connections implementation based on OpenSSL. * Pull a newer version of hiredis with TLS support. * Tests, redis-cli updates for TLS support.
-
- 24 May, 2018 1 commit
-
-
antirez authored
-
- 06 Jul, 2016 1 commit
-
-
antirez authored
They were under /deps since they originate from a different source tree, however at this point they are very modified and we took ownership of both the files making changes, fixing bugs, so there is no upgrade path from the original code tree. Given that, better to move the code under /src with proper dependencies and with a more simpler editing experience.
-
- 18 May, 2016 1 commit
-
-
antirez authored
This change is documented in deps/README.md but was lost in one way or the other, neutralizing the benefits of 24 bytes size classes (and others). Close #3208.
-
- 22 Jun, 2015 1 commit
-
-
Matt Stancliff authored
Current todo: - replace functions in zset.{c,h} with a new unified Redis zset access API. Once we get the zset interface fixed, we can squash relevant commits in this branch and have one nice commit to merge into unstable. This commit adds: - Geo commands - Tests; runnable with: ./runtest --single unit/geo - Geo helpers in deps/geohash-int/ - src/geo.{c,h} and src/geojson.{c,h} implementing geo commands - Updated build configurations to get everything working - TEMPORARY: src/zset.{c,h} implementing zset score and zset range reading without writing to client output buffers. - Modified linkage of one t_zset.c function for use in zset.c Conflicts: src/Makefile src/redis.c
-
- 02 Jan, 2015 1 commit
-
-
Matt Stancliff authored
This also defines REDIS_STATIC='' for building everything inside src/ and everything inside deps/lua/.
-
- 14 Nov, 2014 1 commit
-
-
Matt Stancliff authored
The new cjson has some improvements over our current version including increased platform compatability, a new resource limit to restrict decode depth, and better invalid number handling. One minor change was required to deps/Makefile because this version of cjson doesn't export itself globally, so we added a quick little define of -DENABLE_CJSON_GLOBAL. cjson now has an optional higher performing float parsing interface, but we are not including it (g_fmt.c, dtoa.c) because it requires endianness declaration during compile time. This commit is exactly lua_cjson.c from 2.1.0 with one minor change of altering the two Lua includes for local search instead of system-wide importing.
-
- 12 Aug, 2014 1 commit
-
-
Daniel Price authored
Closes #997
-
- 16 Mar, 2013 1 commit
-
-
Johan Bergström authored
-
- 01 Nov, 2012 1 commit
-
-
antirez authored
It failed because of the way jemalloc was compiled (without passing the right flags to make, but just to configure). Now the same set of flags are also passed to the make command, fixing the issue. This fixes issue #744
-
- 11 Apr, 2012 1 commit
-
-
Pieter Noordhuis authored
Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
-
- 15 Nov, 2011 1 commit
-
-
Pieter Noordhuis authored
This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
-