1. 28 Feb, 2023 1 commit
    • uriyage's avatar
      Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766) · f084778c
      uriyage authored
      
      
      In #7875 (Redis 6.2), we changed the sds alloc to be the usable allocation
      size in order to:
      
      > reduce the need for realloc calls by making the sds implicitly take over
      the internal fragmentation
      
      This change was done most sds functions, excluding `sdsRemoveFreeSpace` and
      `sdsResize`, the reason is that in some places (e.g. clientsCronResizeQueryBuffer)
      we call sdsRemoveFreeSpace when we see excessive free space and want to trim it.
      so if we don't trim it exactly to size, the caller may still see excessive free space and
      call it again and again.
      
      However, this resulted in some excessive calls to realloc, even when there's no need
      and it's gonna be a no-op (e.g. when reducing 15 bytes allocation to 13).
      
      It turns out that a call for realloc with jemalloc can be expensive even if it ends up
      doing nothing, so this PR adds a check using `je_nallocx`, which is cheap to avoid
      the call for realloc.
      
      in addition to that this PR unifies sdsResize and sdsRemoveFreeSpace into common
      code. the difference between them was that sdsResize would avoid using SDS_TYPE_5,
      since it want to keep the string ready to be resized again, while sdsRemoveFreeSpace
      would permit using SDS_TYPE_5 and get an optimal memory consumption.
      now both methods take a `would_regrow` argument that makes it more explicit.
      
      the only actual impact of that is that in clientsCronResizeQueryBuffer we call both sdsResize
      and sdsRemoveFreeSpace for in different cases, and we now prevent the use of SDS_TYPE_5 in both.
      
      The new test that was added to cover this concern used to pass before this PR as well,
      this PR is just a performance optimization and cleanup.
      
      Benchmark:
      `redis-benchmark -c 100 -t set  -d 512 -P 10  -n  100000000`
      on i7-9850H with jemalloc, shows improvement from 1021k ops/sec to 1067k (average of 3 runs).
      some 4.5% improvement.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      (cherry picked from commit 46393f98)
      (cherry picked from commit b12eeccddd9318a5d97a5aee2dad88999dfad53f)
      f084778c
  2. 21 Jul, 2021 1 commit
    • Oran Agra's avatar
      Adjustments to recent RM_StringTruncate fix (#3718) (#9125) · 37b0f361
      Oran Agra authored
      - Introduce a new sdssubstr api as a building block for sdsrange.
        The API of sdsrange is many times hard to work with and also has
        corner case that cause bugs. sdsrange is easy to work with and also
        simplifies the implementation of sdsrange.
      - Revert the fix to RM_StringTruncate and just use sdssubstr instead of
        sdsrange.
      - Solve valgrind warnings from the new tests introduced by the previous
        PR.
      
      (cherry picked from commit ae418eca)
      37b0f361
  3. 10 Mar, 2021 1 commit
    • sundb's avatar
      Add run all test support with define REDIS_TEST (#8570) · 95d6297d
      sundb authored
      1. Add `redis-server test all` support to run all tests.
      2. Add redis test to daily ci.
      3. Add `--accurate` option to run slow tests for more iterations (so that
         by default we run less cycles (shorter time, and less prints).
      4. Move dict benchmark to REDIS_TEST.
      5. fix some leaks in tests
      6. make quicklist tests run on a specific fill set of options rather than huge ranges
      7. move some prints in quicklist test outside their loops to reduce prints
      8. removing sds.h from dict.c since it is now used in both redis-server and
         redis-cli (uses hiredis sds)
      95d6297d
  4. 28 Jan, 2021 1 commit
  5. 06 Dec, 2020 1 commit
  6. 21 Dec, 2019 1 commit
  7. 29 Dec, 2017 1 commit
    • Oran Agra's avatar
      fix processing of large bulks (above 2GB) · 60a4f12f
      Oran Agra authored
      - protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
        readQueryFromClient potential overflow
      - rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
      - several places in sds.c that used int for string length or index.
      - bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
      - RM_SaveStringBuffer was limitted to 32bit length
      60a4f12f
  8. 23 Feb, 2017 1 commit
  9. 17 Nov, 2015 1 commit
    • antirez's avatar
      Lua debugger: use sds_malloc() to allocate eval cli array. · e57cccde
      antirez authored
      Redis-cli handles the debugger "eval" command in a special way since
      sdssplitargs() would not be ok: we need to send the Redis debugger the
      whole Lua script without any parsing. However in order to later free the
      argument vector inside redis-cli using just sdsfreesplitres(), we need
      to allocate the array of SDS pointers using the same allocator SDS is
      using, that may differ to what Redis is using.
      
      So now a newer version of SDS exports sds_malloc() and other allocator
      functions to give access, to the program it is linked to, the allocator
      used internally by SDS.
      e57cccde
  10. 25 Jul, 2015 4 commits
  11. 15 Jul, 2015 1 commit
    • antirez's avatar
      SDS: New sds type 5 implemented. · 0ab27a45
      antirez authored
      This is an attempt to use the refcount feature of the sds.c fork
      provided in the Pull Request #2509. A new type, SDS_TYPE_5 is introduced
      having a one byte header with just the string length, without
      information about the available additional length at the end of the
      string (this means that sdsMakeRoomFor() will be required each time
      we want to append something, since the string will always report to have
      0 bytes available).
      
      More work needed in order to avoid common SDS functions will pay the
      cost of this type. For example both sdscatprintf() and sdscatfmt()
      should try to upgrade to SDS_TYPE_8 ASAP when appending chars.
      0ab27a45
  12. 14 Jul, 2015 1 commit
  13. 08 Jan, 2015 1 commit
  14. 02 Jan, 2015 1 commit
    • Matt Stancliff's avatar
      Add sdsnative() · e1619772
      Matt Stancliff authored
      Use the existing memory space for an SDS to convert it to a regular
      character buffer so we don't need to allocate duplicate space just
      to extract a usable buffer for native operations.
      e1619772
  15. 23 Dec, 2014 1 commit
    • Matt Stancliff's avatar
      Allow all code tests to run using Redis args · 8febcffd
      Matt Stancliff authored
      Previously, many files had individual main() functions for testing,
      but each required being compiled with their own testing flags.
      That gets difficult when you have 8 different flags you need
      to set just to run all tests (plus, some test files required
      other files to be compiled aaginst them, and it seems some didn't
      build at all without including the rest of Redis).
      
      Now all individual test main() funcions are renamed to a test
      function for the file itself and one global REDIS_TEST define enables
      testing across the entire codebase.
      
      Tests can now be run with:
        - `./redis-server test <test>`
      
        e.g. ./redis-server test ziplist
      
      If REDIS_TEST is not defined, then no tests get included and no
      tests are included in the final redis-server binary.
      8febcffd
  16. 11 Dec, 2014 2 commits
  17. 13 Aug, 2014 1 commit
  18. 28 Apr, 2014 1 commit
  19. 12 Aug, 2013 1 commit
  20. 24 Jul, 2013 1 commit
    • antirez's avatar
      sdsrange() does not need to return a value. · 6ea8e094
      antirez authored
      Actaully the string is modified in-place and a reallocation is never
      needed, so there is no need to return the new sds string pointer as
      return value of the function, that is now just "void".
      6ea8e094
  21. 04 Jul, 2013 1 commit
  22. 06 Mar, 2013 1 commit
  23. 30 Mar, 2012 1 commit
  24. 14 Mar, 2012 1 commit
  25. 16 Jan, 2012 1 commit
  26. 21 Nov, 2011 1 commit
  27. 02 Nov, 2011 1 commit
  28. 13 Sep, 2011 1 commit
  29. 25 May, 2011 1 commit
  30. 05 May, 2011 1 commit
  31. 07 Apr, 2011 1 commit
  32. 10 Dec, 2010 2 commits
  33. 09 Dec, 2010 1 commit
  34. 02 Sep, 2010 1 commit
  35. 05 Aug, 2010 1 commit