1. 01 May, 2023 1 commit
    • Oran Agra's avatar
      Upgrade to jemalloc 5.3.0 · 0897c8af
      Oran Agra authored
      * Regenerate configure script sccording to deps/README
      * update iget_defrag_hint by following changes to arena_dalloc_no_tcache
      0897c8af
  2. 21 Nov, 2021 1 commit
    • Oran Agra's avatar
      Improve active defrag in jemalloc 5.2 (#9778) · d4e7ffb3
      Oran Agra authored
      Background:
      Following the upgrade to jemalloc 5.2, there was a test that used to be flaky and
      started failing consistently (on 32bit), so we disabled it ​(see #9645).
      
      This is a test that i introduced in #7289 when i attempted to solve a rare stagnation
      problem, and it later turned out i failed to solve it, ans what's more i added a test that
      caused it to be not so rare, and as i mentioned, now in jemalloc 5.2 it became consistent on 32bit.
      
      Stagnation can happen when all the slabs of the bin are equally utilized, so the decision
      to move an allocation from a relatively empty slab to a relatively full one, will never
      happen, and in that test all the slabs are at 50% utilization, so the defragger could just
      keep scanning the keyspace and not move anything.
      
      What this PR changes:
      * First, finally in jemalloc 5.2 we have the count of non-full slabs, so when we compare
        the utilization of the current slab, we can compare it to the average utilization of the non-full
        slabs in our bin, instead of the total average of our bin. this takes the full slabs out of the game,
        since they're not candidates for migration (neither source nor target).
      * Secondly, We add some 12% (100/8) to the decision to defrag an allocation, this is the part
        that aims to avoid stagnation, and it's especially important since the above mentioned change
        can get us closer to stagnation.
      * Thirdly, since jemalloc 5.2 adds sharded bins, we take into account all shards (something
        that's missing from the original PR that merged it), this isn't expected to make any difference
        since anyway there should be just one shard.
      
      How this was benchmarked.
      What i did was run the memefficiency test unit with `--verbose` and compare the defragger hits
      and misses the tests reported.
      At first, when i took into consideration only the non-full slabs, it got a lot worse (i got into
      stagnation, or just got a lot of misses and a lot of hits), but when i added the 10% i got back
      to results that were slightly better than the ones of the jemalloc 5.1 branch. i.e. full defragmentation
      was achieved with fewer hits (relocations), and fewer misses (keyspace scans).
      d4e7ffb3
  3. 12 Oct, 2021 2 commits
    • Oran Agra's avatar
      Resolve nonsense static analysis warnings · ed92a3e8
      Oran Agra authored
      ed92a3e8
    • Oran Agra's avatar
      fix a rare active defrag edge case bug leading to stagnation · c6a26519
      Oran Agra authored
      There's a rare case which leads to stagnation in the defragger, causing
      it to keep scanning the keyspace and do nothing (not moving any
      allocation), this happens when all the allocator slabs of a certain bin
      have the same % utilization, but the slab from which new allocations are
      made have a lower utilization.
      
      this commit fixes it by removing the current slab from the overall
      average utilization of the bin, and also eliminate any precision loss in
      the utilization calculation and move the decision about the defrag to
      reside inside jemalloc.
      
      and also add a test that consistently reproduce this issue.
      c6a26519
  4. 10 Oct, 2021 4 commits
  5. 03 May, 2021 1 commit
  6. 20 May, 2020 1 commit
    • Oran Agra's avatar
      fix a rare active defrag edge case bug leading to stagnation · 88d71f47
      Oran Agra authored
      There's a rare case which leads to stagnation in the defragger, causing
      it to keep scanning the keyspace and do nothing (not moving any
      allocation), this happens when all the allocator slabs of a certain bin
      have the same % utilization, but the slab from which new allocations are
      made have a lower utilization.
      
      this commit fixes it by removing the current slab from the overall
      average utilization of the bin, and also eliminate any precision loss in
      the utilization calculation and move the decision about the defrag to
      reside inside jemalloc.
      
      and also add a test that consistently reproduce this issue.
      88d71f47
  7. 11 Jul, 2018 1 commit
  8. 27 Jun, 2018 1 commit
  9. 24 May, 2018 1 commit
  10. 22 Apr, 2017 1 commit
  11. 30 Jan, 2017 1 commit
    • antirez's avatar
      Jemalloc updated to 4.4.0. · 27e29f4f
      antirez authored
      The original jemalloc source tree was modified to:
      
      1. Remove the configure error that prevents nested builds.
      2. Insert the Redis private Jemalloc API in order to allow the
      Redis fragmentation function to work.
      27e29f4f
  12. 10 Jan, 2017 1 commit
  13. 06 Oct, 2015 1 commit
  14. 24 Jul, 2015 1 commit
    • antirez's avatar
      Jemalloc: use LG_QUANTUM of 3 for AMD64 and I386. · 6b836b6b
      antirez authored
      This gives us a 24 bytes size class which is dict.c dictEntry size, thus
      improving the memory efficiency of Redis significantly.
      Moreover other non 16 bytes aligned tiny classes are added that further
      reduce the fragmentation of the allocator.
      
      Technically speaking LG_QUANTUM should be 4 on i386 / AMD64 because of
      SSE types and other 16 bytes types, however we don't use those, and our
      jemalloc only targets Redis.
      
      New versions of Jemalloc will have an explicit configure switch in order
      to specify the quantum value for a platform without requiring any change
      to the Jemalloc source code: we'll switch to this system when available.
      
      This change was originally proposed by Oran Agra (@oranagra) as a change
      to the Jemalloc script to generate the size classes define. We ended
      doing it differently by changing LG_QUANTUM since it is apparently the
      supported Jemalloc method to obtain a 24 bytes size class, moreover it
      also provides us other potentially useful size classes.
      
      Related to issue #2510.
      6b836b6b
  15. 20 Jun, 2014 1 commit
  16. 28 Nov, 2012 1 commit
  17. 16 May, 2012 1 commit
  18. 23 Nov, 2011 1 commit
  19. 20 Jun, 2011 1 commit