• Oran Agra's avatar
    Set Jemalloc --disable-cache-oblivious to reduce memory overhead (#12315) · 07c14672
    Oran Agra authored
    Apparently for large size classes Jemalloc allocate some extra
    memory (can be up to 25% overhead for allocations of 16kb).
    see https://github.com/jemalloc/jemalloc/issues/1098#issuecomment-1589870476
    
    p.s. from Redis's perspective that looks like external fragmentation,
    (i.e. allocated bytes will be low, and active pages bytes will be large)
    which  can cause active-defrag to eat CPU cycles in vain.
    
    Some details about this mechanism we disable:
    ---------------------------------------------------------------
    Disabling this mechanism only affects large allocations (above 16kb)
    Not only that it isn't expected to cause any performance regressions,
    it's actually recommended, unless you have a specific workload pattern
    and hardware that benefit from this feature -- by default it's enabled and
    adds address randomization to all large buffers, by over allocating 1 page
    per large size class, and offsetting into that page to make the starting
    address of the user buffer randomized. Workloads such as scientific
    computation often handle multiple big matrixes at the same time, and the
    randomization makes sure that the cacheline level accesses don't suffer
    bad conflicts (when they all start from page-aligned addresses).
    
    However the downsize is also quite noticeable, like you observed that extra
    page per large size can cause memory overhead, plus the extra TLB entry.
    The other factor is, hardware in the last few years started doing the
    randomization at the hardware level, i.e. the address to cacheline mapping isn't
    a direct mapping anymore. So there's debate to disable the randomization by default,
    but we are still hesitant because when it matters, it could matter a lot, and having
    it enabled by default limits that worst case behavior, even though it means the
    majority of workloads suffers a regression.
    
    So in short, it's safe and offers better performance in most cases.
    07c14672
Makefile 3.14 KB