1. 11 Feb, 2024 1 commit
    • debing.sun's avatar
      Fix the failure of defrag test under 32-bit (#13013) · 676f27ac
      debing.sun authored
      Fail CI:
      https://github.com/redis/redis/actions/runs/7837608438/job/21387609715
      
      
      
      ## Why defragment tests only failed under 32-bit
      
      First of all, under 32-bit jemalloc will allocate more small bins and
      less large bins, which will also lead to more external fragmentation,
      therefore, the fragmentation ratio is higher in 32-bit than in 64-bit,
      so the defragment tests(`Active defrag eval scripts: cluster` and
      `Active defrag big keys: cluster`) always fails in 32-bit.
      
      ## Why defragment tests only failed with cluster
      The fowllowing is the result of `Active defrag eval scripts: cluster`
      test.
      
      1) Before #11695, the fragmentation ratio is 3.11%.
      
      2) After #11695, the fragmentation ratio grew to 4.58%.
      Since we are using per-slot dictionary to manage slots, we will only
      defragment the contents of these dictionaries (keys, values), but not
      the dictionaries' struct and ht_table, which means that frequent
      shrinking and expanding of the dictionaries, will make more fragments.
      
      3) After #12850 and #12948, In cluster mode, a large number of cluster
      slot dicts will be shrunk, creating additional fragmention, and the
      dictionary will not be defragged.
      
      ## Solution
      * Add defragmentation of the per-slot dictionary's own structures, dict
      struct and ht_table.
      
      ## Other change
      * Increase floating point print precision of `frags` and `rss` in debug
      logs for defrag
      
      ---------
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      676f27ac
  2. 07 Feb, 2024 1 commit
  3. 06 Feb, 2024 1 commit
  4. 05 Feb, 2024 1 commit
    • guybe7's avatar
      Refactor the per-slot dict-array db.c into a new kvstore data structure (#12822) · 8cd62f82
      guybe7 authored
      # Description
      Gather most of the scattered `redisDb`-related code from the per-slot
      dict PR (#11695) and turn it to a new data structure, `kvstore`. i.e.
      it's a class that represents an array of dictionaries.
      
      # Motivation
      The main motivation is code cleanliness, the idea of using an array of
      dictionaries is very well-suited to becoming a self-contained data
      structure.
      This allowed cleaning some ugly code, among others: loops that run twice
      on the main dict and expires dict, and duplicate code for allocating and
      releasing this data structure.
      
      # Notes
      1. This PR reverts the part of https://github.com/redis/redis/pull/12848
      where the `rehashing` list is global (handling rehashing `dict`s is
      under the responsibility of `kvstore`, and should not be managed by the
      server)
      2. This PR also replaces the type of `server.pubsubshard_channels` from
      `dict**` to `kvstore` (original PR:
      https://github.com/redis/redis/pull/12804). After that was done,
      server.pubsub_channels was also chosen to be a `kvstore` (with only one
      `dict`, which seems odd) just to make the code cleaner by making it the
      same type as `server.pubsubshard_channels`, see
      `pubsubtype.serverPubSubChannels`
      3. the keys and expires kvstores are currenlty configured to allocate
      the individual dicts only when the first key is added (unlike before, in
      which they allocated them in advance), but they won't release them when
      the last key is deleted.
      
      Worth mentioning that due to the recent change the reply of DEBUG
      HTSTATS changed, in case no keys were ever added to the db.
      
      before:
      ```
      127.0.0.1:6379> DEBUG htstats 9
      [Dictionary HT]
      Hash table 0 stats (main hash table):
      No stats available for empty dictionaries
      [Expires HT]
      Hash table 0 stats (main hash table):
      No stats available for empty dictionaries
      ```
      
      after:
      ```
      127.0.0.1:6379> DEBUG htstats 9
      [Dictionary HT]
      [Expires HT]
      ```
      8cd62f82