Commit 75addb4f authored by Oran Agra's avatar Oran Agra Committed by antirez
Browse files

optimize memory usage of deferred replies - fixed

When deffered reply is added the previous reply node cannot be used so
all the extra space we allocated in it is wasted. in case someone uses
deffered replies in a loop, each time adding a small reply, each of
these reply nodes (the small string reply) would have consumed a 16k
block.
now when we add anther diferred reply node, we trim the unused portion
of the previous reply block.

see #7123

cherry picked from commit fb732f7a
with fix to handle a crash with LIBC allocator, which apparently can
return the same pointer despite changing it's size.
i.e. shrinking an allocation of 16k into 56 bytes without changing the
pointer.
parent c57d9146
...@@ -436,6 +436,34 @@ void addReplyStatusFormat(client *c, const char *fmt, ...) { ...@@ -436,6 +436,34 @@ void addReplyStatusFormat(client *c, const char *fmt, ...) {
sdsfree(s); sdsfree(s);
} }
/* Sometimes we are forced to create a new reply node, and we can't append to
* the previous one, when that happens, we wanna try to trim the unused space
* at the end of the last reply node which we won't use anymore. */
void trimReplyUnusedTailSpace(client *c) {
listNode *ln = listLast(c->reply);
clientReplyBlock *tail = ln? listNodeValue(ln): NULL;
/* Note that 'tail' may be NULL even if we have a tail node, becuase when
* addDeferredMultiBulkLength() is used */
if (!tail) return;
/* We only try to trim the space is relatively high (more than a 1/4 of the
* allocation), otherwise there's a high chance realloc will NOP.
* Also, to avoid large memmove which happens as part of realloc, we only do
* that if the used part is small. */
if (tail->size - tail->used > tail->size / 4 &&
tail->used < PROTO_REPLY_CHUNK_BYTES)
{
size_t old_size = tail->size;
tail = zrealloc(tail, tail->used + sizeof(clientReplyBlock));
/* take over the allocation's internal fragmentation (at least for
* memory usage tracking) */
tail->size = zmalloc_usable(tail) - sizeof(clientReplyBlock);
c->reply_bytes += tail->size - old_size;
listNodeValue(ln) = tail;
}
}
/* Adds an empty object to the reply list that will contain the multi bulk /* Adds an empty object to the reply list that will contain the multi bulk
* length, which is not known when this function is called. */ * length, which is not known when this function is called. */
void *addReplyDeferredLen(client *c) { void *addReplyDeferredLen(client *c) {
...@@ -443,6 +471,7 @@ void *addReplyDeferredLen(client *c) { ...@@ -443,6 +471,7 @@ void *addReplyDeferredLen(client *c) {
* ready to be sent, since we are sure that before returning to the * ready to be sent, since we are sure that before returning to the
* event loop setDeferredAggregateLen() will be called. */ * event loop setDeferredAggregateLen() will be called. */
if (prepareClientToWrite(c) != C_OK) return NULL; if (prepareClientToWrite(c) != C_OK) return NULL;
trimReplyUnusedTailSpace(c);
listAddNodeTail(c->reply,NULL); /* NULL is our placeholder. */ listAddNodeTail(c->reply,NULL); /* NULL is our placeholder. */
return listLast(c->reply); return listLast(c->reply);
} }
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment