Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ruanhaishen
redis
Commits
8b511219
Commit
8b511219
authored
Dec 05, 2017
by
Itamar Haber
Browse files
Merge remote-tracking branch 'upstream/unstable' into help_subcommands
parents
51eb6cb3
62a4b817
Changes
45
Hide whitespace changes
Inline
Side-by-side
README.md
View file @
8b511219
This README is just a fast
*quick start*
document. You can find more detailed documentation at http://redis.io.
This README is just a fast
*quick start*
document. You can find more detailed documentation at
[
redis.io
](
http
s
://redis.io
)
.
What is Redis?
What is Redis?
--------------
--------------
...
...
redis.conf
View file @
8b511219
...
@@ -59,7 +59,7 @@
...
@@ -59,7 +59,7 @@
# internet, binding to all the interfaces is dangerous and will expose the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# following bind directive, that will force Redis to listen only into
# the IPv4 loo
k
back interface address (this means Redis will be able to
# the IPv4 loo
p
back interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# accept connections only from clients running into the same computer it
# is running).
# is running).
#
#
...
@@ -296,7 +296,9 @@ dir ./
...
@@ -296,7 +296,9 @@ dir ./
#
#
# 2) if slave-serve-stale-data is set to 'no' the slave will reply with
# 2) if slave-serve-stale-data is set to 'no' the slave will reply with
# an error "SYNC with master in progress" to all the kind of commands
# an error "SYNC with master in progress" to all the kind of commands
# but to INFO and SLAVEOF.
# but to INFO, SLAVEOF, AUTH, PING, SHUTDOWN, REPLCONF, ROLE, CONFIG,
# SUBSCRIBE, UNSUBSCRIBE, PSUBSCRIBE, PUNSUBSCRIBE, PUBLISH, PUBSUB,
# COMMAND, POST, HOST: and LATENCY.
#
#
slave
-
serve
-
stale
-
data
yes
slave
-
serve
-
stale
-
data
yes
...
@@ -606,7 +608,7 @@ slave-priority 100
...
@@ -606,7 +608,7 @@ slave-priority 100
# deletion of the object. It means that the server stops processing new commands
# deletion of the object. It means that the server stops processing new commands
# in order to reclaim all the memory associated with an object in a synchronous
# in order to reclaim all the memory associated with an object in a synchronous
# way. If the key deleted is associated with a small object, the time needed
# way. If the key deleted is associated with a small object, the time needed
# in order to execute th DEL command is very small and comparable to most other
# in order to execute th
e
DEL command is very small and comparable to most other
# O(1) or O(log_N) commands in Redis. However if the key is associated with an
# O(1) or O(log_N) commands in Redis. However if the key is associated with an
# aggregated value containing millions of elements, the server can block for
# aggregated value containing millions of elements, the server can block for
# a long time (even seconds) in order to complete the operation.
# a long time (even seconds) in order to complete the operation.
...
@@ -621,7 +623,7 @@ slave-priority 100
...
@@ -621,7 +623,7 @@ slave-priority 100
# It's up to the design of the application to understand when it is a good
# It's up to the design of the application to understand when it is a good
# idea to use one or the other. However the Redis server sometimes has to
# idea to use one or the other. However the Redis server sometimes has to
# delete keys or flush the whole database as a side effect of other operations.
# delete keys or flush the whole database as a side effect of other operations.
# Specifically Redis deletes objects independently of a
n
user call in the
# Specifically Redis deletes objects independently of a user call in the
# following scenarios:
# following scenarios:
#
#
# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
...
@@ -914,7 +916,7 @@ lua-time-limit 5000
...
@@ -914,7 +916,7 @@ lua-time-limit 5000
# Docker and other containers).
# Docker and other containers).
#
#
# In order to make Redis Cluster working in such environments, a static
# In order to make Redis Cluster working in such environments, a static
# configuration where each node know
n
its public address is needed. The
# configuration where each node know
s
its public address is needed. The
# following two options are used for this scope, and are:
# following two options are used for this scope, and are:
#
#
# * cluster-announce-ip
# * cluster-announce-ip
...
...
src/Makefile
View file @
8b511219
...
@@ -144,7 +144,7 @@ endif
...
@@ -144,7 +144,7 @@ endif
REDIS_SERVER_NAME
=
redis-server
REDIS_SERVER_NAME
=
redis-server
REDIS_SENTINEL_NAME
=
redis-sentinel
REDIS_SENTINEL_NAME
=
redis-sentinel
REDIS_SERVER_OBJ
=
adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o
REDIS_SERVER_OBJ
=
adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o
t_stream.o listpack.c
REDIS_CLI_NAME
=
redis-cli
REDIS_CLI_NAME
=
redis-cli
REDIS_CLI_OBJ
=
anet.o adlist.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o
REDIS_CLI_OBJ
=
anet.o adlist.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o
REDIS_BENCHMARK_NAME
=
redis-benchmark
REDIS_BENCHMARK_NAME
=
redis-benchmark
...
...
src/adlist.c
View file @
8b511219
...
@@ -353,7 +353,7 @@ void listJoin(list *l, list *o) {
...
@@ -353,7 +353,7 @@ void listJoin(list *l, list *o) {
else
else
l
->
head
=
o
->
head
;
l
->
head
=
o
->
head
;
l
->
tail
=
o
->
tail
;
if
(
o
->
tail
)
l
->
tail
=
o
->
tail
;
l
->
len
+=
o
->
len
;
l
->
len
+=
o
->
len
;
/* Setup other as an empty list. */
/* Setup other as an empty list. */
...
...
src/anet.c
View file @
8b511219
...
@@ -237,7 +237,7 @@ int anetResolveIP(char *err, char *host, char *ipbuf, size_t ipbuf_len) {
...
@@ -237,7 +237,7 @@ int anetResolveIP(char *err, char *host, char *ipbuf, size_t ipbuf_len) {
static
int
anetSetReuseAddr
(
char
*
err
,
int
fd
)
{
static
int
anetSetReuseAddr
(
char
*
err
,
int
fd
)
{
int
yes
=
1
;
int
yes
=
1
;
/* Make sure connection-intensive things like the redis benc
k
mark
/* Make sure connection-intensive things like the redis benc
h
mark
* will be able to close/open sockets a zillion of times */
* will be able to close/open sockets a zillion of times */
if
(
setsockopt
(
fd
,
SOL_SOCKET
,
SO_REUSEADDR
,
&
yes
,
sizeof
(
yes
))
==
-
1
)
{
if
(
setsockopt
(
fd
,
SOL_SOCKET
,
SO_REUSEADDR
,
&
yes
,
sizeof
(
yes
))
==
-
1
)
{
anetSetError
(
err
,
"setsockopt SO_REUSEADDR: %s"
,
strerror
(
errno
));
anetSetError
(
err
,
"setsockopt SO_REUSEADDR: %s"
,
strerror
(
errno
));
...
...
src/aof.c
View file @
8b511219
...
@@ -1031,6 +1031,37 @@ int rewriteHashObject(rio *r, robj *key, robj *o) {
...
@@ -1031,6 +1031,37 @@ int rewriteHashObject(rio *r, robj *key, robj *o) {
return
1
;
return
1
;
}
}
/* Emit the commands needed to rebuild a stream object.
* The function returns 0 on error, 1 on success. */
int
rewriteStreamObject
(
rio
*
r
,
robj
*
key
,
robj
*
o
)
{
streamIterator
si
;
streamIteratorStart
(
&
si
,
o
->
ptr
,
NULL
,
NULL
,
0
);
streamID
id
;
int64_t
numfields
;
while
(
streamIteratorGetID
(
&
si
,
&
id
,
&
numfields
))
{
/* Emit a two elements array for each item. The first is
* the ID, the second is an array of field-value pairs. */
/* Emit the XADD <key> <id> ...fields... command. */
if
(
rioWriteBulkCount
(
r
,
'*'
,
3
+
numfields
*
2
)
==
0
)
return
0
;
if
(
rioWriteBulkString
(
r
,
"XADD"
,
4
)
==
0
)
return
0
;
if
(
rioWriteBulkObject
(
r
,
key
)
==
0
)
return
0
;
sds
replyid
=
sdscatfmt
(
sdsempty
(),
"%U.%U"
,
id
.
ms
,
id
.
seq
);
if
(
rioWriteBulkString
(
r
,
replyid
,
sdslen
(
replyid
))
==
0
)
return
0
;
sdsfree
(
replyid
);
while
(
numfields
--
)
{
unsigned
char
*
field
,
*
value
;
int64_t
field_len
,
value_len
;
streamIteratorGetField
(
&
si
,
&
field
,
&
value
,
&
field_len
,
&
value_len
);
if
(
rioWriteBulkString
(
r
,(
char
*
)
field
,
field_len
)
==
0
)
return
0
;
if
(
rioWriteBulkString
(
r
,(
char
*
)
value
,
value_len
)
==
0
)
return
0
;
}
}
streamIteratorStop
(
&
si
);
return
1
;
}
/* Call the module type callback in order to rewrite a data type
/* Call the module type callback in order to rewrite a data type
* that is exported by a module and is not handled by Redis itself.
* that is exported by a module and is not handled by Redis itself.
* The function returns 0 on error, 1 on success. */
* The function returns 0 on error, 1 on success. */
...
@@ -1111,6 +1142,8 @@ int rewriteAppendOnlyFileRio(rio *aof) {
...
@@ -1111,6 +1142,8 @@ int rewriteAppendOnlyFileRio(rio *aof) {
if
(
rewriteSortedSetObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
if
(
rewriteSortedSetObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
}
else
if
(
o
->
type
==
OBJ_HASH
)
{
}
else
if
(
o
->
type
==
OBJ_HASH
)
{
if
(
rewriteHashObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
if
(
rewriteHashObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
}
else
if
(
o
->
type
==
OBJ_STREAM
)
{
if
(
rewriteStreamObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
}
else
if
(
o
->
type
==
OBJ_MODULE
)
{
}
else
if
(
o
->
type
==
OBJ_MODULE
)
{
if
(
rewriteModuleObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
if
(
rewriteModuleObject
(
aof
,
&
key
,
o
)
==
0
)
goto
werr
;
}
else
{
}
else
{
...
...
src/blocked.c
View file @
8b511219
...
@@ -65,6 +65,8 @@
...
@@ -65,6 +65,8 @@
#include "server.h"
#include "server.h"
int
serveClientBlockedOnList
(
client
*
receiver
,
robj
*
key
,
robj
*
dstkey
,
redisDb
*
db
,
robj
*
value
,
int
where
);
/* Get a timeout value from an object and store it into 'timeout'.
/* Get a timeout value from an object and store it into 'timeout'.
* The final timeout is always stored as milliseconds as a time where the
* The final timeout is always stored as milliseconds as a time where the
* timeout will expire, however the parsing is performed according to
* timeout will expire, however the parsing is performed according to
...
@@ -100,7 +102,8 @@ int getTimeoutFromObjectOrReply(client *c, robj *object, mstime_t *timeout, int
...
@@ -100,7 +102,8 @@ int getTimeoutFromObjectOrReply(client *c, robj *object, mstime_t *timeout, int
void
blockClient
(
client
*
c
,
int
btype
)
{
void
blockClient
(
client
*
c
,
int
btype
)
{
c
->
flags
|=
CLIENT_BLOCKED
;
c
->
flags
|=
CLIENT_BLOCKED
;
c
->
btype
=
btype
;
c
->
btype
=
btype
;
server
.
bpop_blocked_clients
++
;
server
.
blocked_clients
++
;
server
.
blocked_clients_by_type
[
btype
]
++
;
}
}
/* This function is called in the beforeSleep() function of the event loop
/* This function is called in the beforeSleep() function of the event loop
...
@@ -132,7 +135,7 @@ void processUnblockedClients(void) {
...
@@ -132,7 +135,7 @@ void processUnblockedClients(void) {
/* Unblock a client calling the right function depending on the kind
/* Unblock a client calling the right function depending on the kind
* of operation the client is blocking for. */
* of operation the client is blocking for. */
void
unblockClient
(
client
*
c
)
{
void
unblockClient
(
client
*
c
)
{
if
(
c
->
btype
==
BLOCKED_LIST
)
{
if
(
c
->
btype
==
BLOCKED_LIST
||
c
->
btype
==
BLOCKED_STREAM
)
{
unblockClientWaitingData
(
c
);
unblockClientWaitingData
(
c
);
}
else
if
(
c
->
btype
==
BLOCKED_WAIT
)
{
}
else
if
(
c
->
btype
==
BLOCKED_WAIT
)
{
unblockClientWaitingReplicas
(
c
);
unblockClientWaitingReplicas
(
c
);
...
@@ -143,9 +146,10 @@ void unblockClient(client *c) {
...
@@ -143,9 +146,10 @@ void unblockClient(client *c) {
}
}
/* Clear the flags, and put the client in the unblocked list so that
/* Clear the flags, and put the client in the unblocked list so that
* we'll process new commands in its query buffer ASAP. */
* we'll process new commands in its query buffer ASAP. */
server
.
blocked_clients
--
;
server
.
blocked_clients_by_type
[
c
->
btype
]
--
;
c
->
flags
&=
~
CLIENT_BLOCKED
;
c
->
flags
&=
~
CLIENT_BLOCKED
;
c
->
btype
=
BLOCKED_NONE
;
c
->
btype
=
BLOCKED_NONE
;
server
.
bpop_blocked_clients
--
;
/* The client may already be into the unblocked list because of a previous
/* The client may already be into the unblocked list because of a previous
* blocking operation, don't add back it into the list multiple times. */
* blocking operation, don't add back it into the list multiple times. */
if
(
!
(
c
->
flags
&
CLIENT_UNBLOCKED
))
{
if
(
!
(
c
->
flags
&
CLIENT_UNBLOCKED
))
{
...
@@ -158,7 +162,7 @@ void unblockClient(client *c) {
...
@@ -158,7 +162,7 @@ void unblockClient(client *c) {
* send it a reply of some kind. After this function is called,
* send it a reply of some kind. After this function is called,
* unblockClient() will be called with the same client as argument. */
* unblockClient() will be called with the same client as argument. */
void
replyToBlockedClientTimedOut
(
client
*
c
)
{
void
replyToBlockedClientTimedOut
(
client
*
c
)
{
if
(
c
->
btype
==
BLOCKED_LIST
)
{
if
(
c
->
btype
==
BLOCKED_LIST
||
c
->
btype
==
BLOCKED_STREAM
)
{
addReply
(
c
,
shared
.
nullmultibulk
);
addReply
(
c
,
shared
.
nullmultibulk
);
}
else
if
(
c
->
btype
==
BLOCKED_WAIT
)
{
}
else
if
(
c
->
btype
==
BLOCKED_WAIT
)
{
addReplyLongLong
(
c
,
replicationCountAcksByOffset
(
c
->
bpop
.
reploffset
));
addReplyLongLong
(
c
,
replicationCountAcksByOffset
(
c
->
bpop
.
reploffset
));
...
@@ -193,3 +197,283 @@ void disconnectAllBlockedClients(void) {
...
@@ -193,3 +197,283 @@ void disconnectAllBlockedClients(void) {
}
}
}
}
}
}
/* This function should be called by Redis every time a single command,
* a MULTI/EXEC block, or a Lua script, terminated its execution after
* being called by a client.
*
* All the keys with at least one client blocked that received at least
* one new element via some PUSH/XADD operation are accumulated into
* the server.ready_keys list. This function will run the list and will
* serve clients accordingly. Note that the function will iterate again and
* again as a result of serving BRPOPLPUSH we can have new blocking clients
* to serve because of the PUSH side of BRPOPLPUSH. */
void
handleClientsBlockedOnKeys
(
void
)
{
while
(
listLength
(
server
.
ready_keys
)
!=
0
)
{
list
*
l
;
/* Point server.ready_keys to a fresh list and save the current one
* locally. This way as we run the old list we are free to call
* signalKeyAsReady() that may push new elements in server.ready_keys
* when handling clients blocked into BRPOPLPUSH. */
l
=
server
.
ready_keys
;
server
.
ready_keys
=
listCreate
();
while
(
listLength
(
l
)
!=
0
)
{
listNode
*
ln
=
listFirst
(
l
);
readyList
*
rl
=
ln
->
value
;
/* First of all remove this key from db->ready_keys so that
* we can safely call signalKeyAsReady() against this key. */
dictDelete
(
rl
->
db
->
ready_keys
,
rl
->
key
);
/* Serve clients blocked on list key. */
robj
*
o
=
lookupKeyWrite
(
rl
->
db
,
rl
->
key
);
if
(
o
!=
NULL
&&
o
->
type
==
OBJ_LIST
)
{
dictEntry
*
de
;
/* We serve clients in the same order they blocked for
* this key, from the first blocked to the last. */
de
=
dictFind
(
rl
->
db
->
blocking_keys
,
rl
->
key
);
if
(
de
)
{
list
*
clients
=
dictGetVal
(
de
);
int
numclients
=
listLength
(
clients
);
while
(
numclients
--
)
{
listNode
*
clientnode
=
listFirst
(
clients
);
client
*
receiver
=
clientnode
->
value
;
if
(
receiver
->
btype
!=
BLOCKED_LIST
)
{
/* Put on the tail, so that at the next call
* we'll not run into it again. */
listDelNode
(
clients
,
clientnode
);
listAddNodeTail
(
clients
,
receiver
);
continue
;
}
robj
*
dstkey
=
receiver
->
bpop
.
target
;
int
where
=
(
receiver
->
lastcmd
&&
receiver
->
lastcmd
->
proc
==
blpopCommand
)
?
LIST_HEAD
:
LIST_TAIL
;
robj
*
value
=
listTypePop
(
o
,
where
);
if
(
value
)
{
/* Protect receiver->bpop.target, that will be
* freed by the next unblockClient()
* call. */
if
(
dstkey
)
incrRefCount
(
dstkey
);
unblockClient
(
receiver
);
if
(
serveClientBlockedOnList
(
receiver
,
rl
->
key
,
dstkey
,
rl
->
db
,
value
,
where
)
==
C_ERR
)
{
/* If we failed serving the client we need
* to also undo the POP operation. */
listTypePush
(
o
,
value
,
where
);
}
if
(
dstkey
)
decrRefCount
(
dstkey
);
decrRefCount
(
value
);
}
else
{
break
;
}
}
}
if
(
listTypeLength
(
o
)
==
0
)
{
dbDelete
(
rl
->
db
,
rl
->
key
);
}
/* We don't call signalModifiedKey() as it was already called
* when an element was pushed on the list. */
}
/* Serve clients blocked on stream key. */
else
if
(
o
!=
NULL
&&
o
->
type
==
OBJ_STREAM
)
{
dictEntry
*
de
=
dictFind
(
rl
->
db
->
blocking_keys
,
rl
->
key
);
stream
*
s
=
o
->
ptr
;
/* We need to provide the new data arrived on the stream
* to all the clients that are waiting for an offset smaller
* than the current top item. */
if
(
de
)
{
list
*
clients
=
dictGetVal
(
de
);
listNode
*
ln
;
listIter
li
;
listRewind
(
clients
,
&
li
);
while
((
ln
=
listNext
(
&
li
)))
{
client
*
receiver
=
listNodeValue
(
ln
);
if
(
receiver
->
btype
!=
BLOCKED_STREAM
)
continue
;
streamID
*
gt
=
dictFetchValue
(
receiver
->
bpop
.
keys
,
rl
->
key
);
if
(
s
->
last_id
.
ms
>
gt
->
ms
||
(
s
->
last_id
.
ms
==
gt
->
ms
&&
s
->
last_id
.
seq
>
gt
->
seq
))
{
streamID
start
=
*
gt
;
start
.
seq
++
;
/* Can't overflow, it's an uint64_t */
/* Note that after we unblock the client, 'gt'
* is no longer valid, so we must do it after
* we copied the ID into the 'start' variable. */
unblockClient
(
receiver
);
/* Emit the two elements sub-array consisting of
* the name of the stream and the data we
* extracted from it. Wrapped in a single-item
* array, since we have just one key. */
addReplyMultiBulkLen
(
receiver
,
1
);
addReplyMultiBulkLen
(
receiver
,
2
);
addReplyBulk
(
receiver
,
rl
->
key
);
streamReplyWithRange
(
receiver
,
s
,
&
start
,
NULL
,
receiver
->
bpop
.
xread_count
,
0
);
}
}
}
}
/* Free this item. */
decrRefCount
(
rl
->
key
);
zfree
(
rl
);
listDelNode
(
l
,
ln
);
}
listRelease
(
l
);
/* We have the new list on place at this point. */
}
}
/* This is how the current blocking lists/streams work, we use BLPOP as
* example, but the concept is the same for other list ops and XREAD.
* - If the user calls BLPOP and the key exists and contains a non empty list
* then LPOP is called instead. So BLPOP is semantically the same as LPOP
* if blocking is not required.
* - If instead BLPOP is called and the key does not exists or the list is
* empty we need to block. In order to do so we remove the notification for
* new data to read in the client socket (so that we'll not serve new
* requests if the blocking request is not served). Also we put the client
* in a dictionary (db->blocking_keys) mapping keys to a list of clients
* blocking for this keys.
* - If a PUSH operation against a key with blocked clients waiting is
* performed, we mark this key as "ready", and after the current command,
* MULTI/EXEC block, or script, is executed, we serve all the clients waiting
* for this list, from the one that blocked first, to the last, accordingly
* to the number of elements we have in the ready list.
*/
/* Set a client in blocking mode for the specified key (list or stream), with
* the specified timeout. The 'type' argument is BLOCKED_LIST or BLOCKED_STREAM
* depending on the kind of operation we are waiting for an empty key in
* order to awake the client. The client is blocked for all the 'numkeys'
* keys as in the 'keys' argument. When we block for stream keys, we also
* provide an array of streamID structures: clients will be unblocked only
* when items with an ID greater or equal to the specified one is appended
* to the stream. */
void
blockForKeys
(
client
*
c
,
int
btype
,
robj
**
keys
,
int
numkeys
,
mstime_t
timeout
,
robj
*
target
,
streamID
*
ids
)
{
dictEntry
*
de
;
list
*
l
;
int
j
;
c
->
bpop
.
timeout
=
timeout
;
c
->
bpop
.
target
=
target
;
if
(
target
!=
NULL
)
incrRefCount
(
target
);
for
(
j
=
0
;
j
<
numkeys
;
j
++
)
{
/* The value associated with the key name in the bpop.keys dictionary
* is NULL for lists, or the stream ID for streams. */
void
*
key_data
=
NULL
;
if
(
btype
==
BLOCKED_STREAM
)
{
key_data
=
zmalloc
(
sizeof
(
streamID
));
memcpy
(
key_data
,
ids
+
j
,
sizeof
(
streamID
));
}
/* If the key already exists in the dictionary ignore it. */
if
(
dictAdd
(
c
->
bpop
.
keys
,
keys
[
j
],
key_data
)
!=
DICT_OK
)
{
zfree
(
key_data
);
continue
;
}
incrRefCount
(
keys
[
j
]);
/* And in the other "side", to map keys -> clients */
de
=
dictFind
(
c
->
db
->
blocking_keys
,
keys
[
j
]);
if
(
de
==
NULL
)
{
int
retval
;
/* For every key we take a list of clients blocked for it */
l
=
listCreate
();
retval
=
dictAdd
(
c
->
db
->
blocking_keys
,
keys
[
j
],
l
);
incrRefCount
(
keys
[
j
]);
serverAssertWithInfo
(
c
,
keys
[
j
],
retval
==
DICT_OK
);
}
else
{
l
=
dictGetVal
(
de
);
}
listAddNodeTail
(
l
,
c
);
}
blockClient
(
c
,
btype
);
}
/* Unblock a client that's waiting in a blocking operation such as BLPOP.
* You should never call this function directly, but unblockClient() instead. */
void
unblockClientWaitingData
(
client
*
c
)
{
dictEntry
*
de
;
dictIterator
*
di
;
list
*
l
;
serverAssertWithInfo
(
c
,
NULL
,
dictSize
(
c
->
bpop
.
keys
)
!=
0
);
di
=
dictGetIterator
(
c
->
bpop
.
keys
);
/* The client may wait for multiple keys, so unblock it for every key. */
while
((
de
=
dictNext
(
di
))
!=
NULL
)
{
robj
*
key
=
dictGetKey
(
de
);
/* Remove this client from the list of clients waiting for this key. */
l
=
dictFetchValue
(
c
->
db
->
blocking_keys
,
key
);
serverAssertWithInfo
(
c
,
key
,
l
!=
NULL
);
listDelNode
(
l
,
listSearchKey
(
l
,
c
));
/* If the list is empty we need to remove it to avoid wasting memory */
if
(
listLength
(
l
)
==
0
)
dictDelete
(
c
->
db
->
blocking_keys
,
key
);
}
dictReleaseIterator
(
di
);
/* Cleanup the client structure */
dictEmpty
(
c
->
bpop
.
keys
,
NULL
);
if
(
c
->
bpop
.
target
)
{
decrRefCount
(
c
->
bpop
.
target
);
c
->
bpop
.
target
=
NULL
;
}
if
(
c
->
bpop
.
xread_group
)
{
decrRefCount
(
c
->
bpop
.
xread_group
);
c
->
bpop
.
xread_group
=
NULL
;
}
}
/* If the specified key has clients blocked waiting for list pushes, this
* function will put the key reference into the server.ready_keys list.
* Note that db->ready_keys is a hash table that allows us to avoid putting
* the same key again and again in the list in case of multiple pushes
* made by a script or in the context of MULTI/EXEC.
*
* The list will be finally processed by handleClientsBlockedOnLists() */
void
signalKeyAsReady
(
redisDb
*
db
,
robj
*
key
)
{
readyList
*
rl
;
/* No clients blocking for this key? No need to queue it. */
if
(
dictFind
(
db
->
blocking_keys
,
key
)
==
NULL
)
return
;
/* Key was already signaled? No need to queue it again. */
if
(
dictFind
(
db
->
ready_keys
,
key
)
!=
NULL
)
return
;
/* Ok, we need to queue this key into server.ready_keys. */
rl
=
zmalloc
(
sizeof
(
*
rl
));
rl
->
key
=
key
;
rl
->
db
=
db
;
incrRefCount
(
key
);
listAddNodeTail
(
server
.
ready_keys
,
rl
);
/* We also add the key in the db->ready_keys dictionary in order
* to avoid adding it multiple times into a list with a simple O(1)
* check. */
incrRefCount
(
key
);
serverAssert
(
dictAdd
(
db
->
ready_keys
,
key
,
NULL
)
==
DICT_OK
);
}
src/cluster.c
View file @
8b511219
...
@@ -653,7 +653,7 @@ unsigned int keyHashSlot(char *key, int keylen) {
...
@@ -653,7 +653,7 @@ unsigned int keyHashSlot(char *key, int keylen) {
for
(
e
=
s
+
1
;
e
<
keylen
;
e
++
)
for
(
e
=
s
+
1
;
e
<
keylen
;
e
++
)
if
(
key
[
e
]
==
'}'
)
break
;
if
(
key
[
e
]
==
'}'
)
break
;
/* No '}' or nothing betwee
e
n {} ? Hash the whole key. */
/* No '}' or nothing between {} ? Hash the whole key. */
if
(
e
==
keylen
||
e
==
s
+
1
)
return
crc16
(
key
,
keylen
)
&
0x3FFF
;
if
(
e
==
keylen
||
e
==
s
+
1
)
return
crc16
(
key
,
keylen
)
&
0x3FFF
;
/* If we are here there is both a { and a } on its right. Hash
/* If we are here there is both a { and a } on its right. Hash
...
...
src/config.c
View file @
8b511219
...
@@ -330,13 +330,13 @@ void loadServerConfigFromString(char *config) {
...
@@ -330,13 +330,13 @@ void loadServerConfigFromString(char *config) {
}
}
}
else
if
(
!
strcasecmp
(
argv
[
0
],
"lfu-log-factor"
)
&&
argc
==
2
)
{
}
else
if
(
!
strcasecmp
(
argv
[
0
],
"lfu-log-factor"
)
&&
argc
==
2
)
{
server
.
lfu_log_factor
=
atoi
(
argv
[
1
]);
server
.
lfu_log_factor
=
atoi
(
argv
[
1
]);
if
(
server
.
maxmemory_samples
<
0
)
{
if
(
server
.
lfu_log_factor
<
0
)
{
err
=
"lfu-log-factor must be 0 or greater"
;
err
=
"lfu-log-factor must be 0 or greater"
;
goto
loaderr
;
goto
loaderr
;
}
}
}
else
if
(
!
strcasecmp
(
argv
[
0
],
"lfu-decay-time"
)
&&
argc
==
2
)
{
}
else
if
(
!
strcasecmp
(
argv
[
0
],
"lfu-decay-time"
)
&&
argc
==
2
)
{
server
.
lfu_decay_time
=
atoi
(
argv
[
1
]);
server
.
lfu_decay_time
=
atoi
(
argv
[
1
]);
if
(
server
.
maxmemory_samples
<
1
)
{
if
(
server
.
lfu_decay_time
<
0
)
{
err
=
"lfu-decay-time must be 0 or greater"
;
err
=
"lfu-decay-time must be 0 or greater"
;
goto
loaderr
;
goto
loaderr
;
}
}
...
@@ -1221,6 +1221,8 @@ void configGetCommand(client *c) {
...
@@ -1221,6 +1221,8 @@ void configGetCommand(client *c) {
/* Numerical values */
/* Numerical values */
config_get_numerical_field
(
"maxmemory"
,
server
.
maxmemory
);
config_get_numerical_field
(
"maxmemory"
,
server
.
maxmemory
);
config_get_numerical_field
(
"maxmemory-samples"
,
server
.
maxmemory_samples
);
config_get_numerical_field
(
"maxmemory-samples"
,
server
.
maxmemory_samples
);
config_get_numerical_field
(
"lfu-log-factor"
,
server
.
lfu_log_factor
);
config_get_numerical_field
(
"lfu-decay-time"
,
server
.
lfu_decay_time
);
config_get_numerical_field
(
"timeout"
,
server
.
maxidletime
);
config_get_numerical_field
(
"timeout"
,
server
.
maxidletime
);
config_get_numerical_field
(
"active-defrag-threshold-lower"
,
server
.
active_defrag_threshold_lower
);
config_get_numerical_field
(
"active-defrag-threshold-lower"
,
server
.
active_defrag_threshold_lower
);
config_get_numerical_field
(
"active-defrag-threshold-upper"
,
server
.
active_defrag_threshold_upper
);
config_get_numerical_field
(
"active-defrag-threshold-upper"
,
server
.
active_defrag_threshold_upper
);
...
@@ -1992,6 +1994,8 @@ int rewriteConfig(char *path) {
...
@@ -1992,6 +1994,8 @@ int rewriteConfig(char *path) {
rewriteConfigBytesOption
(
state
,
"maxmemory"
,
server
.
maxmemory
,
CONFIG_DEFAULT_MAXMEMORY
);
rewriteConfigBytesOption
(
state
,
"maxmemory"
,
server
.
maxmemory
,
CONFIG_DEFAULT_MAXMEMORY
);
rewriteConfigEnumOption
(
state
,
"maxmemory-policy"
,
server
.
maxmemory_policy
,
maxmemory_policy_enum
,
CONFIG_DEFAULT_MAXMEMORY_POLICY
);
rewriteConfigEnumOption
(
state
,
"maxmemory-policy"
,
server
.
maxmemory_policy
,
maxmemory_policy_enum
,
CONFIG_DEFAULT_MAXMEMORY_POLICY
);
rewriteConfigNumericalOption
(
state
,
"maxmemory-samples"
,
server
.
maxmemory_samples
,
CONFIG_DEFAULT_MAXMEMORY_SAMPLES
);
rewriteConfigNumericalOption
(
state
,
"maxmemory-samples"
,
server
.
maxmemory_samples
,
CONFIG_DEFAULT_MAXMEMORY_SAMPLES
);
rewriteConfigNumericalOption
(
state
,
"lfu-log-factor"
,
server
.
lfu_log_factor
,
CONFIG_DEFAULT_LFU_LOG_FACTOR
);
rewriteConfigNumericalOption
(
state
,
"lfu-decay-time"
,
server
.
lfu_decay_time
,
CONFIG_DEFAULT_LFU_DECAY_TIME
);
rewriteConfigNumericalOption
(
state
,
"active-defrag-threshold-lower"
,
server
.
active_defrag_threshold_lower
,
CONFIG_DEFAULT_DEFRAG_THRESHOLD_LOWER
);
rewriteConfigNumericalOption
(
state
,
"active-defrag-threshold-lower"
,
server
.
active_defrag_threshold_lower
,
CONFIG_DEFAULT_DEFRAG_THRESHOLD_LOWER
);
rewriteConfigNumericalOption
(
state
,
"active-defrag-threshold-upper"
,
server
.
active_defrag_threshold_upper
,
CONFIG_DEFAULT_DEFRAG_THRESHOLD_UPPER
);
rewriteConfigNumericalOption
(
state
,
"active-defrag-threshold-upper"
,
server
.
active_defrag_threshold_upper
,
CONFIG_DEFAULT_DEFRAG_THRESHOLD_UPPER
);
rewriteConfigBytesOption
(
state
,
"active-defrag-ignore-bytes"
,
server
.
active_defrag_ignore_bytes
,
CONFIG_DEFAULT_DEFRAG_IGNORE_BYTES
);
rewriteConfigBytesOption
(
state
,
"active-defrag-ignore-bytes"
,
server
.
active_defrag_ignore_bytes
,
CONFIG_DEFAULT_DEFRAG_IGNORE_BYTES
);
...
...
src/db.c
View file @
8b511219
...
@@ -38,6 +38,15 @@
...
@@ -38,6 +38,15 @@
* C-level DB API
* C-level DB API
*----------------------------------------------------------------------------*/
*----------------------------------------------------------------------------*/
/* Update LFU when an object is accessed.
* Firstly, decrement the counter if the decrement time is reached.
* Then logarithmically increment the counter, and update the access time. */
void
updateLFU
(
robj
*
val
)
{
unsigned
long
counter
=
LFUDecrAndReturn
(
val
);
counter
=
LFULogIncr
(
counter
);
val
->
lru
=
(
LFUGetTimeInMinutes
()
<<
8
)
|
counter
;
}
/* Low level key lookup API, not actually called directly from commands
/* Low level key lookup API, not actually called directly from commands
* implementations that should instead rely on lookupKeyRead(),
* implementations that should instead rely on lookupKeyRead(),
* lookupKeyWrite() and lookupKeyReadWithFlags(). */
* lookupKeyWrite() and lookupKeyReadWithFlags(). */
...
@@ -54,9 +63,7 @@ robj *lookupKey(redisDb *db, robj *key, int flags) {
...
@@ -54,9 +63,7 @@ robj *lookupKey(redisDb *db, robj *key, int flags) {
!
(
flags
&
LOOKUP_NOTOUCH
))
!
(
flags
&
LOOKUP_NOTOUCH
))
{
{
if
(
server
.
maxmemory_policy
&
MAXMEMORY_FLAG_LFU
)
{
if
(
server
.
maxmemory_policy
&
MAXMEMORY_FLAG_LFU
)
{
unsigned
long
ldt
=
val
->
lru
>>
8
;
updateLFU
(
val
);
unsigned
long
counter
=
LFULogIncr
(
val
->
lru
&
255
);
val
->
lru
=
(
ldt
<<
8
)
|
counter
;
}
else
{
}
else
{
val
->
lru
=
LRU_CLOCK
();
val
->
lru
=
LRU_CLOCK
();
}
}
...
@@ -162,9 +169,9 @@ void dbAdd(redisDb *db, robj *key, robj *val) {
...
@@ -162,9 +169,9 @@ void dbAdd(redisDb *db, robj *key, robj *val) {
int
retval
=
dictAdd
(
db
->
dict
,
copy
,
val
);
int
retval
=
dictAdd
(
db
->
dict
,
copy
,
val
);
serverAssertWithInfo
(
NULL
,
key
,
retval
==
DICT_OK
);
serverAssertWithInfo
(
NULL
,
key
,
retval
==
DICT_OK
);
if
(
val
->
type
==
OBJ_LIST
)
signal
List
AsReady
(
db
,
key
);
if
(
val
->
type
==
OBJ_LIST
)
signal
Key
AsReady
(
db
,
key
);
if
(
server
.
cluster_enabled
)
slotToKeyAdd
(
key
);
if
(
server
.
cluster_enabled
)
slotToKeyAdd
(
key
);
}
}
/* Overwrite an existing key with a new value. Incrementing the reference
/* Overwrite an existing key with a new value. Incrementing the reference
* count of the new value is up to the caller.
* count of the new value is up to the caller.
...
@@ -180,6 +187,9 @@ void dbOverwrite(redisDb *db, robj *key, robj *val) {
...
@@ -180,6 +187,9 @@ void dbOverwrite(redisDb *db, robj *key, robj *val) {
int
saved_lru
=
old
->
lru
;
int
saved_lru
=
old
->
lru
;
dictReplace
(
db
->
dict
,
key
->
ptr
,
val
);
dictReplace
(
db
->
dict
,
key
->
ptr
,
val
);
val
->
lru
=
saved_lru
;
val
->
lru
=
saved_lru
;
/* LFU should be not only copied but also updated
* when a key is overwritten. */
updateLFU
(
val
);
}
else
{
}
else
{
dictReplace
(
db
->
dict
,
key
->
ptr
,
val
);
dictReplace
(
db
->
dict
,
key
->
ptr
,
val
);
}
}
...
@@ -788,6 +798,7 @@ void typeCommand(client *c) {
...
@@ -788,6 +798,7 @@ void typeCommand(client *c) {
case
OBJ_SET
:
type
=
"set"
;
break
;
case
OBJ_SET
:
type
=
"set"
;
break
;
case
OBJ_ZSET
:
type
=
"zset"
;
break
;
case
OBJ_ZSET
:
type
=
"zset"
;
break
;
case
OBJ_HASH
:
type
=
"hash"
;
break
;
case
OBJ_HASH
:
type
=
"hash"
;
break
;
case
OBJ_STREAM
:
type
=
"stream"
;
break
;
case
OBJ_MODULE
:
{
case
OBJ_MODULE
:
{
moduleValue
*
mv
=
o
->
ptr
;
moduleValue
*
mv
=
o
->
ptr
;
type
=
mv
->
type
->
name
;
type
=
mv
->
type
->
name
;
...
@@ -941,8 +952,8 @@ void scanDatabaseForReadyLists(redisDb *db) {
...
@@ -941,8 +952,8 @@ void scanDatabaseForReadyLists(redisDb *db) {
while
((
de
=
dictNext
(
di
))
!=
NULL
)
{
while
((
de
=
dictNext
(
di
))
!=
NULL
)
{
robj
*
key
=
dictGetKey
(
de
);
robj
*
key
=
dictGetKey
(
de
);
robj
*
value
=
lookupKey
(
db
,
key
,
LOOKUP_NOTOUCH
);
robj
*
value
=
lookupKey
(
db
,
key
,
LOOKUP_NOTOUCH
);
if
(
value
&&
value
->
type
==
OBJ_LIST
)
if
(
value
&&
(
value
->
type
==
OBJ_LIST
||
value
->
type
==
OBJ_STREAM
)
)
signal
List
AsReady
(
db
,
key
);
signal
Key
AsReady
(
db
,
key
);
}
}
dictReleaseIterator
(
di
);
dictReleaseIterator
(
di
);
}
}
...
@@ -1352,6 +1363,36 @@ int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numk
...
@@ -1352,6 +1363,36 @@ int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numk
return
keys
;
return
keys
;
}
}
/* XREAD [BLOCK <milliseconds>] [COUNT <count>] [GROUP <groupname> <ttl>]
* [RETRY <milliseconds> <ttl>] STREAMS key_1 key_2 ... key_N
* ID_1 ID_2 ... ID_N */
int
*
xreadGetKeys
(
struct
redisCommand
*
cmd
,
robj
**
argv
,
int
argc
,
int
*
numkeys
)
{
int
i
,
num
,
*
keys
;
UNUSED
(
cmd
);
/* We need to seek the last argument that contains "STREAMS", because other
* arguments before may contain it (for example the group name). */
int
streams_pos
=
-
1
;
for
(
i
=
1
;
i
<
argc
;
i
++
)
{
char
*
arg
=
argv
[
i
]
->
ptr
;
if
(
!
strcasecmp
(
arg
,
"streams"
))
streams_pos
=
i
;
}
if
(
streams_pos
!=
-
1
)
num
=
argc
-
streams_pos
-
1
;
/* Syntax error. */
if
(
streams_pos
==
-
1
||
num
%
2
!=
0
)
{
*
numkeys
=
0
;
return
NULL
;
}
num
/=
2
;
/* We have half the keys as there are arguments because
there are also the IDs, one per key. */
keys
=
zmalloc
(
sizeof
(
int
)
*
num
);
for
(
i
=
streams_pos
+
1
;
i
<
argc
;
i
++
)
keys
[
i
-
streams_pos
-
1
]
=
i
;
*
numkeys
=
num
;
return
keys
;
}
/* Slot to Key API. This is used by Redis Cluster in order to obtain in
/* Slot to Key API. This is used by Redis Cluster in order to obtain in
* a fast way a key that belongs to a specified hash slot. This is useful
* a fast way a key that belongs to a specified hash slot. This is useful
* while rehashing the cluster and in other conditions when we need to
* while rehashing the cluster and in other conditions when we need to
...
...
src/debug.c
View file @
8b511219
...
@@ -239,6 +239,27 @@ void computeDatasetDigest(unsigned char *final) {
...
@@ -239,6 +239,27 @@ void computeDatasetDigest(unsigned char *final) {
xorDigest
(
digest
,
eledigest
,
20
);
xorDigest
(
digest
,
eledigest
,
20
);
}
}
hashTypeReleaseIterator
(
hi
);
hashTypeReleaseIterator
(
hi
);
}
else
if
(
o
->
type
==
OBJ_STREAM
)
{
streamIterator
si
;
streamIteratorStart
(
&
si
,
o
->
ptr
,
NULL
,
NULL
,
0
);
streamID
id
;
int64_t
numfields
;
while
(
streamIteratorGetID
(
&
si
,
&
id
,
&
numfields
))
{
sds
itemid
=
sdscatfmt
(
sdsempty
(),
"%U.%U"
,
id
.
ms
,
id
.
seq
);
mixDigest
(
digest
,
itemid
,
sdslen
(
itemid
));
sdsfree
(
itemid
);
while
(
numfields
--
)
{
unsigned
char
*
field
,
*
value
;
int64_t
field_len
,
value_len
;
streamIteratorGetField
(
&
si
,
&
field
,
&
value
,
&
field_len
,
&
value_len
);
mixDigest
(
digest
,
field
,
field_len
);
mixDigest
(
digest
,
value
,
value_len
);
}
}
streamIteratorStop
(
&
si
);
}
else
if
(
o
->
type
==
OBJ_MODULE
)
{
}
else
if
(
o
->
type
==
OBJ_MODULE
)
{
RedisModuleDigest
md
;
RedisModuleDigest
md
;
moduleValue
*
mv
=
o
->
ptr
;
moduleValue
*
mv
=
o
->
ptr
;
...
@@ -262,14 +283,10 @@ void computeDatasetDigest(unsigned char *final) {
...
@@ -262,14 +283,10 @@ void computeDatasetDigest(unsigned char *final) {
}
}
void
debugCommand
(
client
*
c
)
{
void
debugCommand
(
client
*
c
)
{
if
(
c
->
argc
==
1
)
{
addReplyError
(
c
,
"You must specify a subcommand for DEBUG. Try DEBUG HELP for info."
);
return
;
}
if
(
c
->
argc
==
2
&&
!
strcasecmp
(
c
->
argv
[
1
]
->
ptr
,
"help"
))
{
if
(
c
->
argc
==
2
&&
!
strcasecmp
(
c
->
argv
[
1
]
->
ptr
,
"help"
))
{
const
char
*
help
[]
=
{
const
char
*
help
[]
=
{
"assert -- Crash by assertion failed."
,
"assert -- Crash by assertion failed."
,
"change-repl-id -- Change the replication IDs of the instance. Dangerous, should be used only for testing the replication subsystem."
,
"crash-and-recovery <milliseconds> -- Hard crash and restart after <milliseconds> delay."
,
"crash-and-recovery <milliseconds> -- Hard crash and restart after <milliseconds> delay."
,
"digest -- Outputs an hex signature representing the current DB content."
,
"digest -- Outputs an hex signature representing the current DB content."
,
"htstats <dbid> -- Return hash table statistics of the specified Redis database."
,
"htstats <dbid> -- Return hash table statistics of the specified Redis database."
,
...
@@ -351,13 +368,13 @@ void debugCommand(client *c) {
...
@@ -351,13 +368,13 @@ void debugCommand(client *c) {
val
=
dictGetVal
(
de
);
val
=
dictGetVal
(
de
);
strenc
=
strEncoding
(
val
->
encoding
);
strenc
=
strEncoding
(
val
->
encoding
);
char
extra
[
1
2
8
]
=
{
0
};
char
extra
[
1
3
8
]
=
{
0
};
if
(
val
->
encoding
==
OBJ_ENCODING_QUICKLIST
)
{
if
(
val
->
encoding
==
OBJ_ENCODING_QUICKLIST
)
{
char
*
nextra
=
extra
;
char
*
nextra
=
extra
;
int
remaining
=
sizeof
(
extra
);
int
remaining
=
sizeof
(
extra
);
quicklist
*
ql
=
val
->
ptr
;
quicklist
*
ql
=
val
->
ptr
;
/* Add number of quicklist nodes */
/* Add number of quicklist nodes */
int
used
=
snprintf
(
nextra
,
remaining
,
" ql_nodes:%u"
,
ql
->
len
);
int
used
=
snprintf
(
nextra
,
remaining
,
" ql_nodes:%
l
u"
,
ql
->
len
);
nextra
+=
used
;
nextra
+=
used
;
remaining
-=
used
;
remaining
-=
used
;
/* Add average quicklist fill factor */
/* Add average quicklist fill factor */
...
@@ -530,6 +547,11 @@ void debugCommand(client *c) {
...
@@ -530,6 +547,11 @@ void debugCommand(client *c) {
stats
=
sdscat
(
stats
,
buf
);
stats
=
sdscat
(
stats
,
buf
);
addReplyBulkSds
(
c
,
stats
);
addReplyBulkSds
(
c
,
stats
);
}
else
if
(
!
strcasecmp
(
c
->
argv
[
1
]
->
ptr
,
"change-repl-id"
)
&&
c
->
argc
==
2
)
{
serverLog
(
LL_WARNING
,
"Changing replication IDs after receiving DEBUG change-repl-id"
);
changeReplicationId
();
clearReplicationId2
();
addReply
(
c
,
shared
.
ok
);
}
else
{
}
else
{
addReplyErrorFormat
(
c
,
"Unknown subcommand or wrong number of arguments for '%s'. Try DEBUG help"
,
addReplyErrorFormat
(
c
,
"Unknown subcommand or wrong number of arguments for '%s'. Try DEBUG help"
,
(
char
*
)
c
->
argv
[
1
]
->
ptr
);
(
char
*
)
c
->
argv
[
1
]
->
ptr
);
...
...
src/defrag.c
View file @
8b511219
...
@@ -289,7 +289,7 @@ int defragKey(redisDb *db, dictEntry *de) {
...
@@ -289,7 +289,7 @@ int defragKey(redisDb *db, dictEntry *de) {
/* Dirty code:
/* Dirty code:
* I can't search in db->expires for that key after i already released
* I can't search in db->expires for that key after i already released
* the pointer it holds it won't be able to do the string compare */
* the pointer it holds it won't be able to do the string compare */
u
nsigned
in
t
hash
=
dictGetHash
(
db
->
dict
,
de
->
key
);
u
int64_
t
hash
=
dictGetHash
(
db
->
dict
,
de
->
key
);
replaceSateliteDictKeyPtrAndOrDefragDictEntry
(
db
->
expires
,
keysds
,
newsds
,
hash
,
&
defragged
);
replaceSateliteDictKeyPtrAndOrDefragDictEntry
(
db
->
expires
,
keysds
,
newsds
,
hash
,
&
defragged
);
}
}
...
...
src/dict.c
View file @
8b511219
...
@@ -66,7 +66,7 @@ static unsigned int dict_force_resize_ratio = 5;
...
@@ -66,7 +66,7 @@ static unsigned int dict_force_resize_ratio = 5;
static
int
_dictExpandIfNeeded
(
dict
*
ht
);
static
int
_dictExpandIfNeeded
(
dict
*
ht
);
static
unsigned
long
_dictNextPower
(
unsigned
long
size
);
static
unsigned
long
_dictNextPower
(
unsigned
long
size
);
static
int
_dictKeyIndex
(
dict
*
ht
,
const
void
*
key
,
u
nsigned
in
t
hash
,
dictEntry
**
existing
);
static
long
_dictKeyIndex
(
dict
*
ht
,
const
void
*
key
,
u
int64_
t
hash
,
dictEntry
**
existing
);
static
int
_dictInit
(
dict
*
ht
,
dictType
*
type
,
void
*
privDataPtr
);
static
int
_dictInit
(
dict
*
ht
,
dictType
*
type
,
void
*
privDataPtr
);
/* -------------------------- hash functions -------------------------------- */
/* -------------------------- hash functions -------------------------------- */
...
@@ -202,7 +202,7 @@ int dictRehash(dict *d, int n) {
...
@@ -202,7 +202,7 @@ int dictRehash(dict *d, int n) {
de
=
d
->
ht
[
0
].
table
[
d
->
rehashidx
];
de
=
d
->
ht
[
0
].
table
[
d
->
rehashidx
];
/* Move all the keys in this bucket from the old to the new hash HT */
/* Move all the keys in this bucket from the old to the new hash HT */
while
(
de
)
{
while
(
de
)
{
u
nsigned
in
t
h
;
u
int64_
t
h
;
nextde
=
de
->
next
;
nextde
=
de
->
next
;
/* Get the index in the new hash table */
/* Get the index in the new hash table */
...
@@ -291,7 +291,7 @@ int dictAdd(dict *d, void *key, void *val)
...
@@ -291,7 +291,7 @@ int dictAdd(dict *d, void *key, void *val)
*/
*/
dictEntry
*
dictAddRaw
(
dict
*
d
,
void
*
key
,
dictEntry
**
existing
)
dictEntry
*
dictAddRaw
(
dict
*
d
,
void
*
key
,
dictEntry
**
existing
)
{
{
int
index
;
long
index
;
dictEntry
*
entry
;
dictEntry
*
entry
;
dictht
*
ht
;
dictht
*
ht
;
...
@@ -362,7 +362,7 @@ dictEntry *dictAddOrFind(dict *d, void *key) {
...
@@ -362,7 +362,7 @@ dictEntry *dictAddOrFind(dict *d, void *key) {
* dictDelete() and dictUnlink(), please check the top comment
* dictDelete() and dictUnlink(), please check the top comment
* of those functions. */
* of those functions. */
static
dictEntry
*
dictGenericDelete
(
dict
*
d
,
const
void
*
key
,
int
nofree
)
{
static
dictEntry
*
dictGenericDelete
(
dict
*
d
,
const
void
*
key
,
int
nofree
)
{
u
nsigned
in
t
h
,
idx
;
u
int64_
t
h
,
idx
;
dictEntry
*
he
,
*
prevHe
;
dictEntry
*
he
,
*
prevHe
;
int
table
;
int
table
;
...
@@ -476,7 +476,7 @@ void dictRelease(dict *d)
...
@@ -476,7 +476,7 @@ void dictRelease(dict *d)
dictEntry
*
dictFind
(
dict
*
d
,
const
void
*
key
)
dictEntry
*
dictFind
(
dict
*
d
,
const
void
*
key
)
{
{
dictEntry
*
he
;
dictEntry
*
he
;
u
nsigned
in
t
h
,
idx
,
table
;
u
int64_
t
h
,
idx
,
table
;
if
(
d
->
ht
[
0
].
used
+
d
->
ht
[
1
].
used
==
0
)
return
NULL
;
/* dict is empty */
if
(
d
->
ht
[
0
].
used
+
d
->
ht
[
1
].
used
==
0
)
return
NULL
;
/* dict is empty */
if
(
dictIsRehashing
(
d
))
_dictRehashStep
(
d
);
if
(
dictIsRehashing
(
d
))
_dictRehashStep
(
d
);
...
@@ -610,7 +610,7 @@ void dictReleaseIterator(dictIterator *iter)
...
@@ -610,7 +610,7 @@ void dictReleaseIterator(dictIterator *iter)
dictEntry
*
dictGetRandomKey
(
dict
*
d
)
dictEntry
*
dictGetRandomKey
(
dict
*
d
)
{
{
dictEntry
*
he
,
*
orighe
;
dictEntry
*
he
,
*
orighe
;
unsigned
int
h
;
unsigned
long
h
;
int
listlen
,
listele
;
int
listlen
,
listele
;
if
(
dictSize
(
d
)
==
0
)
return
NULL
;
if
(
dictSize
(
d
)
==
0
)
return
NULL
;
...
@@ -955,9 +955,9 @@ static unsigned long _dictNextPower(unsigned long size)
...
@@ -955,9 +955,9 @@ static unsigned long _dictNextPower(unsigned long size)
*
*
* Note that if we are in the process of rehashing the hash table, the
* Note that if we are in the process of rehashing the hash table, the
* index is always returned in the context of the second (new) hash table. */
* index is always returned in the context of the second (new) hash table. */
static
int
_dictKeyIndex
(
dict
*
d
,
const
void
*
key
,
u
nsigned
in
t
hash
,
dictEntry
**
existing
)
static
long
_dictKeyIndex
(
dict
*
d
,
const
void
*
key
,
u
int64_
t
hash
,
dictEntry
**
existing
)
{
{
unsigned
int
idx
,
table
;
unsigned
long
idx
,
table
;
dictEntry
*
he
;
dictEntry
*
he
;
if
(
existing
)
*
existing
=
NULL
;
if
(
existing
)
*
existing
=
NULL
;
...
@@ -995,7 +995,7 @@ void dictDisableResize(void) {
...
@@ -995,7 +995,7 @@ void dictDisableResize(void) {
dict_can_resize
=
0
;
dict_can_resize
=
0
;
}
}
u
nsigned
in
t
dictGetHash
(
dict
*
d
,
const
void
*
key
)
{
u
int64_
t
dictGetHash
(
dict
*
d
,
const
void
*
key
)
{
return
dictHashKey
(
d
,
key
);
return
dictHashKey
(
d
,
key
);
}
}
...
@@ -1004,9 +1004,9 @@ unsigned int dictGetHash(dict *d, const void *key) {
...
@@ -1004,9 +1004,9 @@ unsigned int dictGetHash(dict *d, const void *key) {
* the hash value should be provided using dictGetHash.
* the hash value should be provided using dictGetHash.
* no string / key comparison is performed.
* no string / key comparison is performed.
* return value is the reference to the dictEntry if found, or NULL if not found. */
* return value is the reference to the dictEntry if found, or NULL if not found. */
dictEntry
**
dictFindEntryRefByPtrAndHash
(
dict
*
d
,
const
void
*
oldptr
,
u
nsigned
in
t
hash
)
{
dictEntry
**
dictFindEntryRefByPtrAndHash
(
dict
*
d
,
const
void
*
oldptr
,
u
int64_
t
hash
)
{
dictEntry
*
he
,
**
heref
;
dictEntry
*
he
,
**
heref
;
unsigned
int
idx
,
table
;
unsigned
long
idx
,
table
;
if
(
d
->
ht
[
0
].
used
+
d
->
ht
[
1
].
used
==
0
)
return
NULL
;
/* dict is empty */
if
(
d
->
ht
[
0
].
used
+
d
->
ht
[
1
].
used
==
0
)
return
NULL
;
/* dict is empty */
for
(
table
=
0
;
table
<=
1
;
table
++
)
{
for
(
table
=
0
;
table
<=
1
;
table
++
)
{
...
...
src/dict.h
View file @
8b511219
...
@@ -178,8 +178,8 @@ int dictRehashMilliseconds(dict *d, int ms);
...
@@ -178,8 +178,8 @@ int dictRehashMilliseconds(dict *d, int ms);
void
dictSetHashFunctionSeed
(
uint8_t
*
seed
);
void
dictSetHashFunctionSeed
(
uint8_t
*
seed
);
uint8_t
*
dictGetHashFunctionSeed
(
void
);
uint8_t
*
dictGetHashFunctionSeed
(
void
);
unsigned
long
dictScan
(
dict
*
d
,
unsigned
long
v
,
dictScanFunction
*
fn
,
dictScanBucketFunction
*
bucketfn
,
void
*
privdata
);
unsigned
long
dictScan
(
dict
*
d
,
unsigned
long
v
,
dictScanFunction
*
fn
,
dictScanBucketFunction
*
bucketfn
,
void
*
privdata
);
u
nsigned
in
t
dictGetHash
(
dict
*
d
,
const
void
*
key
);
u
int64_
t
dictGetHash
(
dict
*
d
,
const
void
*
key
);
dictEntry
**
dictFindEntryRefByPtrAndHash
(
dict
*
d
,
const
void
*
oldptr
,
u
nsigned
in
t
hash
);
dictEntry
**
dictFindEntryRefByPtrAndHash
(
dict
*
d
,
const
void
*
oldptr
,
u
int64_
t
hash
);
/* Hash table types */
/* Hash table types */
extern
dictType
dictTypeHeapStringCopyKey
;
extern
dictType
dictTypeHeapStringCopyKey
;
...
...
src/evict.c
View file @
8b511219
...
@@ -60,8 +60,6 @@ struct evictionPoolEntry {
...
@@ -60,8 +60,6 @@ struct evictionPoolEntry {
static
struct
evictionPoolEntry
*
EvictionPoolLRU
;
static
struct
evictionPoolEntry
*
EvictionPoolLRU
;
unsigned
long
LFUDecrAndReturn
(
robj
*
o
);
/* ----------------------------------------------------------------------------
/* ----------------------------------------------------------------------------
* Implementation of eviction, aging and LRU
* Implementation of eviction, aging and LRU
* --------------------------------------------------------------------------*/
* --------------------------------------------------------------------------*/
...
@@ -302,8 +300,8 @@ unsigned long LFUGetTimeInMinutes(void) {
...
@@ -302,8 +300,8 @@ unsigned long LFUGetTimeInMinutes(void) {
return
(
server
.
unixtime
/
60
)
&
65535
;
return
(
server
.
unixtime
/
60
)
&
65535
;
}
}
/* Given an object last
decrement
time, compute the minimum number of minutes
/* Given an object last
access
time, compute the minimum number of minutes
* that elapsed since the last
decrement
. Handle overflow (ldt greater than
* that elapsed since the last
access
. Handle overflow (ldt greater than
* the current 16 bits minutes time) considering the time as wrapping
* the current 16 bits minutes time) considering the time as wrapping
* exactly once. */
* exactly once. */
unsigned
long
LFUTimeElapsed
(
unsigned
long
ldt
)
{
unsigned
long
LFUTimeElapsed
(
unsigned
long
ldt
)
{
...
@@ -324,25 +322,22 @@ uint8_t LFULogIncr(uint8_t counter) {
...
@@ -324,25 +322,22 @@ uint8_t LFULogIncr(uint8_t counter) {
return
counter
;
return
counter
;
}
}
/* If the object decrement time is reached, decrement the LFU counter and
/* If the object decrement time is reached decrement the LFU counter but
* update the decrement time field. Return the object frequency counter.
* do not update LFU fields of the object, we update the access time
* and counter in an explicit way when the object is really accessed.
* And we will times halve the counter according to the times of
* elapsed time than server.lfu_decay_time.
* Return the object frequency counter.
*
*
* This function is used in order to scan the dataset for the best object
* This function is used in order to scan the dataset for the best object
* to fit: as we check for the candidate, we incrementally decrement the
* to fit: as we check for the candidate, we incrementally decrement the
* counter of the scanned objects if needed. */
* counter of the scanned objects if needed. */
#define LFU_DECR_INTERVAL 1
unsigned
long
LFUDecrAndReturn
(
robj
*
o
)
{
unsigned
long
LFUDecrAndReturn
(
robj
*
o
)
{
unsigned
long
ldt
=
o
->
lru
>>
8
;
unsigned
long
ldt
=
o
->
lru
>>
8
;
unsigned
long
counter
=
o
->
lru
&
255
;
unsigned
long
counter
=
o
->
lru
&
255
;
if
(
LFUTimeElapsed
(
ldt
)
>=
server
.
lfu_decay_time
&&
counter
)
{
unsigned
long
num_periods
=
server
.
lfu_decay_time
?
LFUTimeElapsed
(
ldt
)
/
server
.
lfu_decay_time
:
0
;
if
(
counter
>
LFU_INIT_VAL
*
2
)
{
if
(
num_periods
)
counter
/=
2
;
counter
=
(
num_periods
>
counter
)
?
0
:
counter
-
num_periods
;
if
(
counter
<
LFU_INIT_VAL
*
2
)
counter
=
LFU_INIT_VAL
*
2
;
}
else
{
counter
--
;
}
o
->
lru
=
(
LFUGetTimeInMinutes
()
<<
8
)
|
counter
;
}
return
counter
;
return
counter
;
}
}
...
...
src/listpack.c
0 → 100644
View file @
8b511219
/* Listpack -- A lists of strings serialization format
*
* This file implements the specification you can find at:
*
* https://github.com/antirez/listpack
*
* Copyright (c) 2017, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdint.h>
#include <limits.h>
#include <sys/types.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "listpack.h"
#include "listpack_malloc.h"
#define LP_HDR_SIZE 6
/* 32 bit total len + 16 bit number of elements. */
#define LP_HDR_NUMELE_UNKNOWN UINT16_MAX
#define LP_MAX_INT_ENCODING_LEN 9
#define LP_MAX_BACKLEN_SIZE 5
#define LP_MAX_ENTRY_BACKLEN 34359738367ULL
#define LP_ENCODING_INT 0
#define LP_ENCODING_STRING 1
#define LP_ENCODING_7BIT_UINT 0
#define LP_ENCODING_7BIT_UINT_MASK 0x80
#define LP_ENCODING_IS_7BIT_UINT(byte) (((byte)&LP_ENCODING_7BIT_UINT_MASK)==LP_ENCODING_7BIT_UINT)
#define LP_ENCODING_6BIT_STR 0x80
#define LP_ENCODING_6BIT_STR_MASK 0xC0
#define LP_ENCODING_IS_6BIT_STR(byte) (((byte)&LP_ENCODING_6BIT_STR_MASK)==LP_ENCODING_6BIT_STR)
#define LP_ENCODING_13BIT_INT 0xC0
#define LP_ENCODING_13BIT_INT_MASK 0xE0
#define LP_ENCODING_IS_13BIT_INT(byte) (((byte)&LP_ENCODING_13BIT_INT_MASK)==LP_ENCODING_13BIT_INT)
#define LP_ENCODING_12BIT_STR 0xE0
#define LP_ENCODING_12BIT_STR_MASK 0xF0
#define LP_ENCODING_IS_12BIT_STR(byte) (((byte)&LP_ENCODING_12BIT_STR_MASK)==LP_ENCODING_12BIT_STR)
#define LP_ENCODING_16BIT_INT 0xF1
#define LP_ENCODING_16BIT_INT_MASK 0xFF
#define LP_ENCODING_IS_16BIT_INT(byte) (((byte)&LP_ENCODING_16BIT_INT_MASK)==LP_ENCODING_16BIT_INT)
#define LP_ENCODING_24BIT_INT 0xF2
#define LP_ENCODING_24BIT_INT_MASK 0xFF
#define LP_ENCODING_IS_24BIT_INT(byte) (((byte)&LP_ENCODING_24BIT_INT_MASK)==LP_ENCODING_24BIT_INT)
#define LP_ENCODING_32BIT_INT 0xF3
#define LP_ENCODING_32BIT_INT_MASK 0xFF
#define LP_ENCODING_IS_32BIT_INT(byte) (((byte)&LP_ENCODING_32BIT_INT_MASK)==LP_ENCODING_32BIT_INT)
#define LP_ENCODING_64BIT_INT 0xF4
#define LP_ENCODING_64BIT_INT_MASK 0xFF
#define LP_ENCODING_IS_64BIT_INT(byte) (((byte)&LP_ENCODING_64BIT_INT_MASK)==LP_ENCODING_64BIT_INT)
#define LP_ENCODING_32BIT_STR 0xF0
#define LP_ENCODING_32BIT_STR_MASK 0xFF
#define LP_ENCODING_IS_32BIT_STR(byte) (((byte)&LP_ENCODING_32BIT_STR_MASK)==LP_ENCODING_32BIT_STR)
#define LP_EOF 0xFF
#define LP_ENCODING_6BIT_STR_LEN(p) ((p)[0] & 0x3F)
#define LP_ENCODING_12BIT_STR_LEN(p) ((((p)[0] & 0xF) << 8) | (p)[1])
#define LP_ENCODING_32BIT_STR_LEN(p) (((uint32_t)(p)[1]<<0) | \
((uint32_t)(p)[2]<<8) | \
((uint32_t)(p)[3]<<16) | \
((uint32_t)(p)[4]<<24))
#define lpGetTotalBytes(p) (((uint32_t)(p)[0]<<0) | \
((uint32_t)(p)[1]<<8) | \
((uint32_t)(p)[2]<<16) | \
((uint32_t)(p)[3]<<24))
#define lpGetNumElements(p) (((uint32_t)(p)[4]<<0) | \
((uint32_t)(p)[5]<<8))
#define lpSetTotalBytes(p,v) do { \
(p)[0] = (v)&0xff; \
(p)[1] = ((v)>>8)&0xff; \
(p)[2] = ((v)>>16)&0xff; \
(p)[3] = ((v)>>24)&0xff; \
} while(0)
#define lpSetNumElements(p,v) do { \
(p)[4] = (v)&0xff; \
(p)[5] = ((v)>>8)&0xff; \
} while(0)
/* Convert a string into a signed 64 bit integer.
* The function returns 1 if the string could be parsed into a (non-overflowing)
* signed 64 bit int, 0 otherwise. The 'value' will be set to the parsed value
* when the function returns success.
*
* Note that this function demands that the string strictly represents
* a int64 value: no spaces or other characters before or after the string
* representing the number are accepted, nor zeroes at the start if not
* for the string "0" representing the zero number.
*
* Because of its strictness, it is safe to use this function to check if
* you can convert a string into a long long, and obtain back the string
* from the number without any loss in the string representation. *
*
* -----------------------------------------------------------------------------
*
* Credits: this function was adapted from the Redis source code, file
* "utils.c", function string2ll(), and is copyright:
*
* Copyright(C) 2011, Pieter Noordhuis
* Copyright(C) 2011, Salvatore Sanfilippo
*
* The function is released under the BSD 3-clause license.
*/
int
lpStringToInt64
(
const
char
*
s
,
unsigned
long
slen
,
int64_t
*
value
)
{
const
char
*
p
=
s
;
unsigned
long
plen
=
0
;
int
negative
=
0
;
uint64_t
v
;
if
(
plen
==
slen
)
return
0
;
/* Special case: first and only digit is 0. */
if
(
slen
==
1
&&
p
[
0
]
==
'0'
)
{
if
(
value
!=
NULL
)
*
value
=
0
;
return
1
;
}
if
(
p
[
0
]
==
'-'
)
{
negative
=
1
;
p
++
;
plen
++
;
/* Abort on only a negative sign. */
if
(
plen
==
slen
)
return
0
;
}
/* First digit should be 1-9, otherwise the string should just be 0. */
if
(
p
[
0
]
>=
'1'
&&
p
[
0
]
<=
'9'
)
{
v
=
p
[
0
]
-
'0'
;
p
++
;
plen
++
;
}
else
if
(
p
[
0
]
==
'0'
&&
slen
==
1
)
{
*
value
=
0
;
return
1
;
}
else
{
return
0
;
}
while
(
plen
<
slen
&&
p
[
0
]
>=
'0'
&&
p
[
0
]
<=
'9'
)
{
if
(
v
>
(
UINT64_MAX
/
10
))
/* Overflow. */
return
0
;
v
*=
10
;
if
(
v
>
(
UINT64_MAX
-
(
p
[
0
]
-
'0'
)))
/* Overflow. */
return
0
;
v
+=
p
[
0
]
-
'0'
;
p
++
;
plen
++
;
}
/* Return if not all bytes were used. */
if
(
plen
<
slen
)
return
0
;
if
(
negative
)
{
if
(
v
>
((
uint64_t
)(
-
(
INT64_MIN
+
1
))
+
1
))
/* Overflow. */
return
0
;
if
(
value
!=
NULL
)
*
value
=
-
v
;
}
else
{
if
(
v
>
INT64_MAX
)
/* Overflow. */
return
0
;
if
(
value
!=
NULL
)
*
value
=
v
;
}
return
1
;
}
/* Create a new, empty listpack.
* On success the new listpack is returned, otherwise an error is returned. */
unsigned
char
*
lpNew
(
void
)
{
unsigned
char
*
lp
=
lp_malloc
(
LP_HDR_SIZE
+
1
);
if
(
lp
==
NULL
)
return
NULL
;
lpSetTotalBytes
(
lp
,
LP_HDR_SIZE
+
1
);
lpSetNumElements
(
lp
,
0
);
lp
[
LP_HDR_SIZE
]
=
LP_EOF
;
return
lp
;
}
/* Free the specified listpack. */
void
lpFree
(
unsigned
char
*
lp
)
{
lp_free
(
lp
);
}
/* Given an element 'ele' of size 'size', determine if the element can be
* represented inside the listpack encoded as integer, and returns
* LP_ENCODING_INT if so. Otherwise returns LP_ENCODING_STR if no integer
* encoding is possible.
*
* If the LP_ENCODING_INT is returned, the function stores the integer encoded
* representation of the element in the 'intenc' buffer.
*
* Regardless of the returned encoding, 'enclen' is populated by reference to
* the number of bytes that the string or integer encoded element will require
* in order to be represented. */
int
lpEncodeGetType
(
unsigned
char
*
ele
,
uint32_t
size
,
unsigned
char
*
intenc
,
uint64_t
*
enclen
)
{
int64_t
v
;
if
(
lpStringToInt64
((
const
char
*
)
ele
,
size
,
&
v
))
{
if
(
v
>=
0
&&
v
<=
127
)
{
/* Single byte 0-127 integer. */
intenc
[
0
]
=
v
;
*
enclen
=
1
;
}
else
if
(
v
>=
-
4096
&&
v
<=
4095
)
{
/* 13 bit integer. */
if
(
v
<
0
)
v
=
((
int64_t
)
1
<<
13
)
+
v
;
intenc
[
0
]
=
(
v
>>
8
)
|
LP_ENCODING_13BIT_INT
;
intenc
[
1
]
=
v
&
0xff
;
*
enclen
=
2
;
}
else
if
(
v
>=
-
32768
&&
v
<=
32767
)
{
/* 16 bit integer. */
if
(
v
<
0
)
v
=
((
int64_t
)
1
<<
16
)
+
v
;
intenc
[
0
]
=
LP_ENCODING_16BIT_INT
;
intenc
[
1
]
=
v
&
0xff
;
intenc
[
2
]
=
v
>>
8
;
*
enclen
=
3
;
}
else
if
(
v
>=
-
8388608
&&
v
<=
8388607
)
{
/* 24 bit integer. */
if
(
v
<
0
)
v
=
((
int64_t
)
1
<<
24
)
+
v
;
intenc
[
0
]
=
LP_ENCODING_24BIT_INT
;
intenc
[
1
]
=
v
&
0xff
;
intenc
[
2
]
=
(
v
>>
8
)
&
0xff
;
intenc
[
3
]
=
v
>>
16
;
*
enclen
=
4
;
}
else
if
(
v
>=
-
2147483648
&&
v
<=
2147483647
)
{
/* 32 bit integer. */
if
(
v
<
0
)
v
=
((
int64_t
)
1
<<
32
)
+
v
;
intenc
[
0
]
=
LP_ENCODING_32BIT_INT
;
intenc
[
1
]
=
v
&
0xff
;
intenc
[
2
]
=
(
v
>>
8
)
&
0xff
;
intenc
[
3
]
=
(
v
>>
16
)
&
0xff
;
intenc
[
4
]
=
v
>>
24
;
*
enclen
=
5
;
}
else
{
/* 64 bit integer. */
uint64_t
uv
=
v
;
intenc
[
0
]
=
LP_ENCODING_64BIT_INT
;
intenc
[
1
]
=
uv
&
0xff
;
intenc
[
2
]
=
(
uv
>>
8
)
&
0xff
;
intenc
[
3
]
=
(
uv
>>
16
)
&
0xff
;
intenc
[
4
]
=
(
uv
>>
24
)
&
0xff
;
intenc
[
5
]
=
(
uv
>>
32
)
&
0xff
;
intenc
[
6
]
=
(
uv
>>
40
)
&
0xff
;
intenc
[
7
]
=
(
uv
>>
48
)
&
0xff
;
intenc
[
8
]
=
uv
>>
56
;
*
enclen
=
9
;
}
return
LP_ENCODING_INT
;
}
else
{
if
(
size
<
64
)
*
enclen
=
1
+
size
;
else
if
(
size
<
4096
)
*
enclen
=
2
+
size
;
else
*
enclen
=
5
+
size
;
return
LP_ENCODING_STRING
;
}
}
/* Store a reverse-encoded variable length field, representing the length
* of the previous element of size 'l', in the target buffer 'buf'.
* The function returns the number of bytes used to encode it, from
* 1 to 5. If 'buf' is NULL the funciton just returns the number of bytes
* needed in order to encode the backlen. */
unsigned
long
lpEncodeBacklen
(
unsigned
char
*
buf
,
uint64_t
l
)
{
if
(
l
<=
127
)
{
if
(
buf
)
buf
[
0
]
=
l
;
return
1
;
}
else
if
(
l
<
16383
)
{
if
(
buf
)
{
buf
[
0
]
=
l
>>
7
;
buf
[
1
]
=
(
l
&
127
)
|
128
;
}
return
2
;
}
else
if
(
l
<
2097151
)
{
if
(
buf
)
{
buf
[
0
]
=
l
>>
14
;
buf
[
1
]
=
((
l
>>
7
)
&
127
)
|
128
;
buf
[
2
]
=
(
l
&
127
)
|
128
;
}
return
3
;
}
else
if
(
l
<
268435455
)
{
if
(
buf
)
{
buf
[
0
]
=
l
>>
21
;
buf
[
1
]
=
((
l
>>
14
)
&
127
)
|
128
;
buf
[
2
]
=
((
l
>>
7
)
&
127
)
|
128
;
buf
[
3
]
=
(
l
&
127
)
|
128
;
}
return
4
;
}
else
{
if
(
buf
)
{
buf
[
0
]
=
l
>>
28
;
buf
[
1
]
=
((
l
>>
21
)
&
127
)
|
128
;
buf
[
2
]
=
((
l
>>
14
)
&
127
)
|
128
;
buf
[
3
]
=
((
l
>>
7
)
&
127
)
|
128
;
buf
[
4
]
=
(
l
&
127
)
|
128
;
}
return
5
;
}
}
/* Decode the backlen and returns it. If the encoding looks invalid (more than
* 5 bytes are used), UINT64_MAX is returned to report the problem. */
uint64_t
lpDecodeBacklen
(
unsigned
char
*
p
)
{
uint64_t
val
=
0
;
uint64_t
shift
=
0
;
do
{
val
|=
(
uint64_t
)(
p
[
0
]
&
127
)
<<
shift
;
if
(
!
(
p
[
0
]
&
128
))
break
;
shift
+=
7
;
p
--
;
if
(
shift
>
28
)
return
UINT64_MAX
;
}
while
(
1
);
return
val
;
}
/* Encode the string element pointed by 's' of size 'len' in the target
* buffer 's'. The function should be called with 'buf' having always enough
* space for encoding the string. This is done by calling lpEncodeGetType()
* before calling this function. */
void
lpEncodeString
(
unsigned
char
*
buf
,
unsigned
char
*
s
,
uint32_t
len
)
{
if
(
len
<
64
)
{
buf
[
0
]
=
len
|
LP_ENCODING_6BIT_STR
;
memcpy
(
buf
+
1
,
s
,
len
);
}
else
if
(
len
<
4096
)
{
buf
[
0
]
=
(
len
>>
8
)
|
LP_ENCODING_12BIT_STR
;
buf
[
1
]
=
len
&
0xff
;
memcpy
(
buf
+
2
,
s
,
len
);
}
else
{
buf
[
0
]
=
LP_ENCODING_32BIT_STR
;
buf
[
1
]
=
len
&
0xff
;
buf
[
2
]
=
(
len
>>
8
)
&
0xff
;
buf
[
3
]
=
(
len
>>
16
)
&
0xff
;
buf
[
4
]
=
(
len
>>
24
)
&
0xff
;
memcpy
(
buf
+
5
,
s
,
len
);
}
}
/* Return the encoded length of the listpack element pointed by 'p'. If the
* element encoding is wrong then 0 is returned. */
uint32_t
lpCurrentEncodedSize
(
unsigned
char
*
p
)
{
if
(
LP_ENCODING_IS_7BIT_UINT
(
p
[
0
]))
return
1
;
if
(
LP_ENCODING_IS_6BIT_STR
(
p
[
0
]))
return
1
+
LP_ENCODING_6BIT_STR_LEN
(
p
);
if
(
LP_ENCODING_IS_13BIT_INT
(
p
[
0
]))
return
2
;
if
(
LP_ENCODING_IS_16BIT_INT
(
p
[
0
]))
return
3
;
if
(
LP_ENCODING_IS_24BIT_INT
(
p
[
0
]))
return
4
;
if
(
LP_ENCODING_IS_32BIT_INT
(
p
[
0
]))
return
5
;
if
(
LP_ENCODING_IS_64BIT_INT
(
p
[
0
]))
return
9
;
if
(
LP_ENCODING_IS_12BIT_STR
(
p
[
0
]))
return
2
+
LP_ENCODING_12BIT_STR_LEN
(
p
);
if
(
LP_ENCODING_IS_32BIT_STR
(
p
[
0
]))
return
5
+
LP_ENCODING_32BIT_STR_LEN
(
p
);
if
(
p
[
0
]
==
LP_EOF
)
return
1
;
return
0
;
}
/* Skip the current entry returning the next. It is invalid to call this
* function if the current element is the EOF element at the end of the
* listpack, however, while this function is used to implement lpNext(),
* it does not return NULL when the EOF element is encountered. */
unsigned
char
*
lpSkip
(
unsigned
char
*
p
)
{
unsigned
long
entrylen
=
lpCurrentEncodedSize
(
p
);
entrylen
+=
lpEncodeBacklen
(
NULL
,
entrylen
);
p
+=
entrylen
;
return
p
;
}
/* If 'p' points to an element of the listpack, calling lpNext() will return
* the pointer to the next element (the one on the right), or NULL if 'p'
* already pointed to the last element of the listpack. */
unsigned
char
*
lpNext
(
unsigned
char
*
lp
,
unsigned
char
*
p
)
{
((
void
)
lp
);
/* lp is not used for now. However lpPrev() uses it. */
p
=
lpSkip
(
p
);
if
(
p
[
0
]
==
LP_EOF
)
return
NULL
;
return
p
;
}
/* If 'p' points to an element of the listpack, calling lpPrev() will return
* the pointer to the preivous element (the one on the left), or NULL if 'p'
* already pointed to the first element of the listpack. */
unsigned
char
*
lpPrev
(
unsigned
char
*
lp
,
unsigned
char
*
p
)
{
if
(
p
-
lp
==
LP_HDR_SIZE
)
return
NULL
;
p
--
;
/* Seek the first backlen byte of the last element. */
uint64_t
prevlen
=
lpDecodeBacklen
(
p
);
prevlen
+=
lpEncodeBacklen
(
NULL
,
prevlen
);
return
p
-
prevlen
+
1
;
/* Seek the first byte of the previous entry. */
}
/* Return a pointer to the first element of the listpack, or NULL if the
* listpack has no elements. */
unsigned
char
*
lpFirst
(
unsigned
char
*
lp
)
{
lp
+=
LP_HDR_SIZE
;
/* Skip the header. */
if
(
lp
[
0
]
==
LP_EOF
)
return
NULL
;
return
lp
;
}
/* Return a pointer to the last element of the listpack, or NULL if the
* listpack has no elements. */
unsigned
char
*
lpLast
(
unsigned
char
*
lp
)
{
unsigned
char
*
p
=
lp
+
lpGetTotalBytes
(
lp
)
-
1
;
/* Seek EOF element. */
return
lpPrev
(
lp
,
p
);
/* Will return NULL if EOF is the only element. */
}
/* Return the number of elements inside the listpack. This function attempts
* to use the cached value when within range, otherwise a full scan is
* needed. As a side effect of calling this function, the listpack header
* could be modified, because if the count is found to be already within
* the 'numele' header field range, the new value is set. */
uint32_t
lpLength
(
unsigned
char
*
lp
)
{
uint32_t
numele
=
lpGetNumElements
(
lp
);
if
(
numele
!=
LP_HDR_NUMELE_UNKNOWN
)
return
numele
;
/* Too many elements inside the listpack. We need to scan in order
* to get the total number. */
uint32_t
count
=
0
;
unsigned
char
*
p
=
lpFirst
(
lp
);
while
(
p
)
{
count
++
;
p
=
lpNext
(
lp
,
p
);
}
/* If the count is again within range of the header numele field,
* set it. */
if
(
count
<
LP_HDR_NUMELE_UNKNOWN
)
lpSetNumElements
(
lp
,
count
);
return
count
;
}
/* Return the listpack element pointed by 'p'.
*
* The function changes behavior depending on the passed 'intbuf' value.
* Specifically, if 'intbuf' is NULL:
*
* If the element is internally encoded as an integer, the function returns
* NULL and populates the integer value by reference in 'count'. Otherwise if
* the element is encoded as a string a pointer to the string (pointing inside
* the listpack itself) is returned, and 'count' is set to the length of the
* string.
*
* If instead 'intbuf' points to a buffer passed by the caller, that must be
* at least LP_INTBUF_SIZE bytes, the function always returns the element as
* it was a string (returning the pointer to the string and setting the
* 'count' argument to the string length by reference). However if the element
* is encoded as an integer, the 'intbuf' buffer is used in order to store
* the string representation.
*
* The user should use one or the other form depending on what the value will
* be used for. If there is immediate usage for an integer value returned
* by the function, than to pass a buffer (and convert it back to a number)
* is of course useless.
*
* If the function is called against a badly encoded ziplist, so that there
* is no valid way to parse it, the function returns like if there was an
* integer encoded with value 12345678900000000 + <unrecognized byte>, this may
* be an hint to understand that something is wrong. To crash in this case is
* not sensible because of the different requirements of the application using
* this lib.
*
* Similarly, there is no error returned since the listpack normally can be
* assumed to be valid, so that would be a very high API cost. However a function
* in order to check the integrity of the listpack at load time is provided,
* check lpIsValid(). */
unsigned
char
*
lpGet
(
unsigned
char
*
p
,
int64_t
*
count
,
unsigned
char
*
intbuf
)
{
int64_t
val
;
uint64_t
uval
,
negstart
,
negmax
;
if
(
LP_ENCODING_IS_7BIT_UINT
(
p
[
0
]))
{
negstart
=
UINT64_MAX
;
/* 7 bit ints are always positive. */
negmax
=
0
;
uval
=
p
[
0
]
&
0x7f
;
}
else
if
(
LP_ENCODING_IS_6BIT_STR
(
p
[
0
]))
{
*
count
=
LP_ENCODING_6BIT_STR_LEN
(
p
);
return
p
+
1
;
}
else
if
(
LP_ENCODING_IS_13BIT_INT
(
p
[
0
]))
{
uval
=
((
p
[
0
]
&
0x1f
)
<<
8
)
|
p
[
1
];
negstart
=
(
uint64_t
)
1
<<
12
;
negmax
=
8191
;
}
else
if
(
LP_ENCODING_IS_16BIT_INT
(
p
[
0
]))
{
uval
=
(
uint64_t
)
p
[
1
]
|
(
uint64_t
)
p
[
2
]
<<
8
;
negstart
=
(
uint64_t
)
1
<<
15
;
negmax
=
UINT16_MAX
;
}
else
if
(
LP_ENCODING_IS_24BIT_INT
(
p
[
0
]))
{
uval
=
(
uint64_t
)
p
[
1
]
|
(
uint64_t
)
p
[
2
]
<<
8
|
(
uint64_t
)
p
[
3
]
<<
16
;
negstart
=
(
uint64_t
)
1
<<
23
;
negmax
=
UINT32_MAX
>>
8
;
}
else
if
(
LP_ENCODING_IS_32BIT_INT
(
p
[
0
]))
{
uval
=
(
uint64_t
)
p
[
1
]
|
(
uint64_t
)
p
[
2
]
<<
8
|
(
uint64_t
)
p
[
3
]
<<
16
|
(
uint64_t
)
p
[
4
]
<<
24
;
negstart
=
(
uint64_t
)
1
<<
31
;
negmax
=
UINT32_MAX
;
}
else
if
(
LP_ENCODING_IS_64BIT_INT
(
p
[
0
]))
{
uval
=
(
uint64_t
)
p
[
1
]
|
(
uint64_t
)
p
[
2
]
<<
8
|
(
uint64_t
)
p
[
3
]
<<
16
|
(
uint64_t
)
p
[
4
]
<<
24
|
(
uint64_t
)
p
[
5
]
<<
32
|
(
uint64_t
)
p
[
6
]
<<
40
|
(
uint64_t
)
p
[
7
]
<<
48
|
(
uint64_t
)
p
[
8
]
<<
56
;
negstart
=
(
uint64_t
)
1
<<
63
;
negmax
=
UINT64_MAX
;
}
else
if
(
LP_ENCODING_IS_12BIT_STR
(
p
[
0
]))
{
*
count
=
LP_ENCODING_12BIT_STR_LEN
(
p
);
return
p
+
2
;
}
else
if
(
LP_ENCODING_IS_32BIT_STR
(
p
[
0
]))
{
*
count
=
LP_ENCODING_32BIT_STR_LEN
(
p
);
return
p
+
5
;
}
else
{
uval
=
12345678900000000ULL
+
p
[
0
];
negstart
=
UINT64_MAX
;
negmax
=
0
;
}
/* We reach this code path only for integer encodings.
* Convert the unsigned value to the signed one using two's complement
* rule. */
if
(
uval
>=
negstart
)
{
/* This three steps conversion should avoid undefined behaviors
* in the unsigned -> signed conversion. */
uval
=
negmax
-
uval
;
val
=
uval
;
val
=
-
val
-
1
;
}
else
{
val
=
uval
;
}
/* Return the string representation of the integer or the value itself
* depending on intbuf being NULL or not. */
if
(
intbuf
)
{
*
count
=
snprintf
((
char
*
)
intbuf
,
LP_INTBUF_SIZE
,
"%lld"
,
val
);
return
intbuf
;
}
else
{
*
count
=
val
;
return
NULL
;
}
}
/* Insert, delete or replace the specified element 'ele' of lenght 'len' at
* the specified position 'p', with 'p' being a listpack element pointer
* obtained with lpFirst(), lpLast(), lpIndex(), lpNext(), lpPrev() or
* lpSeek().
*
* The element is inserted before, after, or replaces the element pointed
* by 'p' depending on the 'where' argument, that can be LP_BEFORE, LP_AFTER
* or LP_REPLACE.
*
* If 'ele' is set to NULL, the function removes the element pointed by 'p'
* instead of inserting one.
*
* Returns NULL on out of memory or when the listpack total length would exceed
* the max allowed size of 2^32-1, otherwise the new pointer to the listpack
* holding the new element is returned (and the old pointer passed is no longer
* considered valid)
*
* If 'newp' is not NULL, at the end of a successful call '*newp' will be set
* to the address of the element just added, so that it will be possible to
* continue an interation with lpNext() and lpPrev().
*
* For deletion operations ('ele' set to NULL) 'newp' is set to the next
* element, on the right of the deleted one, or to NULL if the deleted element
* was the last one. */
unsigned
char
*
lpInsert
(
unsigned
char
*
lp
,
unsigned
char
*
ele
,
uint32_t
size
,
unsigned
char
*
p
,
int
where
,
unsigned
char
**
newp
)
{
unsigned
char
intenc
[
LP_MAX_INT_ENCODING_LEN
];
unsigned
char
backlen
[
LP_MAX_BACKLEN_SIZE
];
uint64_t
enclen
;
/* The length of the encoded element. */
/* An element pointer set to NULL means deletion, which is conceptually
* replacing the element with a zero-length element. So whatever we
* get passed as 'where', set it to LP_REPLACE. */
if
(
ele
==
NULL
)
where
=
LP_REPLACE
;
/* If we need to insert after the current element, we just jump to the
* next element (that could be the EOF one) and handle the case of
* inserting before. So the function will actually deal with just two
* cases: LP_BEFORE and LP_REPLACE. */
if
(
where
==
LP_AFTER
)
{
p
=
lpSkip
(
p
);
where
=
LP_BEFORE
;
}
/* Store the offset of the element 'p', so that we can obtain its
* address again after a reallocation. */
unsigned
long
poff
=
p
-
lp
;
/* Calling lpEncodeGetType() results into the encoded version of the
* element to be stored into 'intenc' in case it is representable as
* an integer: in that case, the function returns LP_ENCODING_INT.
* Otherwise if LP_ENCODING_STR is returned, we'll have to call
* lpEncodeString() to actually write the encoded string on place later.
*
* Whatever the returned encoding is, 'enclen' is populated with the
* length of the encoded element. */
int
enctype
;
if
(
ele
)
{
enctype
=
lpEncodeGetType
(
ele
,
size
,
intenc
,
&
enclen
);
}
else
{
enctype
=
-
1
;
enclen
=
0
;
}
/* We need to also encode the backward-parsable length of the element
* and append it to the end: this allows to traverse the listpack from
* the end to the start. */
unsigned
long
backlen_size
=
ele
?
lpEncodeBacklen
(
backlen
,
enclen
)
:
0
;
uint64_t
old_listpack_bytes
=
lpGetTotalBytes
(
lp
);
uint32_t
replaced_len
=
0
;
if
(
where
==
LP_REPLACE
)
{
replaced_len
=
lpCurrentEncodedSize
(
p
);
replaced_len
+=
lpEncodeBacklen
(
NULL
,
replaced_len
);
}
uint64_t
new_listpack_bytes
=
old_listpack_bytes
+
enclen
+
backlen_size
-
replaced_len
;
if
(
new_listpack_bytes
>
UINT32_MAX
)
return
NULL
;
/* We now need to reallocate in order to make space or shrink the
* allocation (in case 'when' value is LP_REPLACE and the new element is
* smaller). However we do that before memmoving the memory to
* make room for the new element if the final allocation will get
* larger, or we do it after if the final allocation will get smaller. */
unsigned
char
*
dst
=
lp
+
poff
;
/* May be updated after reallocation. */
/* Realloc before: we need more room. */
if
(
new_listpack_bytes
>
old_listpack_bytes
)
{
if
((
lp
=
lp_realloc
(
lp
,
new_listpack_bytes
))
==
NULL
)
return
NULL
;
dst
=
lp
+
poff
;
}
/* Setup the listpack relocating the elements to make the exact room
* we need to store the new one. */
if
(
where
==
LP_BEFORE
)
{
memmove
(
dst
+
enclen
+
backlen_size
,
dst
,
old_listpack_bytes
-
poff
);
}
else
{
/* LP_REPLACE. */
long
lendiff
=
(
enclen
+
backlen_size
)
-
replaced_len
;
memmove
(
dst
+
replaced_len
+
lendiff
,
dst
+
replaced_len
,
old_listpack_bytes
-
poff
-
replaced_len
);
}
/* Realloc after: we need to free space. */
if
(
new_listpack_bytes
<
old_listpack_bytes
)
{
if
((
lp
=
lp_realloc
(
lp
,
new_listpack_bytes
))
==
NULL
)
return
NULL
;
dst
=
lp
+
poff
;
}
/* Store the entry. */
if
(
newp
)
{
*
newp
=
dst
;
/* In case of deletion, set 'newp' to NULL if the next element is
* the EOF element. */
if
(
!
ele
&&
dst
[
0
]
==
LP_EOF
)
*
newp
=
NULL
;
}
if
(
ele
)
{
if
(
enctype
==
LP_ENCODING_INT
)
{
memcpy
(
dst
,
intenc
,
enclen
);
}
else
{
lpEncodeString
(
dst
,
ele
,
size
);
}
dst
+=
enclen
;
memcpy
(
dst
,
backlen
,
backlen_size
);
dst
+=
backlen_size
;
}
/* Update header. */
if
(
where
!=
LP_REPLACE
||
ele
==
NULL
)
{
uint32_t
num_elements
=
lpGetNumElements
(
lp
);
if
(
num_elements
!=
LP_HDR_NUMELE_UNKNOWN
)
{
if
(
ele
)
lpSetNumElements
(
lp
,
num_elements
+
1
);
else
lpSetNumElements
(
lp
,
num_elements
-
1
);
}
}
lpSetTotalBytes
(
lp
,
new_listpack_bytes
);
return
lp
;
}
/* Append the specified element 'ele' of lenght 'len' at the end of the
* listpack. It is implemented in terms of lpInsert(), so the return value is
* the same as lpInsert(). */
unsigned
char
*
lpAppend
(
unsigned
char
*
lp
,
unsigned
char
*
ele
,
uint32_t
size
)
{
uint64_t
listpack_bytes
=
lpGetTotalBytes
(
lp
);
unsigned
char
*
eofptr
=
lp
+
listpack_bytes
-
1
;
return
lpInsert
(
lp
,
ele
,
size
,
eofptr
,
LP_BEFORE
,
NULL
);
}
/* Remove the element pointed by 'p', and return the resulting listpack.
* If 'newp' is not NULL, the next element pointer (to the right of the
* deleted one) is returned by reference. If the deleted element was the
* last one, '*newp' is set to NULL. */
unsigned
char
*
lpDelete
(
unsigned
char
*
lp
,
unsigned
char
*
p
,
unsigned
char
**
newp
)
{
return
lpInsert
(
lp
,
NULL
,
0
,
p
,
LP_REPLACE
,
newp
);
}
/* Return the total number of bytes the listpack is composed of. */
uint32_t
lpBytes
(
unsigned
char
*
lp
)
{
return
lpGetTotalBytes
(
lp
);
}
/* Seek the specified element and returns the pointer to the seeked element.
* Positive indexes specify the zero-based element to seek from the head to
* the tail, negative indexes specify elements starting from the tail, where
* -1 means the last element, -2 the penultimate and so forth. If the index
* is out of range, NULL is returned. */
unsigned
char
*
lpSeek
(
unsigned
char
*
lp
,
long
index
)
{
int
forward
=
1
;
/* Seek forward by default. */
/* We want to seek from left to right or the other way around
* depending on the listpack length and the element position.
* However if the listpack length cannot be obtained in constant time,
* we always seek from left to right. */
uint32_t
numele
=
lpGetNumElements
(
lp
);
if
(
numele
!=
LP_HDR_NUMELE_UNKNOWN
)
{
if
(
index
<
0
)
index
=
(
long
)
numele
+
index
;
if
(
index
<
0
)
return
NULL
;
/* Index still < 0 means out of range. */
if
(
index
>=
numele
)
return
NULL
;
/* Out of range the other side. */
/* We want to scan right-to-left if the element we are looking for
* is past the half of the listpack. */
if
(
index
>
numele
/
2
)
{
forward
=
0
;
/* Left to right scanning always expects a negative index. Convert
* our index to negative form. */
index
-=
numele
;
}
}
else
{
/* If the listpack length is unspecified, for negative indexes we
* want to always scan left-to-right. */
if
(
index
<
0
)
forward
=
0
;
}
/* Forward and backward scanning is trivially based on lpNext()/lpPrev(). */
if
(
forward
)
{
unsigned
char
*
ele
=
lpFirst
(
lp
);
while
(
index
>
0
&&
ele
)
{
ele
=
lpNext
(
lp
,
ele
);
index
--
;
}
return
ele
;
}
else
{
unsigned
char
*
ele
=
lpLast
(
lp
);
while
(
index
<
-
1
&&
ele
)
{
ele
=
lpPrev
(
lp
,
ele
);
index
++
;
}
return
ele
;
}
}
src/listpack.h
0 → 100644
View file @
8b511219
/* Listpack -- A lists of strings serialization format
*
* This file implements the specification you can find at:
*
* https://github.com/antirez/listpack
*
* Copyright (c) 2017, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __LISTPACK_H
#define __LISTPACK_H
#include <stdint.h>
#define LP_INTBUF_SIZE 21
/* 20 digits of -2^63 + 1 null term = 21. */
/* lpInsert() where argument possible values: */
#define LP_BEFORE 0
#define LP_AFTER 1
#define LP_REPLACE 2
unsigned
char
*
lpNew
(
void
);
void
lpFree
(
unsigned
char
*
lp
);
unsigned
char
*
lpInsert
(
unsigned
char
*
lp
,
unsigned
char
*
ele
,
uint32_t
size
,
unsigned
char
*
p
,
int
where
,
unsigned
char
**
newp
);
unsigned
char
*
lpAppend
(
unsigned
char
*
lp
,
unsigned
char
*
ele
,
uint32_t
size
);
unsigned
char
*
lpDelete
(
unsigned
char
*
lp
,
unsigned
char
*
p
,
unsigned
char
**
newp
);
uint32_t
lpLength
(
unsigned
char
*
lp
);
unsigned
char
*
lpGet
(
unsigned
char
*
p
,
int64_t
*
count
,
unsigned
char
*
intbuf
);
unsigned
char
*
lpFirst
(
unsigned
char
*
lp
);
unsigned
char
*
lpLast
(
unsigned
char
*
lp
);
unsigned
char
*
lpNext
(
unsigned
char
*
lp
,
unsigned
char
*
p
);
unsigned
char
*
lpPrev
(
unsigned
char
*
lp
,
unsigned
char
*
p
);
uint32_t
lpBytes
(
unsigned
char
*
lp
);
unsigned
char
*
lpSeek
(
unsigned
char
*
lp
,
long
index
);
#endif
src/listpack_malloc.h
0 → 100644
View file @
8b511219
/* Listpack -- A lists of strings serialization format
* https://github.com/antirez/listpack
*
* Copyright (c) 2017, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/* Allocator selection.
*
* This file is used in order to change the Rax allocator at compile time.
* Just define the following defines to what you want to use. Also add
* the include of your alternate allocator if needed (not needed in order
* to use the default libc allocator). */
#ifndef LISTPACK_ALLOC_H
#define LISTPACK_ALLOC_H
#include "zmalloc.h"
#define lp_malloc zmalloc
#define lp_realloc zrealloc
#define lp_free zfree
#endif
src/lzfP.h
View file @
8b511219
...
@@ -79,7 +79,11 @@
...
@@ -79,7 +79,11 @@
* Unconditionally aligning does not cost very much, so do it if unsure
* Unconditionally aligning does not cost very much, so do it if unsure
*/
*/
#ifndef STRICT_ALIGN
#ifndef STRICT_ALIGN
# define STRICT_ALIGN !(defined(__i386) || defined (__amd64))
# if !(defined(__i386) || defined (__amd64))
# define STRICT_ALIGN 1
# else
# define STRICT_ALIGN 0
# endif
#endif
#endif
/*
/*
...
...
src/networking.c
View file @
8b511219
...
@@ -67,6 +67,16 @@ int listMatchObjects(void *a, void *b) {
...
@@ -67,6 +67,16 @@ int listMatchObjects(void *a, void *b) {
return
equalStringObjects
(
a
,
b
);
return
equalStringObjects
(
a
,
b
);
}
}
/* This function links the client to the global linked list of clients.
* unlinkClient() does the opposite, among other things. */
void
linkClient
(
client
*
c
)
{
listAddNodeTail
(
server
.
clients
,
c
);
/* Note that we remember the linked list node where the client is stored,
* this way removing the client in unlinkClient() will not require
* a linear scan, but just a constant time operation. */
c
->
client_list_node
=
listLast
(
server
.
clients
);
}
client
*
createClient
(
int
fd
)
{
client
*
createClient
(
int
fd
)
{
client
*
c
=
zmalloc
(
sizeof
(
client
));
client
*
c
=
zmalloc
(
sizeof
(
client
));
...
@@ -124,8 +134,9 @@ client *createClient(int fd) {
...
@@ -124,8 +134,9 @@ client *createClient(int fd) {
listSetDupMethod
(
c
->
reply
,
dupClientReplyValue
);
listSetDupMethod
(
c
->
reply
,
dupClientReplyValue
);
c
->
btype
=
BLOCKED_NONE
;
c
->
btype
=
BLOCKED_NONE
;
c
->
bpop
.
timeout
=
0
;
c
->
bpop
.
timeout
=
0
;
c
->
bpop
.
keys
=
dictCreate
(
&
objectKeyPointerValueDictType
,
NULL
);
c
->
bpop
.
keys
=
dictCreate
(
&
objectKey
Heap
PointerValueDictType
,
NULL
);
c
->
bpop
.
target
=
NULL
;
c
->
bpop
.
target
=
NULL
;
c
->
bpop
.
xread_group
=
NULL
;
c
->
bpop
.
numreplicas
=
0
;
c
->
bpop
.
numreplicas
=
0
;
c
->
bpop
.
reploffset
=
0
;
c
->
bpop
.
reploffset
=
0
;
c
->
woff
=
0
;
c
->
woff
=
0
;
...
@@ -133,9 +144,10 @@ client *createClient(int fd) {
...
@@ -133,9 +144,10 @@ client *createClient(int fd) {
c
->
pubsub_channels
=
dictCreate
(
&
objectKeyPointerValueDictType
,
NULL
);
c
->
pubsub_channels
=
dictCreate
(
&
objectKeyPointerValueDictType
,
NULL
);
c
->
pubsub_patterns
=
listCreate
();
c
->
pubsub_patterns
=
listCreate
();
c
->
peerid
=
NULL
;
c
->
peerid
=
NULL
;
c
->
client_list_node
=
NULL
;
listSetFreeMethod
(
c
->
pubsub_patterns
,
decrRefCountVoid
);
listSetFreeMethod
(
c
->
pubsub_patterns
,
decrRefCountVoid
);
listSetMatchMethod
(
c
->
pubsub_patterns
,
listMatchObjects
);
listSetMatchMethod
(
c
->
pubsub_patterns
,
listMatchObjects
);
if
(
fd
!=
-
1
)
li
stAddNodeTail
(
server
.
c
lient
s
,
c
);
if
(
fd
!=
-
1
)
li
nkC
lient
(
c
);
initClientMultiState
(
c
);
initClientMultiState
(
c
);
return
c
;
return
c
;
}
}
...
@@ -767,9 +779,10 @@ void unlinkClient(client *c) {
...
@@ -767,9 +779,10 @@ void unlinkClient(client *c) {
* fd is already set to -1. */
* fd is already set to -1. */
if
(
c
->
fd
!=
-
1
)
{
if
(
c
->
fd
!=
-
1
)
{
/* Remove from the list of active clients. */
/* Remove from the list of active clients. */
ln
=
listSearchKey
(
server
.
clients
,
c
);
if
(
c
->
client_list_node
)
{
serverAssert
(
ln
!=
NULL
);
listDelNode
(
server
.
clients
,
c
->
client_list_node
);
listDelNode
(
server
.
clients
,
ln
);
c
->
client_list_node
=
NULL
;
}
/* Unregister async I/O handlers and close the socket. */
/* Unregister async I/O handlers and close the socket. */
aeDeleteFileEvent
(
server
.
el
,
c
->
fd
,
AE_READABLE
);
aeDeleteFileEvent
(
server
.
el
,
c
->
fd
,
AE_READABLE
);
...
...
Prev
1
2
3
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment