Commit 4aab50ac authored by rojingeorge's avatar rojingeorge
Browse files

Merge remote-tracking branch 'refs/remotes/antirez/unstable' into unstable

parents 646c958b f60aa4de
......@@ -12,7 +12,7 @@ each source file that you contribute.
PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
bugs in the Github issues system. We'll be very happy to help you and provide
all the support Reddit sub:
all the support at the Reddit sub:
http://reddit.com/r/redis
......@@ -24,7 +24,7 @@ each source file that you contribute.
1. If it is a major feature or a semantical change, please post it as a new submission in r/redis on Reddit at http://reddit.com/r/redis. Try to be passionate about why the feature is needed, make users upvote your proposal to gain traction and so forth. Read feedbacks about the community. But in this first step **please don't write code yet**.
2. If in step 1 you get an acknowledge from the project leaders, use the
2. If in step 1 you get an acknowledgment from the project leaders, use the
following procedure to submit a patch:
a. Fork Redis on github ( http://help.github.com/fork-a-repo/ )
......
......@@ -39,7 +39,7 @@ You can run a 32 bit Redis binary using:
% make 32bit
After building Redis is a good idea to test it, using:
After building Redis, it is a good idea to test it using:
% make test
......@@ -47,8 +47,8 @@ Fixing build problems with dependencies or cached build options
---------
Redis has some dependencies which are included into the `deps` directory.
`make` does not rebuild dependencies automatically, even if something in the
source code of dependencies is changed.
`make` does not automatically rebuild dependencies even if something in
the source code of dependencies changes.
When you update the source code with `git pull` or when code inside the
dependencies tree is modified in any other way, make sure to use the following
......@@ -109,14 +109,14 @@ To run Redis with the default configuration just type:
% cd src
% ./redis-server
If you want to provide your redis.conf, you have to run it using an additional
parameter (the path of the configuration file):
% cd src
% ./redis-server /path/to/redis.conf
It is possible to alter the Redis configuration passing parameters directly
It is possible to alter the Redis configuration by passing parameters directly
as options using the command line. Examples:
% ./redis-server --port 9999 --slaveof 127.0.0.1 6379
......@@ -174,7 +174,7 @@ You'll be able to stop and start Redis using the script named
`/etc/init.d/redis_<portnumber>`, for instance `/etc/init.d/redis_6379`.
Code contributions
---
-----------------
Note: by contributing code to the Redis project in any form, including sending
a pull request via Github, a code fragment or patch via private email or
......@@ -196,8 +196,8 @@ or you just untarred the Redis distribution tar ball. In both the cases
you are basically one step away from the source code, so here we explain
the Redis source code layout, what is in each file as a general idea, the
most important functions and structures inside the Redis server and so forth.
We keep all the discussion at an high level without digging into the details
since this document would be huge otherwise, and our code base changes
We keep all the discussion at a high level without digging into the details
since this document would be huge otherwise and our code base changes
continuously, but a general idea should be a good starting point to
understand more. Moreover most of the code is heavily commented and easy
to follow.
......@@ -206,17 +206,17 @@ Source code layout
---
The Redis root directory just contains this README, the Makefile which
actually calls the real Makefile inside the `src` directory, an example
configuration for Redis and Sentinel. Finally you can find a few shell
calls the real Makefile inside the `src` directory and an example
configuration for Redis and Sentinel. You can find a few shell
scripts that are used in order to execute the Redis, Redis Cluster and
Redis Sentinel unit tests, which are implemented inside the `tests`
directory.
Inside the root directory the are the following important directories:
Inside the root are the following important directories:
* `src`: contains the Redis implementation, written in C.
* `tests`: contains the unit tests, implemented in Tcl.
* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory, your system needs to provide just the `libc`, a POSIX compatible interface, and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `anitrez/redis`. an exception to this rule is `deps/geohash-int` which is the low level geocoding library used by Redis: it originated from a different project, but at this point it diverged so much that it is developed as a separated entity directly inside the Redis repository.
* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory; your system just needs to provide `libc`, a POSIX compatible interface and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `anitrez/redis`. An exception to this rule is `deps/geohash-int` which is the low level geocoding library used by Redis: it originated from a different project, but at this point it diverged so much that it is developed as a separated entity directly inside the Redis repository.
There are a few more directories but they are not very important for our goals
here. We'll focus mostly on `src`, where the Redis implementation is contained,
......@@ -225,34 +225,34 @@ exposed is the logical one to follow in order to disclose different layers
of complexity incrementally.
Note: lately Redis was refactored quite a bit. Function names and file
names changed, so you may find that this documentation reflects the
names have been changed, so you may find that this documentation reflects the
`unstable` branch more closely. For instance in Redis 3.0 the `server.c`
and `server.h` files were renamed `redis.c` and `redis.h`. However the overall
and `server.h` files were named to `redis.c` and `redis.h`. However the overall
structure is the same. Keep in mind that all the new developments and pull
requests should be performed against the `unstable` branch.
server.h
---
The simplest way to understand how a program works, is to understand the
The simplest way to understand how a program works is to understand the
data structures it uses. So we'll start from the main header file of
Redis, which is `server.h`.
All the server configuration and in general all the shared state is
defined in a global structure called `server`, of type `struct redisServer`.
A few important fields in this structure:
A few important fields in this structure are:
* `server.db` is an array of Redis databases, where data is stored.
* `server.commands` is the command table.
* `server.clients` is a linked list of clients connected to the server.
* `server.master` is a special client, the master, if the instance is a slave.
There are tons of other fields, most fields are commented directly inside
There are tons of other fields. Most fields are commented directly inside
the structure definition.
Another important Redis data structure is the one defining a client.
In the past it was called `redisClient`, now just `client`. The structure
has many fields, here we'll show just the main ones:
has many fields, here we'll just show the main ones:
struct client {
int fd;
......@@ -270,7 +270,7 @@ The client structure defines a *connected client*:
* The `fd` field is the client socket file descriptor.
* `argc` and `argv` are populated with the command the client is executing, so that functions implementing a given Redis command can read the arguments.
* `querybuf` accumulates the requests from the client, which are parsed by the Redis server according to the Redis protocol, and executed calling the implementations of the commands the client is executing.
* `querybuf` accumulates the requests from the client, which are parsed by the Redis server according to the Redis protocol and executed by calling the implementations of the commands the client is executing.
* `reply` and `buf` are dynamic and static buffers that accumulate the replies the server sends to the client. These buffers are incrementally written to the socket as soon as the file descriptor is writable.
As you can see in the client structure above, arguments in a command
......@@ -288,16 +288,16 @@ structure, which defines a *Redis object*:
Basically this structure can represent all the basic Redis data types like
strings, lists, sets, sorted sets and so forth. The interesting thing is that
it has a `type` field, so that it is possible to know what type a given
object is, and a `refcount`, so that the same object can be referenced
object has, and a `refcount`, so that the same object can be referenced
in multiple places without allocating it multiple times. Finally the `ptr`
field points to the actual representation of the object, that may vary
field points to the actual representation of the object, which might vary
even for the same type, depending on the `encoding` used.
Redis objects are used extensively in the Redis internals, however in order
to avoid the overhead of indirect accesses, recently in many places
we just use plain dynamic strings not wrapped inside a Redis object.
sever.c
server.c
---
This is the entry point of the Redis server, where the `main()` function
......@@ -306,7 +306,7 @@ the Redis server.
* `initServerConfig()` setups the default values of the `server` structure.
* `initServer()` allocates the data structures needed to operate, setup the listening socket, and so forth.
* `aeMain()` enters the event loop listening for new connections.
* `aeMain()` starts the event loop which listens for new connections.
There are two special functions called periodically by the event loop:
......@@ -328,7 +328,7 @@ This file defines all the I/O functions with clients, masters and slaves
* `createClient()` allocates and initializes a new client.
* the `addReply*()` family of functions are used by commands implementations in order to append data to the client structure, that will be transmitted to the client as a reply for a given command executed.
* `writeToClient()` transmits the data pending in the output buffers to the client, and is called by the *writable event handler* `sendReplyToClient()`.
* `writeToClient()` transmits the data pending in the output buffers to the client and is called by the *writable event handler* `sendReplyToClient()`.
* `readQueryFromClient()` is the *readable event handler* and accumulates data from read from the client into the query buffer.
* `processInputBuffer()` is the entry point in order to parse the client query buffer according to the Redis protocol. Once commands are ready to be processed, it calls `processCommand()` which is defined inside `server.c` in order to actually execute the command.
* `freeClient()` deallocates, disconnects and removes a client.
......@@ -439,9 +439,8 @@ There are tons of commands implementations inside th Redis source code
that can serve as examples of actual commands implementations. To write
a few toy commands can be a good exercise to familiarize with the code base.
There are also many other files not described here, but it is useless to
cover everything, we want just to help you with the first steps,
eventually you'll find your way inside the Redis code base :-)
There are also many other files not described here, but it is useless to
cover everything. We want to just help you with the first steps.
Eventually you'll find your way inside the Redis code base :-)
Enjoy!
......@@ -78,7 +78,7 @@ JEMALLOC_LDFLAGS= $(LDFLAGS)
jemalloc: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
cd jemalloc && ./configure --with-lg-quantum=3 --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
cd jemalloc && $(MAKE) CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" lib/libjemalloc.a
.PHONY: jemalloc
......
......@@ -72,7 +72,7 @@ uint8_t geohashEstimateStepsByRadius(double range_meters, double lat) {
/* Frame to valid range. */
if (step < 1) step = 1;
if (step > 26) step = 25;
if (step > 26) step = 26;
return step;
}
......@@ -89,6 +89,8 @@ int geohashBoundingBox(double longitude, double latitude, double radius_meters,
lonr = deg_rad(longitude);
latr = deg_rad(latitude);
if (radius_meters > EARTH_RADIUS_IN_METERS)
radius_meters = EARTH_RADIUS_IN_METERS;
double distance = radius_meters / EARTH_RADIUS_IN_METERS;
double min_latitude = latr - distance;
double max_latitude = latr + distance;
......
......@@ -125,8 +125,9 @@ timeout 0
# Note that to close the connection the double of the time is needed.
# On other kernels the period depends on the kernel configuration.
#
# A reasonable value for this option is 60 seconds.
tcp-keepalive 0
# A reasonable value for this option is 300 seconds, which is the new
# Redis default starting with Redis 3.2.1.
tcp-keepalive 300
################################# GENERAL #####################################
......@@ -154,7 +155,7 @@ supervised no
#
# Creating a pid file is best effort: if Redis is not able to create it
# nothing bad happens, the server will start and run normally.
pidfile /var/run/redis.pid
pidfile /var/run/redis_6379.pid
# Specify the server verbosity level.
# This can be one of:
......
......@@ -65,17 +65,27 @@ ifeq ($(uname_S),SunOS)
FINAL_LIBS+= -ldl -lnsl -lsocket -lresolv -lpthread -lrt
else
ifeq ($(uname_S),Darwin)
# Darwin (nothing to do)
# Darwin
FINAL_LIBS+= -ldl
else
ifeq ($(uname_S),AIX)
# AIX
FINAL_LDFLAGS+= -Wl,-bexpall
FINAL_LIBS+= -pthread -lcrypt -lbsd
FINAL_LIBS+=-ldl -pthread -lcrypt -lbsd
else
ifeq ($(uname_S),OpenBSD)
# OpenBSD
FINAL_LIBS+= -lpthread
else
ifeq ($(uname_S),FreeBSD)
# FreeBSD
FINAL_LIBS+= -lpthread
else
# All the other OSes (notably Linux)
FINAL_LDFLAGS+= -rdynamic
FINAL_LIBS+= -pthread
FINAL_LIBS+=-ldl -pthread
endif
endif
endif
endif
endif
......@@ -95,7 +105,7 @@ endif
ifeq ($(MALLOC),jemalloc)
DEPENDENCY_TARGETS+= jemalloc
FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a -ldl
FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a
endif
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
......
......@@ -29,6 +29,7 @@
*/
#include <sys/select.h>
#include <string.h>
typedef struct aeApiState {
......
......@@ -693,6 +693,7 @@ int loadAppendOnlyFile(char *filename) {
}
/* Run the command in the context of a fake client */
fakeClient->cmd = cmd;
cmd->proc(fakeClient);
/* The fake client should not have a reply */
......@@ -703,6 +704,7 @@ int loadAppendOnlyFile(char *filename) {
/* Clean up. Command code may have changed argv/argc so we use the
* argv/argc of the client instead of the local variables. */
freeFakeClientArgv(fakeClient);
fakeClient->cmd = NULL;
if (server.aof_load_truncated) valid_up_to = ftello(fp);
}
......@@ -983,6 +985,18 @@ int rewriteHashObject(rio *r, robj *key, robj *o) {
return 1;
}
/* Call the module type callback in order to rewrite a data type
* taht is exported by a module and is not handled by Redis itself.
* The function returns 0 on error, 1 on success. */
int rewriteModuleObject(rio *r, robj *key, robj *o) {
RedisModuleIO io;
moduleValue *mv = o->ptr;
moduleType *mt = mv->type;
moduleInitIOContext(io,mt,r);
mt->aof_rewrite(&io,key,mv->value);
return io.error ? 0 : 1;
}
/* This function is called by the child rewriting the AOF file to read
* the difference accumulated from the parent into a buffer, that is
* concatenated at the end of the rewrite. */
......@@ -1075,6 +1089,8 @@ int rewriteAppendOnlyFile(char *filename) {
if (rewriteSortedSetObject(&aof,&key,o) == 0) goto werr;
} else if (o->type == OBJ_HASH) {
if (rewriteHashObject(&aof,&key,o) == 0) goto werr;
} else if (o->type == OBJ_MODULE) {
if (rewriteModuleObject(&aof,&key,o) == 0) goto werr;
} else {
serverPanic("Unknown object type");
}
......
......@@ -215,12 +215,7 @@ void setUnsignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits, uint6
}
void setSignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits, int64_t value) {
uint64_t uv;
if (value >= 0)
uv = value;
else
uv = UINT64_MAX + value + 1;
uint64_t uv = value; /* Casting will add UINT64_MAX + 1 if v is negative. */
setUnsignedBitfield(p,offset,bits,uv);
}
......@@ -239,9 +234,21 @@ uint64_t getUnsignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits) {
}
int64_t getSignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits) {
int64_t value = getUnsignedBitfield(p,offset,bits);
int64_t value;
union {uint64_t u; int64_t i;} conv;
/* Converting from unsigned to signed is undefined when the value does
* not fit, however here we assume two's complement and the original value
* was obtained from signed -> unsigned conversion, so we'll find the
* most significant bit set if the original value was negative.
*
* Note that two's complement is mandatory for exact-width types
* according to the C99 standard. */
conv.u = getUnsignedBitfield(p,offset,bits);
value = conv.i;
/* If the top significant bit is 1, propagate it to all the
* higher bits for two complement representation of signed
* higher bits for two's complement representation of signed
* integers. */
if (value & ((uint64_t)1 << (bits-1)))
value |= ((uint64_t)-1) << bits;
......@@ -299,7 +306,7 @@ int checkUnsignedBitfieldOverflow(uint64_t value, int64_t incr, uint64_t bits, i
handle_wrap:
{
uint64_t mask = ((int64_t)-1) << bits;
uint64_t mask = ((uint64_t)-1) << bits;
uint64_t res = value+incr;
res &= ~mask;
......@@ -342,7 +349,7 @@ int checkSignedBitfieldOverflow(int64_t value, int64_t incr, uint64_t bits, int
handle_wrap:
{
uint64_t mask = ((int64_t)-1) << bits;
uint64_t mask = ((uint64_t)-1) << bits;
uint64_t msb = (uint64_t)1 << (bits-1);
uint64_t a = value, b = incr, c;
c = a+b; /* Perform addition as unsigned so that's defined. */
......@@ -476,6 +483,37 @@ robj *lookupStringForBitCommand(client *c, size_t maxbit) {
return o;
}
/* Return a pointer to the string object content, and stores its length
* in 'len'. The user is required to pass (likely stack allocated) buffer
* 'llbuf' of at least LONG_STR_SIZE bytes. Such a buffer is used in the case
* the object is integer encoded in order to provide the representation
* without usign heap allocation.
*
* The function returns the pointer to the object array of bytes representing
* the string it contains, that may be a pointer to 'llbuf' or to the
* internal object representation. As a side effect 'len' is filled with
* the length of such buffer.
*
* If the source object is NULL the function is guaranteed to return NULL
* and set 'len' to 0. */
unsigned char *getObjectReadOnlyString(robj *o, long *len, char *llbuf) {
serverAssert(o->type == OBJ_STRING);
unsigned char *p = NULL;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (o && o->encoding == OBJ_ENCODING_INT) {
p = (unsigned char*) llbuf;
if (len) *len = ll2string(llbuf,LONG_STR_SIZE,(long)o->ptr);
} else if (o) {
p = (unsigned char*) o->ptr;
if (len) *len = sdslen(o->ptr);
} else {
if (len) *len = 0;
}
return p;
}
/* SETBIT key offset bitvalue */
void setbitCommand(client *c) {
robj *o;
......@@ -721,21 +759,12 @@ void bitcountCommand(client *c) {
robj *o;
long start, end, strlen;
unsigned char *p;
char llbuf[32];
char llbuf[LONG_STR_SIZE];
/* Lookup, check for type, and return 0 for non existing keys. */
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_STRING)) return;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (o->encoding == OBJ_ENCODING_INT) {
p = (unsigned char*) llbuf;
strlen = ll2string(llbuf,sizeof(llbuf),(long)o->ptr);
} else {
p = (unsigned char*) o->ptr;
strlen = sdslen(o->ptr);
}
p = getObjectReadOnlyString(o,&strlen,llbuf);
/* Parse start/end range if any. */
if (c->argc == 4) {
......@@ -744,6 +773,10 @@ void bitcountCommand(client *c) {
if (getLongFromObjectOrReply(c,c->argv[3],&end,NULL) != C_OK)
return;
/* Convert negative indexes */
if (start < 0 && end < 0 && start > end) {
addReply(c,shared.czero);
return;
}
if (start < 0) start = strlen+start;
if (end < 0) end = strlen+end;
if (start < 0) start = 0;
......@@ -775,7 +808,7 @@ void bitposCommand(client *c) {
robj *o;
long bit, start, end, strlen;
unsigned char *p;
char llbuf[32];
char llbuf[LONG_STR_SIZE];
int end_given = 0;
/* Parse the bit argument to understand what we are looking for, set
......@@ -795,16 +828,7 @@ void bitposCommand(client *c) {
return;
}
if (checkType(c,o,OBJ_STRING)) return;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (o->encoding == OBJ_ENCODING_INT) {
p = (unsigned char*) llbuf;
strlen = ll2string(llbuf,sizeof(llbuf),(long)o->ptr);
} else {
p = (unsigned char*) o->ptr;
strlen = sdslen(o->ptr);
}
p = getObjectReadOnlyString(o,&strlen,llbuf);
/* Parse start/end range if any. */
if (c->argc == 4 || c->argc == 5) {
......@@ -882,6 +906,8 @@ void bitfieldCommand(client *c) {
int j, numops = 0, changes = 0;
struct bitfieldOp *ops = NULL; /* Array of ops to execute at end. */
int owtype = BFOVERFLOW_WRAP; /* Overflow type. */
int readonly = 1;
long higest_write_offset = 0;
for (j = 2; j < c->argc; j++) {
int remargs = c->argc-j-1; /* Remaining args other than current. */
......@@ -929,8 +955,10 @@ void bitfieldCommand(client *c) {
return;
}
/* INCRBY and SET require another argument. */
if (opcode != BITFIELDOP_GET) {
readonly = 0;
higest_write_offset = bitoffset + bits - 1;
/* INCRBY and SET require another argument. */
if (getLongLongFromObjectOrReply(c,c->argv[j+3],&i64,NULL) != C_OK){
zfree(ops);
return;
......@@ -950,6 +978,18 @@ void bitfieldCommand(client *c) {
j += 3 - (opcode == BITFIELDOP_GET);
}
if (readonly) {
/* Lookup for read is ok if key doesn't exit, but errors
* if it's not a string. */
o = lookupKeyRead(c->db,c->argv[1]);
if (o != NULL && checkType(c,o,OBJ_STRING)) return;
} else {
/* Lookup by making room up to the farest bit reached by
* this operation. */
if ((o = lookupStringForBitCommand(c,
higest_write_offset)) == NULL) return;
}
addReplyMultiBulkLen(c,numops);
/* Actually process the operations. */
......@@ -964,11 +1004,6 @@ void bitfieldCommand(client *c) {
* for simplicity. SET return value is the previous value so
* we need fetch & store as well. */
/* Lookup by making room up to the farest bit reached by
* this operation. */
if ((o = lookupStringForBitCommand(c,
thisop->offset + (thisop->bits-1))) == NULL) return;
/* We need two different but very similar code paths for signed
* and unsigned operations, since the set of functions to get/set
* the integers and the used variables types are different. */
......@@ -1035,20 +1070,23 @@ void bitfieldCommand(client *c) {
changes++;
} else {
/* GET */
o = lookupKeyRead(c->db,c->argv[1]);
size_t olen = (o == NULL) ? 0 : sdslen(o->ptr);
unsigned char buf[9];
long strlen = 0;
unsigned char *src = NULL;
char llbuf[LONG_STR_SIZE];
if (o != NULL)
src = getObjectReadOnlyString(o,&strlen,llbuf);
/* For GET we use a trick: before executing the operation
* copy up to 9 bytes to a local buffer, so that we can easily
* execute up to 64 bit operations that are at actual string
* object boundaries. */
memset(buf,0,9);
unsigned char *src = o ? o->ptr : NULL;
int i;
size_t byte = thisop->offset >> 3;
for (i = 0; i < 9; i++) {
if (src == NULL || i+byte >= olen) break;
if (src == NULL || i+byte >= (size_t)strlen) break;
buf[i] = src[i+byte];
}
......
......@@ -4535,7 +4535,7 @@ int verifyDumpPayload(unsigned char *p, size_t len) {
/* Verify RDB version */
rdbver = (footer[1] << 8) | footer[0];
if (rdbver != RDB_VERSION) return C_ERR;
if (rdbver > RDB_VERSION) return C_ERR;
/* Verify CRC64 */
crc = crc64(0,p,len-8);
......
......@@ -153,6 +153,20 @@ void resetServerSaveParams(void) {
server.saveparamslen = 0;
}
void queueLoadModule(sds path, sds *argv, int argc) {
int i;
struct moduleLoadQueueEntry *loadmod;
loadmod = zmalloc(sizeof(struct moduleLoadQueueEntry));
loadmod->argv = zmalloc(sizeof(robj*)*argc);
loadmod->path = sdsnew(path);
loadmod->argc = argc;
for (i = 0; i < argc; i++) {
loadmod->argv[i] = createRawStringObject(argv[i],sdslen(argv[i]));
}
listAddNodeTail(server.loadmodule_queue,loadmod);
}
void loadServerConfigFromString(char *config) {
char *err = NULL;
int linenum = 0, totlines, i;
......@@ -632,8 +646,8 @@ void loadServerConfigFromString(char *config) {
"Allowed values: 'upstart', 'systemd', 'auto', or 'no'";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"loadmodule") && argc == 2) {
listAddNodeTail(server.loadmodule_queue,sdsnew(argv[1]));
} else if (!strcasecmp(argv[0],"loadmodule") && argc >= 2) {
queueLoadModule(argv[1],&argv[2],argc-2);
} else if (!strcasecmp(argv[0],"sentinel")) {
/* argc == 1 is handled by main() as we need to enter the sentinel
* mode ASAP. */
......@@ -719,7 +733,7 @@ void loadServerConfig(char *filename, char *options) {
#define config_set_numerical_field(_name,_var,min,max) \
} else if (!strcasecmp(c->argv[2]->ptr,_name)) { \
if (getLongLongFromObject(o,&ll) == C_ERR || ll < 0) goto badfmt; \
if (getLongLongFromObject(o,&ll) == C_ERR) goto badfmt; \
if (min != LLONG_MIN && ll < min) goto badfmt; \
if (max != LLONG_MAX && ll > max) goto badfmt; \
_var = ll;
......@@ -950,9 +964,9 @@ void configSetCommand(client *c) {
} config_set_numerical_field(
"hash-max-ziplist-value",server.hash_max_ziplist_value,0,LLONG_MAX) {
} config_set_numerical_field(
"list-max-ziplist-size",server.list_max_ziplist_size,0,LLONG_MAX) {
"list-max-ziplist-size",server.list_max_ziplist_size,INT_MIN,INT_MAX) {
} config_set_numerical_field(
"list-compress-depth",server.list_compress_depth,0,LLONG_MAX) {
"list-compress-depth",server.list_compress_depth,0,INT_MAX) {
} config_set_numerical_field(
"set-max-intset-entries",server.set_max_intset_entries,0,LLONG_MAX) {
} config_set_numerical_field(
......
......@@ -38,7 +38,10 @@
* C-level DB API
*----------------------------------------------------------------------------*/
robj *lookupKey(redisDb *db, robj *key) {
/* Low level key lookup API, not actually called directly from commands
* implementations that should instead rely on lookupKeyRead(),
* lookupKeyWrite() and lookupKeyReadWithFlags(). */
robj *lookupKey(redisDb *db, robj *key, int flags) {
dictEntry *de = dictFind(db->dict,key->ptr);
if (de) {
robj *val = dictGetVal(de);
......@@ -46,15 +49,40 @@ robj *lookupKey(redisDb *db, robj *key) {
/* Update the access time for the ageing algorithm.
* Don't do it if we have a saving child, as this will trigger
* a copy on write madness. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1)
if (server.rdb_child_pid == -1 &&
server.aof_child_pid == -1 &&
!(flags & LOOKUP_NOTOUCH))
{
val->lru = LRU_CLOCK();
}
return val;
} else {
return NULL;
}
}
robj *lookupKeyRead(redisDb *db, robj *key) {
/* Lookup a key for read operations, or return NULL if the key is not found
* in the specified DB.
*
* As a side effect of calling this function:
* 1. A key gets expired if it reached it's TTL.
* 2. The key last access time is updated.
* 3. The global keys hits/misses stats are updated (reported in INFO).
*
* This API should not be used when we write to the key after obtaining
* the object linked to the key, but only for read only operations.
*
* Flags change the behavior of this command:
*
* LOOKUP_NONE (or zero): no special flags are passed.
* LOOKUP_NOTOUCH: don't alter the last access time of the key.
*
* Note: this function also returns NULL is the key is logically expired
* but still existing, in case this is a slave, since this API is called only
* for read operations. Even if the key expiry is master-driven, we can
* correctly report a key is expired on slaves even if the master is lagging
* expiring our key via DELs in the replication link. */
robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
robj *val;
if (expireIfNeeded(db,key) == 1) {
......@@ -83,7 +111,7 @@ robj *lookupKeyRead(redisDb *db, robj *key) {
return NULL;
}
}
val = lookupKey(db,key);
val = lookupKey(db,key,flags);
if (val == NULL)
server.stat_keyspace_misses++;
else
......@@ -91,9 +119,20 @@ robj *lookupKeyRead(redisDb *db, robj *key) {
return val;
}
/* Like lookupKeyReadWithFlags(), but does not use any flag, which is the
* common case. */
robj *lookupKeyRead(redisDb *db, robj *key) {
return lookupKeyReadWithFlags(db,key,LOOKUP_NONE);
}
/* Lookup a key for write operations, and as a side effect, if needed, expires
* the key if its TTL is reached.
*
* Returns the linked value object if the key exists or NULL if the key
* does not exist in the specified DB. */
robj *lookupKeyWrite(redisDb *db, robj *key) {
expireIfNeeded(db,key);
return lookupKey(db,key);
return lookupKey(db,key,LOOKUP_NONE);
}
robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply) {
......@@ -721,7 +760,7 @@ void typeCommand(client *c) {
robj *o;
char *type;
o = lookupKeyRead(c->db,c->argv[1]);
o = lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH);
if (o == NULL) {
type = "none";
} else {
......@@ -731,6 +770,10 @@ void typeCommand(client *c) {
case OBJ_SET: type = "set"; break;
case OBJ_ZSET: type = "zset"; break;
case OBJ_HASH: type = "hash"; break;
case OBJ_MODULE: {
moduleValue *mv = o->ptr;
type = mv->type->name;
}; break;
default: type = "unknown"; break;
}
}
......@@ -1045,7 +1088,7 @@ void ttlGenericCommand(client *c, int output_ms) {
long long expire, ttl = -1;
/* If the key does not exist at all, return -2 */
if (lookupKeyRead(c->db,c->argv[1]) == NULL) {
if (lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH) == NULL) {
addReplyLongLong(c,-2);
return;
}
......@@ -1087,6 +1130,14 @@ void persistCommand(client *c) {
}
}
/* TOUCH key1 [key2 key3 ... keyN] */
void touchCommand(client *c) {
int touched = 0;
for (int j = 1; j < c->argc; j++)
if (lookupKeyRead(c->db,c->argv[j]) != NULL) touched++;
addReplyLongLong(c,touched);
}
/* -----------------------------------------------------------------------------
* API to get key arguments from commands
* ---------------------------------------------------------------------------*/
......
......@@ -550,7 +550,7 @@ void debugCommand(client *c) {
/* =========================== Crash handling ============================== */
void _serverAssert(char *estr, char *file, int line) {
void _serverAssert(const char *estr, const char *file, int line) {
bugReportStart();
serverLog(LL_WARNING,"=== ASSERTION FAILED ===");
serverLog(LL_WARNING,"==> %s:%d '%s' is not true",file,line,estr);
......@@ -563,7 +563,7 @@ void _serverAssert(char *estr, char *file, int line) {
*((char*)-1) = 'x';
}
void _serverAssertPrintClientInfo(client *c) {
void _serverAssertPrintClientInfo(const client *c) {
int j;
bugReportStart();
......@@ -587,7 +587,7 @@ void _serverAssertPrintClientInfo(client *c) {
}
}
void serverLogObjectDebugInfo(robj *o) {
void serverLogObjectDebugInfo(const robj *o) {
serverLog(LL_WARNING,"Object type: %d", o->type);
serverLog(LL_WARNING,"Object encoding: %d", o->encoding);
serverLog(LL_WARNING,"Object refcount: %d", o->refcount);
......@@ -607,23 +607,23 @@ void serverLogObjectDebugInfo(robj *o) {
} else if (o->type == OBJ_ZSET) {
serverLog(LL_WARNING,"Sorted set size: %d", (int) zsetLength(o));
if (o->encoding == OBJ_ENCODING_SKIPLIST)
serverLog(LL_WARNING,"Skiplist level: %d", (int) ((zset*)o->ptr)->zsl->level);
serverLog(LL_WARNING,"Skiplist level: %d", (int) ((const zset*)o->ptr)->zsl->level);
}
}
void _serverAssertPrintObject(robj *o) {
void _serverAssertPrintObject(const robj *o) {
bugReportStart();
serverLog(LL_WARNING,"=== ASSERTION FAILED OBJECT CONTEXT ===");
serverLogObjectDebugInfo(o);
}
void _serverAssertWithInfo(client *c, robj *o, char *estr, char *file, int line) {
void _serverAssertWithInfo(const client *c, const robj *o, const char *estr, const char *file, int line) {
if (c) _serverAssertPrintClientInfo(c);
if (o) _serverAssertPrintObject(o);
_serverAssert(estr,file,line);
}
void _serverPanic(char *msg, char *file, int line) {
void _serverPanic(const char *msg, const char *file, int line) {
bugReportStart();
serverLog(LL_WARNING,"------------------------------------------------");
serverLog(LL_WARNING,"!!! Software Failure. Press left mouse button to continue");
......
......@@ -156,9 +156,13 @@ double extractDistanceOrReply(client *c, robj **argv,
return -1;
}
if (distance < 0) {
addReplyError(c,"radius cannot be negative");
return -1;
}
double to_meters = extractUnitOrReply(c,argv[1]);
if (to_meters < 0) {
addReplyError(c,"radius cannot be negative");
return -1;
}
......
......@@ -52,6 +52,11 @@ struct commandHelp {
"Count set bits in a string",
1,
"2.6.0" },
{ "BITFIELD",
"key [GET type offset] [SET type offset value] [INCRBY type offset increment] [OVERFLOW WRAP|SAT|FAIL]",
"Perform arbitrary bitfield integer operations on strings",
1,
"3.2.0" },
{ "BITOP",
"operation destkey key [key ...]",
"Perform bitwise operations between strings",
......@@ -326,32 +331,32 @@ struct commandHelp {
"key longitude latitude member [longitude latitude member ...]",
"Add one or more geospatial items in the geospatial index represented using a sorted set",
13,
"" },
"3.2.0" },
{ "GEODIST",
"key member1 member2 [unit]",
"Returns the distance between two members of a geospatial index",
13,
"" },
"3.2.0" },
{ "GEOHASH",
"key member [member ...]",
"Returns members of a geospatial index as standard geohash strings",
13,
"" },
"3.2.0" },
{ "GEOPOS",
"key member [member ...]",
"Returns longitude and latitude of members of a geospatial index",
13,
"" },
"3.2.0" },
{ "GEORADIUS",
"key longitude latitude radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count] [ASC|DESC]",
"key longitude latitude radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count] [ASC|DESC] [STORE key] [STOREDIST key]",
"Query a sorted set representing a geospatial index to fetch members matching a given maximum distance from a point",
13,
"" },
"3.2.0" },
{ "GEORADIUSBYMEMBER",
"key member radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count] [ASC|DESC]",
"key member radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count] [ASC|DESC] [STORE key] [STOREDIST key]",
"Query a sorted set representing a geospatial index to fetch members matching a given maximum distance from a member",
13,
"" },
"3.2.0" },
{ "GET",
"key",
"Get the value of a key",
......
......@@ -272,7 +272,7 @@ uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value) {
}
/* Return intset length */
uint32_t intsetLen(intset *is) {
uint32_t intsetLen(const intset *is) {
return intrev32ifbe(is->length);
}
......
......@@ -44,7 +44,7 @@ intset *intsetRemove(intset *is, int64_t value, int *success);
uint8_t intsetFind(intset *is, int64_t value);
int64_t intsetRandom(intset *is);
uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value);
uint32_t intsetLen(intset *is);
uint32_t intsetLen(const intset *is);
size_t intsetBlobLen(intset *is);
#ifdef REDIS_TEST
......
This diff is collapsed.
# Modules API reference
## `RM_Alloc`
void *RM_Alloc(size_t bytes);
Use like malloc(). Memory allocated with this function is reported in
Redis INFO memory, used for keys eviction according to maxmemory settings
and in general is taken into account as memory allocated by Redis.
You should avoid to use malloc().
## `RM_Realloc`
void* RM_Realloc(void *ptr, size_t bytes);
Use like realloc() for memory obtained with `RedisModule_Alloc()`.
## `RM_Free`
void RM_Free(void *ptr);
Use like free() for memory obtained by `RedisModule_Alloc()` and
`RedisModule_Realloc()`. However you should never try to free with
`RedisModule_Free()` memory allocated with malloc() inside your module.
## `RM_Strdup`
char *RM_Strdup(const char *str);
Like strdup() but returns memory allocated with `RedisModule_Alloc()`.
## `RM_PoolAlloc`
void *RM_PoolAlloc(RedisModuleCtx *ctx, size_t bytes);
Return heap allocated memory that will be freed automatically when the
module callback function returns. Mostly suitable for small allocations
that are short living and must be released when the callback returns
anyway. The returned memory is aligned to the architecture word size
if at least word size bytes are requested, otherwise it is just
aligned to the next power of two, so for example a 3 bytes request is
4 bytes aligned while a 2 bytes request is 2 bytes aligned.
There is no realloc style function since when this is needed to use the
pool allocator is not a good idea.
The function returns NULL if `bytes` is 0.
## `RM_GetApi`
int RM_GetApi(const char *funcname, void **targetPtrPtr);
......@@ -133,6 +179,16 @@ integer instead of taking a buffer and its length.
The returned string must be released with `RedisModule_FreeString()` or by
enabling automatic memory management.
## `RM_CreateStringFromString`
RedisModuleString *RM_CreateStringFromString(RedisModuleCtx *ctx, const RedisModuleString *str);
Like `RedisModule_CreatString()`, but creates a string starting from an existing
RedisModuleString.
The returned string must be released with `RedisModule_FreeString()` or by
enabling automatic memory management.
## `RM_FreeString`
void RM_FreeString(RedisModuleCtx *ctx, RedisModuleString *str);
......@@ -579,9 +635,9 @@ The output flags are:
On success the function returns `REDISMODULE_OK`. On the following errors
`REDISMODULE_ERR` is returned:
- The key was not opened for writing.
- The key is of the wrong type.
- 'score' double value is not a number (NaN).
* The key was not opened for writing.
* The key is of the wrong type.
* 'score' double value is not a number (NaN).
## `RM_ZsetIncrby`
......@@ -609,8 +665,8 @@ Remove the specified element from the sorted set.
The function returns `REDISMODULE_OK` on success, and `REDISMODULE_ERR`
on one of the following conditions:
- The key was not opened for writing.
- The key is of the wrong type.
* The key was not opened for writing.
* The key is of the wrong type.
The return value does NOT indicate the fact the element was really
removed (since it existed) or not, just if the function was executed
......@@ -632,9 +688,9 @@ On success retrieve the double score associated at the sorted set element
'ele' and returns `REDISMODULE_OK`. Otherwise `REDISMODULE_ERR` is returned
to signal one of the following conditions:
- There is no such element 'ele' in the sorted set.
- The key is not a sorted set.
- The key is an open empty key.
* There is no such element 'ele' in the sorted set.
* The key is not a sorted set.
* The key is an open empty key.
## `RM_ZsetRangeStop`
......@@ -774,8 +830,8 @@ specified because of the XX or NX options).
In the following case the return value is always zero:
- The key was not open for writing.
- The key was associated with a non Hash value.
* The key was not open for writing.
* The key was associated with a non Hash value.
## `RM_HashGet`
......@@ -893,3 +949,197 @@ EPERM: operation in Cluster instance with key in non local slot.
Return a pointer, and a length, to the protocol returned by the command
that returned the reply object.
## `RM_CreateDataType`
moduleType *RM_CreateDataType(RedisModuleCtx *ctx, const char *name, int encver, moduleTypeLoadFunc rdb_load, moduleTypeSaveFunc rdb_save, moduleTypeRewriteFunc aof_rewrite, moduleTypeDigestFunc digest, moduleTypeFreeFunc free);
Register a new data type exported by the module. The parameters are the
following. Please for in depth documentation check the modules API
documentation, especially the INTRO.md file.
* **name**: A 9 characters data type name that MUST be unique in the Redis
Modules ecosystem. Be creative... and there will be no collisions. Use
the charset A-Z a-z 9-0, plus the two "-_" characters. A good
idea is to use, for example `<typename>-<vendor>`. For example
"tree-AntZ" may mean "Tree data structure by @antirez". To use both
lower case and upper case letters helps in order to prevent collisions.
* **encver**: Encoding version, which is, the version of the serialization
that a module used in order to persist data. As long as the "name"
matches, the RDB loading will be dispatched to the type callbacks
whatever 'encver' is used, however the module can understand if
the encoding it must load are of an older version of the module.
For example the module "tree-AntZ" initially used encver=0. Later
after an upgrade, it started to serialize data in a different format
and to register the type with encver=1. However this module may
still load old data produced by an older version if the rdb_load
callback is able to check the encver value and act accordingly.
The encver must be a positive value between 0 and 1023.
* **rdb_load**: A callback function pointer that loads data from RDB files.
* **rdb_save**: A callback function pointer that saves data to RDB files.
* **aof_rewrite**: A callback function pointer that rewrites data as commands.
* **digest**: A callback function pointer that is used for `DEBUG DIGEST`.
* **free**: A callback function pointer that can free a type value.
Note: the module name "AAAAAAAAA" is reserved and produces an error, it
happens to be pretty lame as well.
If there is already a module registering a type with the same name,
and if the module name or encver is invalid, NULL is returned.
Otherwise the new type is registered into Redis, and a reference of
type RedisModuleType is returned: the caller of the function should store
this reference into a gobal variable to make future use of it in the
modules type API, since a single module may register multiple types.
Example code fragment:
static RedisModuleType *BalancedTreeType;
int `RedisModule_OnLoad(RedisModuleCtx` *ctx) {
// some code here ...
BalancedTreeType = `RM_CreateDataType(`...);
}
## `RM_ModuleTypeSetValue`
int RM_ModuleTypeSetValue(RedisModuleKey *key, moduleType *mt, void *value);
If the key is open for writing, set the specified module type object
as the value of the key, deleting the old value if any.
On success `REDISMODULE_OK` is returned. If the key is not open for
writing or there is an active iterator, `REDISMODULE_ERR` is returned.
## `RM_ModuleTypeGetType`
moduleType *RM_ModuleTypeGetType(RedisModuleKey *key);
Assuming `RedisModule_KeyType()` returned `REDISMODULE_KEYTYPE_MODULE` on
the key, returns the moduel type pointer of the value stored at key.
If the key is NULL, is not associated with a module type, or is empty,
then NULL is returned instead.
## `RM_ModuleTypeGetValue`
void *RM_ModuleTypeGetValue(RedisModuleKey *key);
Assuming `RedisModule_KeyType()` returned `REDISMODULE_KEYTYPE_MODULE` on
the key, returns the module type low-level value stored at key, as
it was set by the user via `RedisModule_ModuleTypeSet()`.
If the key is NULL, is not associated with a module type, or is empty,
then NULL is returned instead.
## `RM_SaveUnsigned`
void RM_SaveUnsigned(RedisModuleIO *io, uint64_t value);
Save an unsigned 64 bit value into the RDB file. This function should only
be called in the context of the rdb_save method of modules implementing new
data types.
## `RM_LoadUnsigned`
uint64_t RM_LoadUnsigned(RedisModuleIO *io);
Load an unsigned 64 bit value from the RDB file. This function should only
be called in the context of the rdb_load method of modules implementing
new data types.
## `RM_SaveSigned`
void RM_SaveSigned(RedisModuleIO *io, int64_t value);
Like `RedisModule_SaveUnsigned()` but for signed 64 bit values.
## `RM_LoadSigned`
int64_t RM_LoadSigned(RedisModuleIO *io);
Like `RedisModule_LoadUnsigned()` but for signed 64 bit values.
## `RM_SaveString`
void RM_SaveString(RedisModuleIO *io, RedisModuleString *s);
In the context of the rdb_save method of a module type, saves a
string into the RDB file taking as input a RedisModuleString.
The string can be later loaded with `RedisModule_LoadString()` or
other Load family functions expecting a serialized string inside
the RDB file.
## `RM_SaveStringBuffer`
void RM_SaveStringBuffer(RedisModuleIO *io, const char *str, size_t len);
Like `RedisModule_SaveString()` but takes a raw C pointer and length
as input.
## `RM_LoadString`
RedisModuleString *RM_LoadString(RedisModuleIO *io);
In the context of the rdb_load method of a module data type, loads a string
from the RDB file, that was previously saved with `RedisModule_SaveString()`
functions family.
The returned string is a newly allocated RedisModuleString object, and
the user should at some point free it with a call to `RedisModule_FreeString()`.
If the data structure does not store strings as RedisModuleString objects,
the similar function `RedisModule_LoadStringBuffer()` could be used instead.
## `RM_LoadStringBuffer`
char *RM_LoadStringBuffer(RedisModuleIO *io, size_t *lenptr);
Like `RedisModule_LoadString()` but returns an heap allocated string that
was allocated with `RedisModule_Alloc()`, and can be resized or freed with
`RedisModule_Realloc()` or `RedisModule_Free()`.
The size of the string is stored at '*lenptr' if not NULL.
The returned string is not automatically NULL termianted, it is loaded
exactly as it was stored inisde the RDB file.
## `RM_SaveDouble`
void RM_SaveDouble(RedisModuleIO *io, double value);
In the context of the rdb_save method of a module data type, saves a double
value to the RDB file. The double can be a valid number, a NaN or infinity.
It is possible to load back the value with `RedisModule_LoadDouble()`.
## `RM_LoadDouble`
double RM_LoadDouble(RedisModuleIO *io);
In the context of the rdb_save method of a module data type, loads back the
double value saved by `RedisModule_SaveDouble()`.
## `RM_EmitAOF`
void RM_EmitAOF(RedisModuleIO *io, const char *cmdname, const char *fmt, ...);
Emits a command into the AOF during the AOF rewriting process. This function
is only called in the context of the aof_rewrite method of data types exported
by a module. The command works exactly like `RedisModule_Call()` in the way
the parameters are passed, but it does not return anything as the error
handling is performed by Redis itself.
## `RM_Log`
void RM_Log(RedisModuleCtx *ctx, const char *levelstr, const char *fmt, ...);
Produces a log message to the standard Redis log, the format accepts
printf-alike specifiers, while level is a string describing the log
level to use when emitting the log, and must be one of the following:
* "debug"
* "verbose"
* "notice"
* "warning"
If the specified log level is invalid, verbose is used by default.
There is a fixed limit to the length of the log line this function is able
to emit, this limti is not specified but is guaranteed to be more than
a few lines of text.
Redis Modules API reference manual
Redis Modules: an introduction to the API
===
The modules documentation is composed of the following files:
* `INTRO.md` (this file). An overview about Redis Modules system and API. It's a good idea to start your reading here.
* `API.md` is generated from module.c top comments of RedisMoule functions. It is a good reference in order to understand how each function works.
* `TYPES.md` covers the implementation of native data types into modules.
Redis modules make possible to extend Redis functionality using external
modules, implementing new Redis commands at a speed and with features
similar to what can be done inside the core itself.
......@@ -59,7 +65,7 @@ simple module that implements a command that outputs a random number.
return REDISMODULE_OK;
}
int RedisModule_OnLoad(RedisModuleCtx *ctx) {
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (RedisModule_Init(ctx,"helloworld",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
......@@ -150,6 +156,24 @@ exported.
The module will be able to load into different versions of Redis.
# Passing configuration parameters to Redis modules
When the module is loaded with the `MODULE LOAD` command, or using the
`loadmodule` directive in the `redis.conf` file, the user is able to pass
configuration parameters to the module by adding arguments after the module
file name:
loadmodule mymodule.so foo bar 1234
In the above example the strings `foo`, `bar` and `123` will be passed
to the module `OnLoad()` function in the `argv` argument as an array
of RedisModuleString pointers. The number of arguments passed is into `argc`.
The way you can access those strings will be explained in the rest of this
document. Normally the module will store the module configuration parameters
in some `static` global variable that can be accessed module wide, so that
the configuration can change the behavior of different commands.
# Working with RedisModuleString objects
The command argument vector `argv` passed to module commands, and the
......@@ -162,7 +186,7 @@ There are a few functions in order to work with string objects:
const char *RedisModule_StringPtrLen(RedisModuleString *string, size_t *len);
The above function accesses a string by returning its pointer and setting its
The above function accesses a string by returning its pointer and setting its
length in `len`.
You should never write to a string object pointer, as you can see from the
`const` pointer qualifier.
......@@ -344,7 +368,7 @@ section).
# Releasing call reply objects
Reply objects must be freed using `RedisModule_FreeCallRelpy`. For arrays,
Reply objects must be freed using `RedisModule_FreeCallReply`. For arrays,
you need to free only the top level reply, not the nested replies.
Currently the module implementation provides a protection in order to avoid
crashing if you free a nested reply object for error, however this feature
......@@ -623,7 +647,7 @@ access) for speed. The API will return a pointer and a length, so that's
possible to access and, if needed, modify the string directly.
size_t len, j;
char *myptr = RedisModule_StringDMA(key,REDISMODULE_WRITE,&len);
char *myptr = RedisModule_StringDMA(key,&len,REDISMODULE_WRITE);
for (j = 0; j < len; j++) myptr[j] = 'A';
In the above example we write directly on the string. Note that if you want
......@@ -777,10 +801,56 @@ Automatic memory management is usually the way to go, however experienced
C programmers may not use it in order to gain some speed and memory usage
benefit.
# Allocating memory into modules
Normal C programs use `malloc()` and `free()` in order to allocate and
release memory dynamically. While in Redis modules the use of malloc is
not technically forbidden, it is a lot better to use the Redis Modules
specific functions, that are exact replacements for `malloc`, `free`,
`realloc` and `strdup`. These functions are:
void *RedisModule_Alloc(size_t bytes);
void* RedisModule_Realloc(void *ptr, size_t bytes);
void RedisModule_Free(void *ptr);
void RedisModule_Calloc(size_t nmemb, size_t size);
char *RedisModule_Strdup(const char *str);
They work exactly like their `libc` equivalent calls, however they use
the same allocator Redis uses, and the memory allocated using these
functions is reported by the `INFO` command in the memory section, is
accounted when enforcing the `maxmemory` policy, and in general is
a first citizen of the Redis executable. On the contrar, the method
allocated inside modules with libc `malloc()` is transparent to Redis.
Another reason to use the modules functions in order to allocate memory
is that, when creating native data types inside modules, the RDB loading
functions can return deserialized strings (from the RDB file) directly
as `RedisModule_Alloc()` allocations, so they can be used directly to
populate data structures after loading, instead of having to copy them
to the data structure.
## Pool allocator
Sometimes in commands implementations, it is required to perform many
small allocations that will be not retained at the end of the command
execution, but are just functional to execute the command itself.
This work can be more easily accomplished using the Redis pool allocator:
void *RedisModule_PoolAlloc(RedisModuleCtx *ctx, size_t bytes);
It works similarly to `malloc()`, and returns memory aligned to the
next power of two of greater or equal to `bytes` (for a maximum alignment
of 8 bytes). However it allocates memory in blocks, so it the overhead
of the allocations is small, and more important, the memory allocated
is automatically released when the command returns.
So in general short living allocations are a good candidates for the pool
allocator.
# Writing commands compatible with Redis Cluster
Documentation missing, please check the following functions inside `module.c`:
RedisModule_IsKeysPositionRequest(ctx);
RedisModule_KeyAtPos(ctx,pos);
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment