If your operation is read-only, you can execute it without writing to the log or without replicating it to the other nodes. Still, the Raft library must communicate with the other nodes to verify that local node is still the leader. For that reason, there is another API function to submit read-only requests. When it is safe to execute a request, Raft library will call the callback you provided:
// Cluster is down or this node is not the leader anymore.
// We cannot execute the command.
return;
}
app_execute_readonly(request);
}
```
Log compaction
------------------
Over time, the log file will grow. The library does not initiate log compaction itself. The application should decide when to take a snapshot. (e.g. if the log file grows over a limit)
These are the steps when you want to save a snapshot:
- Call `raft_begin_snapshot()`.
- Save `last_applied_term` and `last_applied_index` and node configuration into the snapshot.
// You may also want to store address info for the node but skipping that
// part in this example as it is not part of the library
serialize_node_into_snapshot(id,voting);
}
app_save_snapshot_to_disk();
raft_end_snapshot(r);
}
```
Restore state after a restart
------------------
Raft stores all its data in three files: Snapshot, log, and the metadata file (see `raft_persist_metadata_f` in `raft.h` for details).
After a restart, we need to restore the state. The correct order would be:
- Initialize the library (described above)
- Restore snapshot (skip this step if you don't have a snapshot)
- Restore log entries
- Restore metadata
### Restoring from snapshot
The application saves `last_applied_term` and `last_applied_index` along with node configuration into the snapshots.
After loading the state into your application on a restart, you need to configure the Raft library from the configuration info in the snapshot and call `raft_restore_snapshot()`.
e.g:
```c
// As an example, assuming you read a list of nodes from the snapshot
app_load_logs_to_impl();// Load log entries in your log implementation
raft_restore_log();
app_read_metadata_file();// Read metadata file and extract term and vote
raft_restore_metadata(r,term,vote);
}
```
Sending a snapshot and loading a snapshot as a follower
------------------
#### Sending a snapshot
The leader node can send a snapshot to the follower node if the follower lags behind. When that happens, the Raft library will call `raft_get_snapshot_chunk_f` callback and require a chunk from the snapshot from the application. An example implementation would be:
```c
// Assuming you have a pointer to snapshot
// (e.g. you can use mmap'ed file or if the snapshot is small, you can have a
// Configure the Raft library from the configuration in the snapshot
// See the example implementation above for more details about this function
app_configure_from_snapshot();
raft_end_load_snapshot(rr->raft);
// Probably, you need to create mmap of the snapshot file or read it into the
// memory to be prepared for the `raft_get_snapshot_chunk_f` callback.
// See the example above for the `raft_get_snapshot_chunk_f` callback for more details.
return0;
}
```
Adding and removing nodes
------------------
You can add or remove a node by submitting configuration change entries. Once the entry is applied, you can submit another configuration change entry. Only one configuration change is allowed at a time.
#### Adding a node
Adding a node is a two-step operation. In the first step, you wait until the new node catches up with the leader. A node with a slow connection may not catch up with the leader. In this case, adding the node without waiting to obtain enough logs would cause unavailability. To prevent that, adding a node is a two-step operation.
e.g.
A single node cluster adds a new node directly. The majority in the cluster becomes two. To commit an entry, we need to replicate it to both nodes, but we cannot replicate a new entry until the new node gets all the existing entries. So, the cluster cannot commit an entry until the new node catches up.
Add a node in two steps:
- Submit an entry with the type `RAFT_LOGTYPE_ADD_NONVOTING_NODE`.
- The Raft Library will call `raft_node_has_sufficient_logs_f` callback once the new node obtains enough log entries and catches up with the leader.
- Inside that callback, you can submit an entry with the type `RAFT_LOGTYPE_ADD_NODE`.
- When that entry is applied, the configuration change will be final.
e.g:
```c
structapp_node_config{
raft_node_id_tid;
intport;
charhost[128];
};
// Step-0 This is an example implementation of `raft_get_node_id_f` callback.