Represents another server in the cluster. More...

#include <RaftConsensus.h>

Inheritance diagram for LogCabin::Server::RaftConsensusInternal::Peer:

Public Types
enum	CallStatus { OK, FAILED, INVALID_REQUEST }
	Returned by callRPC(). More...
Public Member Functions
	Peer (uint64_t serverId, RaftConsensus &consensus)
	Constructor.
	~Peer ()
	Destructor.
void	beginRequestVote ()
	Begin requesting the Server's vote in the current election.
void	beginLeadership ()
	Begin replicating to the Server in the current term.
void	exit ()
	Inform any threads belonging to this Server to exit.
uint64_t	getLastAckEpoch () const
	Return the latest time this Server acknowledged our current term.
uint64_t	getMatchIndex () const
	Return the largest entry ID for which this Server is known to share the same entries up to and including this entry with our log.
bool	haveVote () const
	Return true if this Server has awarded us its vote for this term.
bool	isCaughtUp () const
	Return true once this Server is ready to be added to the cluster.
void	interrupt ()
	Cancel any outstanding RPCs to this Server.
void	scheduleHeartbeat ()
	Make the next heartbeat RPC happen soon.
CallStatus	callRPC (Protocol::Raft::OpCode opCode, const google::protobuf::Message &request, google::protobuf::Message &response, std::unique_lock< Mutex > &lockGuard)
	Execute a remote procedure call on the server's RaftService.
void	startThread (std::shared_ptr< Peer > self)
	Launch this Peer's thread, which should run RaftConsensus::peerThreadMain.
std::ostream &	dumpToStream (std::ostream &os) const
	Virtual method for operator<<.
void	updatePeerStats (Protocol::ServerStats::Raft::Peer &peerStats, Core::Time::SteadyTimeConverter &time) const
	Write this Server's state into the given structure.
Public Attributes
RaftConsensus &	consensus
	Used in startThread.
Event::Loop &	eventLoop
	A reference to the server's event loop, needed to construct new sessions.
bool	exiting
	Set to true when thread should exit.
bool	requestVoteDone
	Set to true if the server has responded to our RequestVote request in the current term, false otherwise.
bool	haveVote_
	See haveVote().
bool	suppressBulkData
	Indicates that the leader and the follower aren't necessarily synchronized.
uint64_t	nextIndex
	The index of the next entry to send to the follower.
uint64_t	matchIndex
	See getMatchIndex().
uint64_t	lastAckEpoch
	See getLastAckEpoch().
TimePoint	nextHeartbeatTime
	When the next heartbeat should be sent to the follower.
TimePoint	backoffUntil
	The minimum time at which the next RPC should be sent.
uint64_t	rpcFailuresSinceLastWarning
	Counts RPC failures to issue fewer warnings.
uint64_t	lastCatchUpIterationMs
TimePoint	thisCatchUpIterationStart
uint64_t	thisCatchUpIterationGoalId
bool	isCaughtUp_
	See isCaughtUp().
std::unique_ptr < Storage::FilesystemUtil::FileContents >	snapshotFile
	A snapshot file to be sent to the follower, or NULL.
uint64_t	snapshotFileOffset
	The number of bytes of 'snapshotFile' that have been acknowledged by the follower already.
uint64_t	lastSnapshotIndex
	The last log index that 'snapshotFile' corresponds to.
Private Member Functions
std::shared_ptr < RPC::ClientSession >	getSession (std::unique_lock< Mutex > &lockGuard)
	Get the current session for this server.
	Peer (const Peer &)
Peer &	operator= (const Peer &)
Private Attributes
std::shared_ptr < RPC::ClientSession >	session
	Caches the result of getSession().
RPC::ClientRPC	rpc
	callRPC() places its RPC here so that interrupt() may cancel it.

Detailed Description

Represents another server in the cluster.

One of these exists for each other server. In addition to tracking state for each other server, this class provides a thread that executes RaftConsensus::peerThreadMain().

This class has no internal locking: in general, the RaftConsensus lock should be held when accessing this class, but there are some exceptions noted below.

Definition at line 258 of file RaftConsensus.h.

Member Enumeration Documentation

enum LogCabin::Server::RaftConsensusInternal::Peer::CallStatus

Returned by callRPC().

Enumerator:

OK

The RPC succeeded and the response was filled in.

FAILED

No reply was received from the server.

Maybe the connection was dropped; maybe the RPC was canceled.

INVALID_REQUEST

The server does not support this RPC or didn't like the arguments given.

Definition at line 284 of file RaftConsensus.h.

Constructor & Destructor Documentation

LogCabin::Server::RaftConsensusInternal::Peer::Peer	(	uint64_t	serverId,
		RaftConsensus &	consensus
	)

Constructor.

Definition at line 162 of file RaftConsensus.cc.

LogCabin::Server::RaftConsensusInternal::Peer::~Peer ( )

Destructor.

Definition at line 193 of file RaftConsensus.cc.

LogCabin::Server::RaftConsensusInternal::Peer::Peer ( const Peer & ) [private]

Member Function Documentation

void LogCabin::Server::RaftConsensusInternal::Peer::beginRequestVote ( ) [virtual]

Begin requesting the Server's vote in the current election.

Return immediately. The condition variable in RaftConsensus will be notified separately.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 198 of file RaftConsensus.cc.

void LogCabin::Server::RaftConsensusInternal::Peer::beginLeadership ( ) [virtual]

Begin replicating to the Server in the current term.

Return immediately. The condition variable in RaftConsensus will be notified separately.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 205 of file RaftConsensus.cc.

void LogCabin::Server::RaftConsensusInternal::Peer::exit ( ) [virtual]

Inform any threads belonging to this Server to exit.

Return immediately. The condition variable in RaftConsensus will be notified separately.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 216 of file RaftConsensus.cc.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::getLastAckEpoch ( ) const [virtual]

Return the latest time this Server acknowledged our current term.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 230 of file RaftConsensus.cc.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::getMatchIndex ( ) const [virtual]

Return the largest entry ID for which this Server is known to share the same entries up to and including this entry with our log.

This is used for advancing the leader's commitIndex. Monotonically increases within a term.

Warning:: Only valid when we're leader.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 236 of file RaftConsensus.cc.

bool LogCabin::Server::RaftConsensusInternal::Peer::haveVote ( ) const [virtual]

Return true if this Server has awarded us its vote for this term.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 242 of file RaftConsensus.cc.

bool LogCabin::Server::RaftConsensusInternal::Peer::isCaughtUp ( ) const [virtual]

Return true once this Server is ready to be added to the cluster.

This means it has received enough of our log to where it is not expected to cause an availability problem when added to the cluster configuration. Should monotonically change from false to true.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 254 of file RaftConsensus.cc.

void LogCabin::Server::RaftConsensusInternal::Peer::interrupt ( ) [virtual]

Cancel any outstanding RPCs to this Server.

The condition variable in RaftConsensus will be notified separately.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 248 of file RaftConsensus.cc.

void LogCabin::Server::RaftConsensusInternal::Peer::scheduleHeartbeat ( ) [virtual]

Make the next heartbeat RPC happen soon.

Return immediately. The condition variable in RaftConsensus will be notified separately.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 260 of file RaftConsensus.cc.

Peer::CallStatus LogCabin::Server::RaftConsensusInternal::Peer::callRPC	(	Protocol::Raft::OpCode	opCode,
		const google::protobuf::Message &	request,
		google::protobuf::Message &	response,
		std::unique_lock< Mutex > &	lockGuard
	)

Execute a remote procedure call on the server's RaftService.

As this operation might take a while, it should be called without RaftConsensus lock.

Parameters:

[in]	opCode	The RPC opcode to execute (see Protocol::Raft::OpCode).
[in]	request	The request that was received from the other server.
[out]	response	Where the reply should be placed, if status is OK.
[in]	lockGuard	The Raft lock, which is released internally to allow for I/O concurrency.

Returns:: See CallStatus.

Definition at line 266 of file RaftConsensus.cc.

void LogCabin::Server::RaftConsensusInternal::Peer::startThread ( std::shared_ptr< Peer > self )

Launch this Peer's thread, which should run RaftConsensus::peerThreadMain.

Parameters:

self	A shared_ptr to this object, which the detached thread uses to make sure this object doesn't go away.

Definition at line 313 of file RaftConsensus.cc.

std::ostream & LogCabin::Server::RaftConsensusInternal::Peer::dumpToStream ( std::ostream & os ) const [virtual]

Virtual method for operator<<.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 346 of file RaftConsensus.cc.

void LogCabin::Server::RaftConsensusInternal::Peer::updatePeerStats	(	Protocol::ServerStats::Raft::Peer &	peerStats,
		Core::Time::SteadyTimeConverter &	time
	)		const `[virtual]`

Write this Server's state into the given structure.

Used for diagnostics.

Implements LogCabin::Server::RaftConsensusInternal::Server.

Definition at line 375 of file RaftConsensus.cc.

std::shared_ptr< RPC::ClientSession > LogCabin::Server::RaftConsensusInternal::Peer::getSession ( std::unique_lock< Mutex > & lockGuard ) [private]

Get the current session for this server.

(This is cached in the session member for efficiency.) As this operation might take a while, it should be called without RaftConsensus lock.

Definition at line 323 of file RaftConsensus.cc.

Peer& LogCabin::Server::RaftConsensusInternal::Peer::operator= ( const Peer & ) [private]

Member Data Documentation

RaftConsensus& LogCabin::Server::RaftConsensusInternal::Peer::consensus

Used in startThread.

TODO(ongaro): reconsider

Definition at line 351 of file RaftConsensus.h.

Event::Loop& LogCabin::Server::RaftConsensusInternal::Peer::eventLoop

A reference to the server's event loop, needed to construct new sessions.

Definition at line 357 of file RaftConsensus.h.

bool LogCabin::Server::RaftConsensusInternal::Peer::exiting

Set to true when thread should exit.

Definition at line 362 of file RaftConsensus.h.

bool LogCabin::Server::RaftConsensusInternal::Peer::requestVoteDone

Set to true if the server has responded to our RequestVote request in the current term, false otherwise.

Definition at line 368 of file RaftConsensus.h.

bool LogCabin::Server::RaftConsensusInternal::Peer::haveVote_

See haveVote().

Definition at line 373 of file RaftConsensus.h.

bool LogCabin::Server::RaftConsensusInternal::Peer::suppressBulkData

Indicates that the leader and the follower aren't necessarily synchronized.

The leader should not send large amounts of data (with many log entries or large chunks of a snapshot file) to the follower while this flag is true. For example, the follower might have been disconnected, or the leader might not know where the follower's log diverges from its own. It's better to sync up using small RPCs like heartbeats, then begin/resume sending bulk data after receiving an acknowledgment.

Only used when leader.

Definition at line 387 of file RaftConsensus.h.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::nextIndex

The index of the next entry to send to the follower.

Only used when leader. Minimum value of 1.

Definition at line 393 of file RaftConsensus.h.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::matchIndex

See getMatchIndex().

Definition at line 398 of file RaftConsensus.h.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::lastAckEpoch

See getLastAckEpoch().

Definition at line 403 of file RaftConsensus.h.

TimePoint LogCabin::Server::RaftConsensusInternal::Peer::nextHeartbeatTime

When the next heartbeat should be sent to the follower.

Only valid while we're leader. The leader sends heartbeats periodically if it has no new data to send, to stop the follower from starting a new election.

Invariant:: This is never more than HEARTBEAT_PERIOD in the future, since new leaders don't reset it.

Definition at line 414 of file RaftConsensus.h.

TimePoint LogCabin::Server::RaftConsensusInternal::Peer::backoffUntil

The minimum time at which the next RPC should be sent.

Only valid while we're a candidate or leader. This is set when an RPC fails so as to not overwhelm the network with retries (some RPCs fail without timing out, for example if the remote kernel rejects the connection).

Definition at line 423 of file RaftConsensus.h.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::rpcFailuresSinceLastWarning

Counts RPC failures to issue fewer warnings.

Accessed only from callRPC() without holding the lock.

Definition at line 429 of file RaftConsensus.h.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::lastCatchUpIterationMs

Definition at line 432 of file RaftConsensus.h.

TimePoint LogCabin::Server::RaftConsensusInternal::Peer::thisCatchUpIterationStart

Definition at line 433 of file RaftConsensus.h.

uint64_t LogCabin::Server::RaftConsensusInternal::Peer::thisCatchUpIterationGoalId

Definition at line 434 of file RaftConsensus.h.

bool LogCabin::Server::RaftConsensusInternal::Peer::isCaughtUp_