The interface class to talk to peer gem5 processes. More...

#include <dist_iface.hh>

Inheritance diagram for DistIface:

Classes
class	RecvScheduler
	Class to encapsulate information about data packets received. More...

class	Sync
	This class implements global sync operations among gem5 peer processes. More...

class	SyncEvent
	The global event to schedule periodic dist sync. More...

class	SyncNode

class	SyncSwitch

Public Types
typedef DistHeaderPkt::Header	Header

Public Member Functions
	DistIface (unsigned dist_rank, unsigned dist_size, Tick sync_start, Tick sync_repeat, EventManager *em, bool use_pseudo_op, bool is_switch, int num_nodes)
	ctor More...

virtual	~DistIface ()

void	packetOut (EthPacketPtr pkt, Tick send_delay)
	Send out an Ethernet packet. More...

EthPacketPtr	packetIn ()
	Fetch the packet scheduled to be received next by the simulated network link. More...

DrainState	drain () override
	Notify an object that it needs to drain its state. More...

void	drainResume () override
	Resume execution after a successful drain. More...

void	init (const Event *e, Tick link_delay)

void	startup ()

void	serialize (CheckpointOut &cp) const override
	Serialize an object. More...

void	unserialize (CheckpointIn &cp) override
	Unserialize an object. More...

Public Member Functions inherited from Drainable
DrainState	drainState () const
	Return the current drain state of an object. More...

virtual void	notifyFork ()
	Notify a child process of a fork. More...

Public Member Functions inherited from Serializable
	Serializable ()

virtual	~Serializable ()

void	serializeSection (CheckpointOut &cp, const char *name) const
	Serialize an object into a new section. More...

void	serializeSection (CheckpointOut &cp, const std::string &name) const

void	unserializeSection (CheckpointIn &cp, const char *name)
	Unserialize an a child object. More...

void	unserializeSection (CheckpointIn &cp, const std::string &name)

Static Public Member Functions
static bool	readyToExit (Tick delay)
	Initiate the exit from the simulation. More...

static bool	readyToCkpt (Tick delay, Tick period)
	Initiate taking a checkpoint. More...

static uint64_t	rankParam ()
	Getter for the dist rank param. More...

static uint64_t	sizeParam ()
	Getter for the dist size param. More...

static void	toggleSync (ThreadContext *tc)
	Trigger the master to start/stop synchronization. More...

Static Public Member Functions inherited from Serializable
static const std::string &	currentSection ()
	Get the fully-qualified name of the active section. More...

static void	serializeAll (const std::string &cpt_dir)

static void	unserializeGlobals (CheckpointIn &cp)

Protected Types
typedef DistHeaderPkt::MsgType	MsgType

typedef DistHeaderPkt::ReqType	ReqType

Protected Attributes
unsigned	rank
	The rank of this process among the gem5 peers. More...

unsigned	size
	The number of gem5 processes comprising this dist simulation. More...

unsigned	distIfaceId
	Unique id for the dist link. More...

bool	isMaster

Static Protected Attributes
static unsigned	distIfaceNum = 0
	Number of DistIface objects (i.e. More...

Private Member Functions
virtual void	sendPacket (const Header &header, const EthPacketPtr &packet)=0
	Send out a data packet to the remote end. More...

virtual void	sendCmd (const Header &header)=0
	Send out a control command to the remote end. More...

virtual bool	recvHeader (Header &header)=0
	Receive a header (i.e. More...

virtual void	recvPacket (const Header &header, EthPacketPtr &packet)=0
	Receive a packet from the remote end. More...

virtual void	initTransport ()=0
	Init hook for the underlaying transport. More...

void	spawnRecvThread (const Event *recv_done, Tick link_delay)
	spawn the receiver thread. More...

void	recvThreadFunc (Event *recv_done, Tick link_delay)
	The function executed by a receiver thread. More...

Private Attributes
Tick	syncStart
	Tick to schedule the first dist sync event. More...

Tick	syncRepeat
	Frequency of dist sync events in ticks. More...

std::thread *	recvThread
	Receiver thread pointer. More...

RecvScheduler	recvScheduler
	Meta information about data packets received. More...

bool	syncStartOnPseudoOp
	Use pseudoOp to start synchronization. More...

Static Private Attributes
static unsigned	recvThreadsNum = 0
	Number of receiver threads (in this gem5 process) More...

static Sync *	sync = nullptr
	The singleton Sync object to perform dist synchronisation. More...

static SyncEvent *	syncEvent = nullptr
	The singleton SyncEvent object to schedule periodic dist sync. More...

static DistIface *	master = nullptr
	The very first DistIface object created becomes the master. More...

static System *	sys = nullptr
	System pointer used to wakeup sleeping threads when stopping sync. More...

static bool	isSwitch = false
	Is this node a switch? More...

Additional Inherited Members
Static Public Attributes inherited from Serializable
static int	ckptCount = 0

static int	ckptMaxCount = 0

static int	ckptPrevCount = -1

Protected Member Functions inherited from Drainable
	Drainable ()

virtual	~Drainable ()

void	signalDrainDone () const
	Signal that an object is drained. More...

Detailed Description

The interface class to talk to peer gem5 processes.

Definition at line 101 of file dist_iface.hh.

Member Typedef Documentation

typedef DistHeaderPkt::Header DistIface::Header

Definition at line 104 of file dist_iface.hh.

typedef DistHeaderPkt::MsgType DistIface::MsgType

protected

Definition at line 107 of file dist_iface.hh.

typedef DistHeaderPkt::ReqType DistIface::ReqType

protected

Definition at line 108 of file dist_iface.hh.

Constructor & Destructor Documentation

DistIface::DistIface	(	unsigned	dist_rank,
		unsigned	dist_size,
		Tick	sync_start,
		Tick	sync_repeat,
		EventManager *	em,
		bool	use_pseudo_op,
		bool	is_switch,
		int	num_nodes
	)

ctor

Parameters

dist_rank	Rank of this gem5 process within the dist run
sync_start	Start tick for dist synchronisation
sync_repeat	Frequency for dist synchronisation
em	The event manager associated with the simulated Ethernet link

Definition at line 606 of file dist_iface.cc.

References distIfaceId, distIfaceNum, DPRINTF, isMaster, isSwitch, master, sync, and syncEvent.

DistIface::~DistIface ( )

virtual

Definition at line 635 of file dist_iface.cc.

References distIfaceNum, master, recvThread, sync, and syncEvent.

Member Function Documentation

DrainState DistIface::drain ( )

overridevirtual

Notify an object that it needs to drain its state.

If the object does not need further simulation to drain internal buffers, it returns DrainState::Drained and automatically switches to the Drained state. If the object needs more simulation, it returns DrainState::Draining and automatically enters the Draining state. Other return values are invalid.

Note: An object that has entered the Drained state can be disturbed by other objects in the system and consequently stop being drained. These perturbations are not visible in the drain state. The simulator therefore repeats the draining process until all objects return DrainState::Drained on the first call to drain().

Returns: DrainState::Drained if the object is drained at this point in time, DrainState::Draining if it needs further simulation.

Implements Drainable.

Definition at line 731 of file dist_iface.cc.

References DPRINTF, Drained, DistIface::SyncEvent::draining(), master, and syncEvent.

void DistIface::drainResume ( )

overridevirtual

Resume execution after a successful drain.

Reimplemented from Drainable.

Definition at line 741 of file dist_iface.cc.

References DPRINTF, DistIface::SyncEvent::draining(), master, recvScheduler, DistIface::RecvScheduler::resumeRecvTicks(), and syncEvent.

void DistIface::init	(	const Event *	e,
		Tick	link_delay
	)

Definition at line 787 of file dist_iface.cc.

References Random::init(), DistIface::Sync::init(), initTransport(), master, random_mt, rank, spawnRecvThread(), sync, syncRepeat, and syncStart.

Referenced by DistEtherLink::init().

virtual void DistIface::initTransport ( )

privatepure virtual

Init hook for the underlaying transport.

Implemented in TCPIface.

Referenced by init().

EthPacketPtr DistIface::packetIn ( )

inline

Fetch the packet scheduled to be received next by the simulated network link.

Note: This method is called within the process() method of the link receive done event. It also schedules the next receive event if the receive queue is not empty.

Definition at line 600 of file dist_iface.hh.

References DistIface::RecvScheduler::popPacket(), and recvScheduler.

Referenced by DistEtherLink::RxLink::rxDone().

void DistIface::packetOut	(	EthPacketPtr	pkt,
		Tick	send_delay
	)

Send out an Ethernet packet.

Parameters

pkt	The Ethernet packet to send.
send_delay	The delay in ticks for the send completion event.

Definition at line 651 of file dist_iface.cc.

References curTick(), DistHeaderPkt::dataDescriptor, DistHeaderPkt::Header::dataPacketLength, DPRINTF, DistHeaderPkt::Header::msgType, DistHeaderPkt::Header::sendDelay, sendPacket(), DistHeaderPkt::Header::sendTick, and DistHeaderPkt::Header::simLength.

Referenced by DistEtherLink::TxLink::transmit().

uint64_t DistIface::rankParam ( )

static

Getter for the dist rank param.

Definition at line 919 of file dist_iface.cc.

References master, rank, X86ISA::val, and warn.

Referenced by PseudoInst::initParam().

bool DistIface::readyToCkpt	(	Tick	delay,
		Tick	period
	)

static

Initiate taking a checkpoint.

Parameters

delay Delay param from the m5 checkpoint command. If Delay is zero then a collaborative checkpoint is requested (i.e. all nodes have to call this method before the checkpoint can be taken). If Delay is not zero then a checkpoint is requested asap (and it will happen at the next sync tick).

Returns: False if we are in dist mode (i.e. exit can happen only at sync), True otherwise.

Definition at line 824 of file dist_iface.cc.

References DistHeaderPkt::collective, DPRINTF, DistHeaderPkt::immediate, inform, master, DistIface::Sync::requestCkpt(), and sync.

Referenced by PseudoInst::m5checkpoint().

bool DistIface::readyToExit ( Tick delay )

static

Initiate the exit from the simulation.

Parameters

delay Delay param from the m5 exit command. If Delay is zero then a collaborative exit is requested (i.e. all nodes have to call this method before the distributed simulation can exit). If Delay is not zero then exit is requested asap (and it will happen at the next sync tick).

Returns: False if we are in distributed mode (i.e. exit can happen only at sync), True otherwise.

Definition at line 894 of file dist_iface.cc.

References DistHeaderPkt::collective, DPRINTF, DistHeaderPkt::immediate, inform, master, DistIface::Sync::requestExit(), BaseGlobalEvent::scheduled(), DistIface::SyncEvent::start(), sync, and syncEvent.

Referenced by PseudoInst::m5exit().

virtual bool DistIface::recvHeader ( Header & header )

privatepure virtual

Receive a header (i.e.

meta info describing a data packet or a control command) from the remote end.

Parameters

header The meta info structure to store the incoming header.

Implemented in TCPIface.

Referenced by recvThreadFunc().

virtual void DistIface::recvPacket	(	const Header &	header,
		EthPacketPtr &	packet
	)

privatepure virtual

Receive a packet from the remote end.

Parameters

header	Meta info about the incoming packet (obtanied by a previous call to the recvHedaer() method).
Pointer	to packet received.

Implemented in TCPIface.

Referenced by recvThreadFunc().

void DistIface::recvThreadFunc	(	Event *	recv_done,
		Tick	link_delay
	)

private

The function executed by a receiver thread.

Definition at line 673 of file dist_iface.cc.

References DistIface::Sync::abort(), curEventQueue(), DistHeaderPkt::dataDescriptor, exitSimLoop(), DistIface::RecvScheduler::init(), EventQueue::lock(), DistHeaderPkt::Header::msgType, DistHeaderPkt::Header::needCkpt, DistHeaderPkt::Header::needExit, DistHeaderPkt::Header::needStopSync, DistIface::Sync::progress(), DistIface::RecvScheduler::pushPacket(), recvHeader(), recvPacket(), recvScheduler, DistHeaderPkt::Header::sendDelay, DistHeaderPkt::Header::sendTick, sync, DistHeaderPkt::Header::syncRepeat, and EventQueue::unlock().

Referenced by spawnRecvThread().

virtual void DistIface::sendCmd ( const Header & header )

privatepure virtual

Send out a control command to the remote end.

Parameters

header Meta info describing the command (e.g. sync request)

Implemented in TCPIface.

Referenced by DistIface::SyncNode::run(), and DistIface::SyncSwitch::run().

virtual void DistIface::sendPacket	(	const Header &	header,
		const EthPacketPtr &	packet
	)

privatepure virtual

Send out a data packet to the remote end.

Parameters

header	Meta info about the packet (which needs to be transferred to the destination alongside the packet).
packet	Pointer to the packet to send.

Implemented in TCPIface.

Referenced by packetOut().

void DistIface::serialize ( CheckpointOut & cp ) const

overridevirtual

Serialize an object.

Output an object's state into the current checkpoint section.

Parameters

cp	Checkpoint state

Implements Serializable.

Definition at line 749 of file dist_iface.cc.

References distIfaceId, DistIface::Sync::drainComplete(), master, rank, recvScheduler, SERIALIZE_SCALAR, Serializable::serializeSection(), and sync.

uint64_t DistIface::sizeParam ( )

static

Getter for the dist size param.

Definition at line 932 of file dist_iface.cc.

References master, size, X86ISA::val, and warn.

Referenced by PseudoInst::initParam().

void DistIface::spawnRecvThread	(	const Event *	recv_done,
		Tick	link_delay
	)

private

spawn the receiver thread.

Parameters

recv_done	The receive done event associated with the simulated Ethernet link.
link_delay	The link delay for the simulated Ethernet link.

Definition at line 719 of file dist_iface.cc.

References recvThread, recvThreadFunc(), and recvThreadsNum.

Referenced by init().

void DistIface::startup ( )

Definition at line 814 of file dist_iface.cc.

References DPRINTF, isSwitch, master, DistIface::SyncEvent::start(), syncEvent, and syncStartOnPseudoOp.

Referenced by DistEtherLink::startup().

void DistIface::toggleSync ( ThreadContext * tc )

static

Trigger the master to start/stop synchronization.

Definition at line 855 of file dist_iface.cc.

References ThreadContext::Active, DistHeaderPkt::collective, ThreadContext::getSystemPtr(), System::getThreadContext(), ArmISA::i, inform, master, System::numContexts(), ThreadContext::quiesce(), ThreadContext::quiesceTick(), DistIface::Sync::requestStopSync(), BaseGlobalEvent::scheduled(), DistIface::SyncEvent::start(), ThreadContext::status(), sync, syncEvent, sys, and BaseGlobalEvent::when().

Referenced by PseudoInst::togglesync().

void DistIface::unserialize ( CheckpointIn & cp )

overridevirtual

Unserialize an object.

Read an object's state from the current checkpoint section.

Parameters

cp	Checkpoint state

Implements Serializable.

Definition at line 768 of file dist_iface.cc.

References distIfaceId, master, panic_if(), rank, recvScheduler, sync, UNSERIALIZE_SCALAR, and Serializable::unserializeSection().

Member Data Documentation

unsigned DistIface::distIfaceId

protected

Unique id for the dist link.

Definition at line 494 of file dist_iface.hh.

Referenced by DistIface(), TCPIface::establishConnection(), serialize(), and unserialize().

unsigned DistIface::distIfaceNum = 0

staticprotected

Number of DistIface objects (i.e.

dist links in this gem5 process)

Definition at line 490 of file dist_iface.hh.

Referenced by DistIface(), TCPIface::establishConnection(), and ~DistIface().

bool DistIface::isMaster

protected

Definition at line 496 of file dist_iface.hh.

Referenced by DistIface(), and TCPIface::TCPIface().

bool DistIface::isSwitch = false

staticprivate

Is this node a switch?

Definition at line 523 of file dist_iface.hh.

Referenced by DistIface(), DistIface::SyncEvent::process(), and startup().

DistIface * DistIface::master = nullptr

staticprivate

The very first DistIface object created becomes the master.

We need a master to co-ordinate the global synchronisation.

Definition at line 515 of file dist_iface.hh.

Referenced by DistIface(), drain(), drainResume(), init(), DistIface::SyncEvent::process(), rankParam(), readyToCkpt(), readyToExit(), DistIface::SyncNode::run(), DistIface::SyncSwitch::run(), serialize(), sizeParam(), startup(), toggleSync(), unserialize(), and ~DistIface().

unsigned DistIface::rank

protected

The rank of this process among the gem5 peers.

Definition at line 482 of file dist_iface.hh.

Referenced by TCPIface::establishConnection(), init(), rankParam(), serialize(), and unserialize().

RecvScheduler DistIface::recvScheduler

private

Meta information about data packets received.

Definition at line 472 of file dist_iface.hh.

Referenced by drainResume(), packetIn(), recvThreadFunc(), serialize(), and unserialize().

std::thread* DistIface::recvThread

private

Receiver thread pointer.

Each DistIface object must have exactly one receiver thread.

Definition at line 468 of file dist_iface.hh.

Referenced by spawnRecvThread(), and ~DistIface().

unsigned DistIface::recvThreadsNum = 0

staticprivate

Number of receiver threads (in this gem5 process)

Definition at line 502 of file dist_iface.hh.

Referenced by DistIface::SyncNode::run(), and spawnRecvThread().

unsigned DistIface::size

protected

The number of gem5 processes comprising this dist simulation.

Definition at line 486 of file dist_iface.hh.

Referenced by sizeParam(), and TCPIface::TCPIface().

DistIface::Sync * DistIface::sync = nullptr

staticprivate

The singleton Sync object to perform dist synchronisation.

Definition at line 506 of file dist_iface.hh.

Referenced by DistIface(), init(), DistIface::SyncEvent::process(), readyToCkpt(), readyToExit(), recvThreadFunc(), serialize(), DistIface::SyncEvent::start(), toggleSync(), unserialize(), and ~DistIface().

DistIface::SyncEvent * DistIface::syncEvent = nullptr

staticprivate

The singleton SyncEvent object to schedule periodic dist sync.

Definition at line 510 of file dist_iface.hh.

Referenced by DistIface(), drain(), drainResume(), readyToExit(), startup(), toggleSync(), and ~DistIface().

Tick DistIface::syncRepeat

private

Frequency of dist sync events in ticks.

Definition at line 463 of file dist_iface.hh.

Referenced by init().

Tick DistIface::syncStart

private

Tick to schedule the first dist sync event.

This is just as optimization : we do not need any dist sync event until the simulated NIC is brought up by the OS.

Definition at line 459 of file dist_iface.hh.

Referenced by init().

bool DistIface::syncStartOnPseudoOp

private

Use pseudoOp to start synchronization.

Definition at line 476 of file dist_iface.hh.

Referenced by startup().

System * DistIface::sys = nullptr

staticprivate

System pointer used to wakeup sleeping threads when stopping sync.

Definition at line 519 of file dist_iface.hh.

Referenced by DistIface::SyncEvent::process(), and toggleSync().

The documentation for this class was generated from the following files:

dev/net/dist_iface.hh
dev/net/dist_iface.cc

Classes

Public Types

Public Member Functions

Static Public Member Functions

Protected Types

Protected Attributes

Static Protected Attributes

Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation