The interface class to talk to peer gem5 processes. More...

#include <dist_iface.hh>

Inheritance diagram for DistIface:

Classes
class	RecvScheduler
	Class to encapsulate information about data packets received. More...

class	Sync

class	SyncEvent
	The global event to schedule periodic dist sync. More...

class	SyncNode

class	SyncSwitch

Public Types
typedef DistHeaderPkt::Header	Header

Public Member Functions
	DistIface (unsigned dist_rank, unsigned dist_size, Tick sync_start, Tick sync_repeat, EventManager *em, bool use_pseudo_op, bool is_switch, int num_nodes)
	ctor More...

virtual	~DistIface ()

void	packetOut (EthPacketPtr pkt, Tick send_delay)
	Send out an Ethernet packet. More...

EthPacketPtr	packetIn ()
	Fetch the packet scheduled to be received next by the simulated network link. More...

DrainState	drain () override
	Draining is the process of clearing out the states of SimObjects.These are the SimObjects that are partially executed or are partially in flight. More...

void	drainResume () override
	Resume execution after a successful drain. More...

void	init (const Event *e, Tick link_delay)

void	startup ()

void	serialize (CheckpointOut &cp) const override
	Serialize an object. More...

void	unserialize (CheckpointIn &cp) override
	Unserialize an object. More...

Public Member Functions inherited from Drainable
DrainState	drainState () const
	Return the current drain state of an object. More...

virtual void	notifyFork ()
	Notify a child process of a fork. More...

Public Member Functions inherited from Serializable
	Serializable ()

virtual	~Serializable ()

void	serializeSection (CheckpointOut &cp, const char *name) const
	Serialize an object into a new section. More...

void	serializeSection (CheckpointOut &cp, const std::string &name) const

void	unserializeSection (CheckpointIn &cp, const char *name)
	Unserialize an a child object. More...

void	unserializeSection (CheckpointIn &cp, const std::string &name)

Static Public Member Functions
static bool	readyToExit (Tick delay)
	Initiate the exit from the simulation. More...

static bool	readyToCkpt (Tick delay, Tick period)
	Initiate taking a checkpoint. More...

static uint64_t	rankParam ()
	Getter for the dist rank param. More...

static uint64_t	sizeParam ()
	Getter for the dist size param. More...

static void	toggleSync (ThreadContext *tc)
	Trigger the primary to start/stop synchronization. More...

Static Public Member Functions inherited from Serializable
static const std::string &	currentSection ()
	Gets the fully-qualified name of the active section. More...

static void	serializeAll (const std::string &cpt_dir)
	Serializes all the SimObjects. More...

static void	unserializeGlobals (CheckpointIn &cp)

Protected Types
typedef DistHeaderPkt::MsgType	MsgType

typedef DistHeaderPkt::ReqType	ReqType

Protected Attributes
unsigned	rank
	The rank of this process among the gem5 peers. More...

unsigned	size
	The number of gem5 processes comprising this dist simulation. More...

unsigned	distIfaceId
	Unique id for the dist link. More...

bool	isPrimary

Static Protected Attributes
static unsigned	distIfaceNum = 0
	Number of DistIface objects (i.e. More...

Private Member Functions
virtual void	sendPacket (const Header &header, const EthPacketPtr &packet)=0
	Send out a data packet to the remote end. More...

virtual void	sendCmd (const Header &header)=0
	Send out a control command to the remote end. More...

virtual bool	recvHeader (Header &header)=0
	Receive a header (i.e. More...

virtual void	recvPacket (const Header &header, EthPacketPtr &packet)=0
	Receive a packet from the remote end. More...

virtual void	initTransport ()=0
	Init hook for the underlaying transport. More...

void	spawnRecvThread (const Event *recv_done, Tick link_delay)
	spawn the receiver thread. More...

void	recvThreadFunc (Event *recv_done, Tick link_delay)
	The function executed by a receiver thread. More...

Private Attributes
Tick	syncStart
	Tick to schedule the first dist sync event. More...

Tick	syncRepeat
	Frequency of dist sync events in ticks. More...

std::thread *	recvThread
	Receiver thread pointer. More...

RecvScheduler	recvScheduler
	Meta information about data packets received. More...

bool	syncStartOnPseudoOp
	Use pseudoOp to start synchronization. More...

Static Private Attributes
static unsigned	recvThreadsNum = 0
	Number of receiver threads (in this gem5 process) More...

static Sync *	sync = nullptr
	The singleton Sync object to perform dist synchronisation. More...

static SyncEvent *	syncEvent = nullptr
	The singleton SyncEvent object to schedule periodic dist sync. More...

static DistIface *	primary = nullptr
	The very first DistIface object created becomes the primary interface. More...

static System *	sys = nullptr
	System pointer used to wakeup sleeping threads when stopping sync. More...

static bool	isSwitch = false
	Is this node a switch? More...

Additional Inherited Members
Protected Member Functions inherited from Drainable
	Drainable ()

virtual	~Drainable ()

void	signalDrainDone () const
	Signal that an object is drained. More...

Detailed Description

The interface class to talk to peer gem5 processes.

Definition at line 99 of file dist_iface.hh.

Member Typedef Documentation

◆ Header

typedef DistHeaderPkt::Header DistIface::Header

Definition at line 102 of file dist_iface.hh.

◆ MsgType

typedef DistHeaderPkt::MsgType DistIface::MsgType

protected

Definition at line 105 of file dist_iface.hh.

◆ ReqType

typedef DistHeaderPkt::ReqType DistIface::ReqType

protected

Definition at line 106 of file dist_iface.hh.

Constructor & Destructor Documentation

◆ DistIface()

DistIface::DistIface	(	unsigned	dist_rank,
		unsigned	dist_size,
		Tick	sync_start,
		Tick	sync_repeat,
		EventManager *	em,
		bool	use_pseudo_op,
		bool	is_switch,
		int	num_nodes
	)

ctor

Parameters

dist_rank	Rank of this gem5 process within the dist run
sync_start	Start tick for dist synchronisation
sync_repeat	Frequency for dist synchronisation
em	The event manager associated with the simulated Ethernet link

Definition at line 602 of file dist_iface.cc.

References distIfaceId, distIfaceNum, DPRINTF, isPrimary, isSwitch, primary, sync, and syncEvent.

◆ ~DistIface()

DistIface::~DistIface ( )

virtual

Definition at line 631 of file dist_iface.cc.

References distIfaceNum, primary, recvThread, sync, and syncEvent.

Member Function Documentation

◆ drain()

DrainState DistIface::drain ( )

overridevirtual

Draining is the process of clearing out the states of SimObjects.These are the SimObjects that are partially executed or are partially in flight.

Draining is mostly used before forking and creating a check point.

This function notifies an object that it needs to drain its state.

If the object does not need further simulation to drain internal buffers, it returns DrainState::Drained and automatically switches to the Drained state. If the object needs more simulation, it returns DrainState::Draining and automatically enters the Draining state. Other return values are invalid.

Note: An object that has entered the Drained state can be disturbed by other objects in the system and consequently stop being drained. These perturbations are not visible in the drain state. The simulator therefore repeats the draining process until all objects return DrainState::Drained on the first call to drain().

Returns: DrainState::Drained if the object is drained at this point in time, DrainState::Draining if it needs further simulation.

Implements Drainable.

Definition at line 727 of file dist_iface.cc.

References DPRINTF, Drained, DistIface::SyncEvent::draining(), primary, and syncEvent.

◆ drainResume()

void DistIface::drainResume ( )

overridevirtual

Resume execution after a successful drain.

Reimplemented from Drainable.

Definition at line 737 of file dist_iface.cc.

References DPRINTF, DistIface::SyncEvent::draining(), primary, recvScheduler, DistIface::RecvScheduler::resumeRecvTicks(), and syncEvent.

◆ init()

void DistIface::init	(	const Event *	e,
		Tick	link_delay
	)

Definition at line 783 of file dist_iface.cc.

References Random::init(), DistIface::Sync::init(), initTransport(), primary, random_mt, rank, spawnRecvThread(), sync, syncRepeat, and syncStart.

Referenced by DistEtherLink::init().

◆ initTransport()

virtual void DistIface::initTransport ( )

privatepure virtual

Init hook for the underlaying transport.

Implemented in TCPIface.

Referenced by init().

◆ packetIn()

EthPacketPtr DistIface::packetIn ( )

inline

Fetch the packet scheduled to be received next by the simulated network link.

Note: This method is called within the process() method of the link receive done event. It also schedules the next receive event if the receive queue is not empty.

Definition at line 598 of file dist_iface.hh.

References DistIface::RecvScheduler::popPacket(), and recvScheduler.

Referenced by DistEtherLink::RxLink::rxDone().

◆ packetOut()

void DistIface::packetOut	(	EthPacketPtr	pkt,
		Tick	send_delay
	)

Send out an Ethernet packet.

Parameters

pkt	The Ethernet packet to send.
send_delay	The delay in ticks for the send completion event.

Definition at line 647 of file dist_iface.cc.

References curTick(), DistHeaderPkt::dataDescriptor, DPRINTF, header, and sendPacket().

Referenced by DistEtherLink::TxLink::transmit().

◆ rankParam()

uint64_t DistIface::rankParam ( )

static

Getter for the dist rank param.

Definition at line 917 of file dist_iface.cc.

References primary, rank, X86ISA::val, and warn.

Referenced by PseudoInst::initParam().

◆ readyToCkpt()

bool DistIface::readyToCkpt	(	Tick	delay,
		Tick	period
	)

static

Initiate taking a checkpoint.

Parameters

delay Delay param from the m5 checkpoint command. If Delay is zero then a collaborative checkpoint is requested (i.e. all nodes have to call this method before the checkpoint can be taken). If Delay is not zero then a checkpoint is requested asap (and it will happen at the next sync tick).

Returns: False if we are in dist mode (i.e. exit can happen only at sync), True otherwise.

Definition at line 820 of file dist_iface.cc.

References DistHeaderPkt::collective, DPRINTF, DistHeaderPkt::immediate, inform, primary, DistIface::Sync::requestCkpt(), and sync.

Referenced by PseudoInst::m5checkpoint().

◆ readyToExit()

bool DistIface::readyToExit ( Tick delay )

static

Initiate the exit from the simulation.

Parameters

delay Delay param from the m5 exit command. If Delay is zero then a collaborative exit is requested (i.e. all nodes have to call this method before the distributed simulation can exit). If Delay is not zero then exit is requested asap (and it will happen at the next sync tick).

Returns: False if we are in distributed mode (i.e. exit can happen only at sync), True otherwise.

Definition at line 892 of file dist_iface.cc.

References DistHeaderPkt::collective, DPRINTF, DistHeaderPkt::immediate, inform, primary, DistIface::Sync::requestExit(), BaseGlobalEvent::scheduled(), DistIface::SyncEvent::start(), sync, and syncEvent.

Referenced by exitImpl(), and PseudoInst::m5exit().

◆ recvHeader()

virtual bool DistIface::recvHeader ( Header & header )

privatepure virtual

Receive a header (i.e.

meta info describing a data packet or a control command) from the remote end.

Parameters

header The meta info structure to store the incoming header.

Implemented in TCPIface.

Referenced by recvThreadFunc().

◆ recvPacket()

virtual void DistIface::recvPacket	(	const Header &	header,
		EthPacketPtr &	packet
	)

privatepure virtual

Receive a packet from the remote end.

Parameters

header	Meta info about the incoming packet (obtanied by a previous call to the recvHedaer() method).
Pointer	to packet received.

Implemented in TCPIface.

Referenced by recvThreadFunc().

◆ recvThreadFunc()

void DistIface::recvThreadFunc	(	Event *	recv_done,
		Tick	link_delay
	)

private

The function executed by a receiver thread.

Definition at line 669 of file dist_iface.cc.

References DistIface::Sync::abort(), curEventQueue(), DistHeaderPkt::dataDescriptor, exitSimLoop(), header, DistIface::RecvScheduler::init(), EventQueue::lock(), DistIface::Sync::progress(), DistIface::RecvScheduler::pushPacket(), recvHeader(), recvPacket(), recvScheduler, sync, and EventQueue::unlock().

Referenced by spawnRecvThread().

◆ sendCmd()

virtual void DistIface::sendCmd ( const Header & header )

privatepure virtual

Send out a control command to the remote end.

Parameters

header Meta info describing the command (e.g. sync request)

Implemented in TCPIface.

Referenced by DistIface::SyncNode::run(), and DistIface::SyncSwitch::run().

◆ sendPacket()

virtual void DistIface::sendPacket	(	const Header &	header,
		const EthPacketPtr &	packet
	)

privatepure virtual

Send out a data packet to the remote end.

Parameters

header	Meta info about the packet (which needs to be transferred to the destination alongside the packet).
packet	Pointer to the packet to send.

Implemented in TCPIface.

Referenced by packetOut().

◆ serialize()

void DistIface::serialize ( CheckpointOut & cp ) const

overridevirtual

Serialize an object.

Output an object's state into the current checkpoint section.

Parameters

cp	Checkpoint state

Implements Serializable.

Definition at line 745 of file dist_iface.cc.

References distIfaceId, DistIface::Sync::drainComplete(), primary, rank, recvScheduler, SERIALIZE_SCALAR, Serializable::serializeSection(), and sync.

◆ sizeParam()

uint64_t DistIface::sizeParam ( )

static

Getter for the dist size param.

Definition at line 930 of file dist_iface.cc.

References primary, size, X86ISA::val, and warn.

Referenced by PseudoInst::initParam().

◆ spawnRecvThread()

void DistIface::spawnRecvThread	(	const Event *	recv_done,
		Tick	link_delay
	)

private

spawn the receiver thread.

Parameters

recv_done	The receive done event associated with the simulated Ethernet link.
link_delay	The link delay for the simulated Ethernet link.

Definition at line 715 of file dist_iface.cc.

References recvThread, recvThreadFunc(), and recvThreadsNum.

Referenced by init().

◆ startup()

void DistIface::startup ( )

Definition at line 810 of file dist_iface.cc.

References DPRINTF, isSwitch, primary, DistIface::SyncEvent::start(), syncEvent, and syncStartOnPseudoOp.

Referenced by DistEtherLink::startup().

◆ toggleSync()

void DistIface::toggleSync ( ThreadContext * tc )

static

Trigger the primary to start/stop synchronization.

Definition at line 851 of file dist_iface.cc.

References ThreadContext::Active, DistHeaderPkt::collective, ThreadContext::getSystemPtr(), inform, primary, ThreadContext::quiesce(), ThreadContext::quiesceTick(), DistIface::Sync::requestStopSync(), BaseGlobalEvent::scheduled(), DistIface::SyncEvent::start(), ThreadContext::status(), sync, syncEvent, sys, System::threads, and BaseGlobalEvent::when().

Referenced by PseudoInst::togglesync().

◆ unserialize()

void DistIface::unserialize ( CheckpointIn & cp )

overridevirtual

Unserialize an object.

Read an object's state from the current checkpoint section.

Parameters

cp	Checkpoint state

Implements Serializable.

Definition at line 764 of file dist_iface.cc.

References distIfaceId, panic_if, primary, rank, recvScheduler, sync, UNSERIALIZE_SCALAR, and Serializable::unserializeSection().

Member Data Documentation

◆ distIfaceId

unsigned DistIface::distIfaceId

protected

Unique id for the dist link.

Definition at line 492 of file dist_iface.hh.

Referenced by DistIface(), TCPIface::establishConnection(), serialize(), and unserialize().

◆ distIfaceNum

unsigned DistIface::distIfaceNum = 0

staticprotected

Number of DistIface objects (i.e.

dist links in this gem5 process)

Definition at line 488 of file dist_iface.hh.

Referenced by DistIface(), TCPIface::establishConnection(), and ~DistIface().

◆ isPrimary

bool DistIface::isPrimary

protected

Definition at line 494 of file dist_iface.hh.

Referenced by DistIface(), and TCPIface::TCPIface().

◆ isSwitch

bool DistIface::isSwitch = false

staticprivate

Is this node a switch?

Definition at line 521 of file dist_iface.hh.

Referenced by DistIface(), DistIface::SyncEvent::process(), and startup().

◆ primary

DistIface * DistIface::primary = nullptr

staticprivate

The very first DistIface object created becomes the primary interface.

We need a primary interface to co-ordinate the global synchronisation.

Definition at line 513 of file dist_iface.hh.

Referenced by DistIface(), drain(), drainResume(), init(), rankParam(), readyToCkpt(), readyToExit(), DistIface::SyncNode::run(), DistIface::SyncSwitch::run(), serialize(), sizeParam(), startup(), toggleSync(), unserialize(), and ~DistIface().

◆ rank

unsigned DistIface::rank

protected

The rank of this process among the gem5 peers.

Definition at line 480 of file dist_iface.hh.

Referenced by TCPIface::establishConnection(), init(), rankParam(), serialize(), and unserialize().

◆ recvScheduler

RecvScheduler DistIface::recvScheduler

private

Meta information about data packets received.

Definition at line 470 of file dist_iface.hh.

Referenced by drainResume(), packetIn(), recvThreadFunc(), serialize(), and unserialize().

◆ recvThread

std::thread* DistIface::recvThread

private

Receiver thread pointer.

Each DistIface object must have exactly one receiver thread.

Definition at line 466 of file dist_iface.hh.

Referenced by spawnRecvThread(), and ~DistIface().

◆ recvThreadsNum

unsigned DistIface::recvThreadsNum = 0

staticprivate

Number of receiver threads (in this gem5 process)

Definition at line 500 of file dist_iface.hh.

Referenced by DistIface::SyncNode::run(), and spawnRecvThread().

◆ size

unsigned DistIface::size

protected

The number of gem5 processes comprising this dist simulation.

Definition at line 484 of file dist_iface.hh.

Referenced by sizeParam(), and TCPIface::TCPIface().

◆ sync

DistIface::Sync * DistIface::sync = nullptr

staticprivate

The singleton Sync object to perform dist synchronisation.

Definition at line 504 of file dist_iface.hh.

Referenced by DistIface(), init(), DistIface::SyncEvent::process(), readyToCkpt(), readyToExit(), recvThreadFunc(), serialize(), DistIface::SyncEvent::start(), toggleSync(), unserialize(), and ~DistIface().

◆ syncEvent

DistIface::SyncEvent * DistIface::syncEvent = nullptr

staticprivate

The singleton SyncEvent object to schedule periodic dist sync.

Definition at line 508 of file dist_iface.hh.

Referenced by DistIface(), drain(), drainResume(), readyToExit(), startup(), toggleSync(), and ~DistIface().

◆ syncRepeat

Tick DistIface::syncRepeat

private

Frequency of dist sync events in ticks.

Definition at line 461 of file dist_iface.hh.

Referenced by init().

◆ syncStart

Tick DistIface::syncStart

private

Tick to schedule the first dist sync event.

This is just as optimization : we do not need any dist sync event until the simulated NIC is brought up by the OS.

Definition at line 457 of file dist_iface.hh.

Referenced by init().

◆ syncStartOnPseudoOp

bool DistIface::syncStartOnPseudoOp

private

Use pseudoOp to start synchronization.

Definition at line 474 of file dist_iface.hh.

Referenced by startup().

◆ sys

System * DistIface::sys = nullptr

staticprivate

System pointer used to wakeup sleeping threads when stopping sync.

Definition at line 517 of file dist_iface.hh.

Referenced by toggleSync().

The documentation for this class was generated from the following files:

dev/net/dist_iface.hh
dev/net/dist_iface.cc

Classes

Public Types

Public Member Functions

Static Public Member Functions

Protected Types

Protected Attributes

Static Protected Attributes

Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ Header

◆ MsgType

◆ ReqType

Constructor & Destructor Documentation

◆ DistIface()

◆ ~DistIface()

Member Function Documentation

◆ drain()

◆ drainResume()

◆ init()

◆ initTransport()

◆ packetIn()

◆ packetOut()

◆ rankParam()

◆ readyToCkpt()

◆ readyToExit()

◆ recvHeader()

◆ recvPacket()

◆ recvThreadFunc()

◆ sendCmd()

◆ sendPacket()

◆ serialize()

◆ sizeParam()

◆ spawnRecvThread()

◆ startup()

◆ toggleSync()

◆ unserialize()

Member Data Documentation

◆ distIfaceId

◆ distIfaceNum

◆ isPrimary

◆ isSwitch

◆ primary

◆ rank

◆ recvScheduler

◆ recvThread

◆ recvThreadsNum

◆ size

◆ sync

◆ syncEvent

◆ syncRepeat

◆ syncStart

◆ syncStartOnPseudoOp

◆ sys