Part II: Modifying and extending gem5

Setting up development environment

• Use the style guide
• Install the style guide
• Use mercurial queues (or git or whatever)

---

Simple SimObject

To come

---

Debugging gem5

To come

---

Event-driven programming

• Add an event wrapper to the HelloObject from last chapter.
• Add a processEvent function

hello_object.hh

private:
  void processEvent();

  EventWrapper<HelloObject, &HelloObject::processEvent> event;

• Initialize the event
• Implement the processEvent function

hello_object.cc

HelloObject::HelloObject(HelloObjectParams *params) :
    SimObject(params), event(*this)

void
HelloObject::processEvent()
{
    DPRINTF(Hello, "Hello world! Processing the event!\n");
}

• Add a startup function to the header
• Schedule an event

hello_object.hh

void startup();

hello_object.cc

void
HelloObject::startup()
{
    schedule(event, 100);
}

• Recompile and run gem5

---

• Add two parameters to class: latency, timesLeft

hello_object.hh

Tick latency;

int timesLeft;

• Initialize these parameters

hello_object.cc

HelloObject::HelloObject(HelloObjectParams *params) :
    SimObject(params), event(*this), latency(100), timesLeft(10)

• update startup and process event

hello_object.cc

void
HelloObject::startup()
{
    schedule(event, latency);
}

void
HelloObject::processEvent()
{
    timesLeft--;
    DPRINTF(Hello, "Hello world! Processing the event! %d left\n", timesLeft);

    if (timesLeft <= 0) {
        DPRINTF(Hello, "Done firing!\n");
    } else {
        schedule(event, curTick() + latency);
    }
}

---

Adding parameters

• Talk about simple parameters

HelloObject.py

class HelloObject(SimObject):
    type = 'HelloObject'
    cxx_header = "learning_gem5/hello_object.hh"

    time_to_wait = Param.Latency("Time before firing the event")
    number_of_fires = Param.Int(1, "Number of times to fire the event before "
                                   "goodbye")

• Update the constructor

hello_object.cc

HelloObject::HelloObject(HelloObjectParams *params) :
    SimObject(params),
    event(*this),
    myName(params->name),
    latency(params->time_to_wait),
    timesLeft(params->number_of_fires)
{
    DPRINTF(Hello, "Created the hello object with the name %s\n", myName);
}

• Run gem5 without updating the config file and get an error
• Fix the above error

run_hello.py

root.hello = HelloObject(time_to_wait = '2us')

• or

root.hello = HelloObject()
root.hello.time_to_wait = '2us'

• Run again
• Modify config to fire more than once
• Run again

root.hello.number_of_fires = 10

---

• Clear changes, add 2 patches to hello-4-goodbye
• Add a goodbye object to the python file
• Talk about what this object is going to do. It has a buffer that it fills at some maximum rate.

HelloObject.py

class GoodbyeObject(SimObject):
    type = 'GoodbyeObject'
    cxx_header = "learning_gem5/goodbye_object.hh"

    buffer_size = Param.MemorySize('1kB',
                                   "Size of buffer to fill with goodbye")
    write_bandwidth = Param.MemoryBandwidth('100MB/s', "Bandwidth to fill "
                                            "the buffer")

• Show “sayGoodbye” interface

goodbye_object.hh

void sayGoodbye(std::string name);

• Implement sayGoodbye

goodbye_object.cc

void
GoodbyeObject::sayGoodbye(std::string other_name)
{
    DPRINTF(Hello, "Saying goodbye to %s\n", other_name);

    message = "Goodbye " + other_name + "!! ";

    fillBuffer();
}

• Implement missing fillBuffer

goodbye_object.cc

if (bufferUsed < bufferSize - 1) {
    // Wait for the next copy for as long as it would have taken
    DPRINTF(Hello, "Scheduling another fillBuffer in %d ticks\n",
            bandwidth * bytes_copied);
    schedule(event, curTick() + bandwidth * bytes_copied);
} else {
    DPRINTF(Hello, "Goodbye done copying!\n");
    // Be sure to take into account the time for the last bytes
    exitSimLoop(buffer, 0, curTick() + bandwidth * bytes_copied);
}

• Make changes to hello object

HelloObject.py

goodbye_object = Param.GoodbyeObject("A goodbye object")

hello_object.hh

GoodbyeObject& goodbye;

hello_object.cc

if (timesLeft <= 0) {
    DPRINTF(Hello, "Done firing!\n");
    goodbye.sayGoodbye(myName);

• Update the config script

run_hello.py

root.hello.goodbye_object = GoodbyeObject(buffer_size='100B')

• Run gem5
• Make other changes for examples

run_hello.py

root.hello = HelloObject(time_to_wait = '2us', number_of_fires = 5)
root.hello2 = HelloObject(time_to_wait = '2.5us', number_of_fires = 3)

root.goodbye = GoodbyeObject(buffer_size = '30B')
root.goodbye2 = GoodbyeObject(buffer_size = '30B')

root.hello.goodbye_object = root.goodbye2
root.hello2.goodbye_object = root.goodbye

• Run gem5

---

MemObjects

• Show slides about master/slave and packets
• packets * Request (addr, requestor) * command (can change) * size * data (pointer)
• port interface

Simple master-slave interaction when both can accept the request and the response.

Simple master-slave interaction when the slave is busy

Simple master-slave interaction when the master is busy

---

• Make the SConscript file

Sconscript

Import('*')

SimObject('SimpleMemobj.py')
Source('simple_memobj.cc')

DebugFlag('SimpleMemobj')

• Create a simple memobj description

SimpleMemobj.py

from m5.params import *
from m5.proxy import *
from MemObject import MemObject

class SimpleMemobj(MemObject):
    type = 'SimpleMemobj'
    cxx_header = "learning_gem5/simple_memobj/simple_memobj.hh"

    inst_port = SlavePort("CPU side port, receives requests")
    data_port = SlavePort("CPU side port, receives requests")
    mem_side = MasterPort("Memory side port, sends requests")

• Define the header file
• Point out “public MemObject”

simple_memobj.hh

#ifndef __LEARNING_GEM5_SIMPLE_MEMOBJ_SIMPLE_MEMOBJ_HH__
#define __LEARNING_GEM5_SIMPLE_MEMOBJ_SIMPLE_MEMOBJ_HH__

#include "mem/mem_object.hh"
#include "params/SimpleMemobj.hh"

class SimpleMemobj : public MemObject
{
  private:

  public:

    /** constructor
     */
    SimpleMemobj(SimpleMemobjParams *params);
};
#endif

• Define the CPU-side slave port
• Talk about each of the functions below

simple_memobj.hh

class CPUSidePort : public SlavePort
{
  private:
    SimpleMemobj *owner;

  public:
    CPUSidePort(const std::string& name, SimpleMemobj *owner) :
        SlavePort(name, owner), owner(owner)
    { }

    AddrRangeList getAddrRanges() const override;

  protected:
    Tick recvAtomic(PacketPtr pkt) override { panic("recvAtomic unimpl."); }
    void recvFunctional(PacketPtr pkt) override;
    bool recvTimingReq(PacketPtr pkt) override;
    void recvRespRetry() override;
};

• define the memory side master port
• Talk about each of the functions below

simple_memobj.hh

class MemSidePort : public MasterPort
{
  private:
    SimpleMemobj *owner;

  public:
    MemSidePort(const std::string& name, SimpleMemobj *owner) :
        MasterPort(name, owner), owner(owner)
    { }

  protected:
    bool recvTimingResp(PacketPtr pkt) override;
    void recvReqRetry() override;
    void recvRangeChange() override;
};

• Define the MemObject interface

simple_memobj.hh

class SimpleMemobj : public MemObject
{
  private:

    <CPUSidePort declaration>
    <MemSidePort declaration>

    CPUSidePort instPort;
    CPUSidePort dataPort;

    MemSidePort memPort;

  public:
    SimpleMemobj(SimpleMemobjParams *params);

    BaseMasterPort& getMasterPort(const std::string& if_name,
                                  PortID idx = InvalidPortID) override;

    BaseSlavePort& getSlavePort(const std::string& if_name,
                                PortID idx = InvalidPortID) override;

};

• Initialize things in construcutor

simple_memobj.cc

SimpleMemobj::SimpleMemobj(SimpleMemobjParams *params) :
    MemObject(params),
    instPort(params->name + ".inst_port", this),
    dataPort(params->name + ".data_port", this),
    memPort(params->name + ".mem_side", this),
{
}

• Implement getMasterPort

simple_memobj.cc

BaseMasterPort&
SimpleMemobj::getMasterPort(const std::string& if_name, PortID idx)
{
    if (if_name == "mem_side") {
        return memPort;
    } else {
        return MemObject::getMasterPort(if_name, idx);
    }
}

• Implement getSlavePort

simple_memobj.cc

BaseSlavePort&
SimpleMemobj::getSlavePort(const std::string& if_name, PortID idx)
{
    if (if_name == "inst_port") {
        return instPort;
    } else if (if_name == "data_port") {
        return dataPort;
    } else {
        return MemObject::getSlavePort(if_name, idx);
    }
}

• Pass through some of the functions for CPU side port

simple_memobj.cc

AddrRangeList
SimpleMemobj::CPUSidePort::getAddrRanges() const
{
    return owner->getAddrRanges();
}

void
SimpleMemobj::CPUSidePort::recvFunctional(PacketPtr pkt)
{
    return owner->handleFunctional(pkt);
}

void
SimpleMemobj::handleFunctional(PacketPtr pkt)
{
    memPort.sendFunctional(pkt);
}

AddrRangeList
SimpleMemobj::getAddrRanges() const
{
    DPRINTF(SimpleMemobj, "Sending new ranges\n");
    return memPort.getAddrRanges();
}

• Pass through some of the functions for Mem side port

simple_memobj.cc

void
SimpleMemobj::MemSidePort::recvRangeChange()
{
    owner->sendRangeChange();
}

void
SimpleMemobj::sendRangeChange()
{
    instPort.sendRangeChange();
    dataPort.sendRangeChange();
}

---

• NOw the fun part. Implementing the send/receives
• Let’s start with receive

simple_memobj.cc

bool
SimpleMemobj::CPUSidePort::recvTimingReq(PacketPtr pkt)
{
    if (!owner->handleRequest(pkt)) {
        needRetry = true;
        return false;
    } else {
        return true;
    }
}

• Now, we need to do handle request

simple_memobj.cc

bool
SimpleMemobj::handleRequest(PacketPtr pkt)
{
    if (blocked) {
        return false;
    }
    DPRINTF(SimpleMemobj, "Got request for addr %#x\n", pkt->getAddr());
    blocked = true;
    memPort.sendPacket(pkt);
    return true;
}

• Let’s add a convienency function in the memside port

simple_memobj.cc

void
SimpleMemobj::MemSidePort::sendPacket(PacketPtr pkt)
{
    panic_if(blockedPacket != nullptr, "Should never try to send if blocked!");
    if (!sendTimingReq(pkt)) {
        blockedPacket = pkt;
    }
}

simple_memobj.hh

class MemSidePort : public MasterPort {
    PacketPtr blockedPacket;
  public:
    void sendPacket(PacketPtr pkt);

• Implement code to handle retries

simple_memobj.cc

void
SimpleMemobj::MemSidePort::recvReqRetry()
{
    assert(blockedPacket != nullptr);

    PacketPtr pkt = blockedPacket;
    blockedPacket = nullptr;

    sendPacket(pkt);
}

---

• Implement the code for receiving requests

simple_memobj.cc

bool
SimpleMemobj::CPUSidePort::recvTimingReq(PacketPtr pkt)
{
    if (!owner->handleRequest(pkt)) {
        needRetry = true;
        return false;
    } else {
        return true;
    }
}

• Add variable to remember when we need to send the CPU a retry

simple_memobj.hh

class CPUSidePort : public SlavePort
{
    bool needRetry;

simple_memobj.cc

bool
SimpleMemobj::handleResponse(PacketPtr pkt)
{
    assert(blocked);
    DPRINTF(SimpleMemobj, "Got response for addr %#x\n", pkt->getAddr());

    blocked = false;

    // Simply forward to the memory port
    if (pkt->req->isInstFetch()) {
        instPort.sendPacket(pkt);
    } else {
        dataPort.sendPacket(pkt);
    }

    return true;
}

• Now, we need the convience function to send packets

simple_memobj.hh

class CPUSidePort : public SlavePort
{
    PacketPtr blockedPacket;
  public:
    void sendPacket(PacketPtr pkt);

simple_memobj.cc

void
SimpleMemobj::CPUSidePort::sendPacket(PacketPtr pkt)
{
    panic_if(blockedPacket != nullptr, "Should never try to send if blocked!");

    if (!sendTimingResp(pkt)) {
        blockedPacket = pkt;
    }
}

• Implement recvRespRetry

simple_memobj.cc

void
SimpleMemobj::CPUSidePort::recvRespRetry()
{
    assert(blockedPacket != nullptr);

    PacketPtr pkt = blockedPacket;
    blockedPacket = nullptr;

    sendPacket(pkt);
}

• Implement trySendRetry

simple_memobj.hh

class CPUSidePort : public SlavePort {
    void trySendRetry();

simple_memobj.cc

void
SimpleMemobj::CPUSidePort::trySendRetry()
{
    if (needRetry && blockedPacket == nullptr) {
        needRetry = false;
        DPRINTF(SimpleMemobj, "Sending retry req for %d\n", id);
        sendRetryReq();
    }
}

simple_memobj.cc

SimpleMemobj::handleResponse(PacketPtr pkt)
{
    instPort.trySendRetry();
    dataPort.trySendRetry();

---

• Update simple config file

simple.py

system.cpu = TimingSimpleCPU()

system.memobj = SimpleMemobj()

system.cpu.icache_port = system.memobj.inst_port
system.cpu.dcache_port = system.memobj.data_port

system.membus = SystemXBar()

system.memobj.mem_side = system.membus.slave

• Run simple.py

---

Making a cache

• Add parameters to memobj

SimpleMemobj.py

latency = Param.Cycles(1, "Cycles taken on a hit or to resolve a miss")

size = Param.MemorySize('16kB', "The size of the cache")

system = Param.System(Parent.any, "The system this cache is part of")

• Talk about the parent.any proxy parameter
• Add latency/size/system to constructor

simple_memobj.cc

latency(params->latency),
blockSize(params->system->cacheLineSize()),
capacity(params->size / blockSize),

• Implement new “handleRequest”

simple_memobj.cc

bool
SimpleMemobj::handleRequest(PacketPtr pkt, int port_id)
{
    if (blocked) {
        return false;
    }
    DPRINTF(SimpleMemobj, "Got request for addr %#x\n", pkt->getAddr());

    blocked = true;
    waitingPortId = port_id;

    schedule(new AccessEvent(this, pkt), clockEdge(latency));

    return true;
}

• Talk about the clockEdge function and clocked-objects
• Implement the access event

simple_memobj.hh

class AccessEvent : public Event
{
  private:
    SimpleMemobj *cache;
    PacketPtr pkt;
  public:
    AccessEvent(SimpleMemobj *cache, PacketPtr pkt) :
        Event(Default_Pri, AutoDelete), cache(cache), pkt(pkt)
    { }
    void process() override {
        cache->accessTiming(pkt);
    }
};

• Implement the accessTiming function

simple_memobj.hh

void accessTiming(PacketPtr pkt);

simple_memobj.cc

void
SimpleMemobj::accessTiming(PacketPtr pkt)
{
    bool hit = accessFunctional(pkt);
    if (hit) {
        pkt->makeResponse();
        sendResponse(pkt);
    } else {
        <miss handling>
    }
}

• Note; It’s a good idea to separate out functional from timing functions
• Miss handling is complicated by the block size

simple_memobj.cc

void
SimpleMemobj::accessTiming(PacketPtr pkt)
{
    bool hit = accessFunctional(pkt);
    if (hit) {
        pkt->makeResponse();
        sendResponse(pkt);
    } else {
        Addr addr = pkt->getAddr();
        Addr block_addr = pkt->getBlockAddr(blockSize);
        unsigned size = pkt->getSize();
        if (addr == block_addr && size == blockSize) {
            DPRINTF(SimpleMemobj, "forwarding packet\n");
            memPort.sendPacket(pkt);
        } else {
            DPRINTF(SimpleMemobj, "Upgrading packet to block size\n");
            panic_if(addr - block_addr + size > blockSize,
                     "Cannot handle accesses that span multiple cache lines");

            assert(pkt->needsResponse());
            MemCmd cmd;
            if (pkt->isWrite() || pkt->isRead()) {
                cmd = MemCmd::ReadReq;
            } else {
                panic("Unknown packet type in upgrade size");
            }

            PacketPtr new_pkt = new Packet(pkt->req, cmd, blockSize);
            new_pkt->allocate();

            outstandingPacket = pkt;

            memPort.sendPacket(new_pkt);
        }
    }
}

simple_memobj.hh

PacketPtr outstandingPacket;

• Update handle response to be able to accept responses from the upgraded packets

simple_memobj.cc

bool
SimpleMemobj::handleResponse(PacketPtr pkt)
{
    assert(blocked);
    DPRINTF(SimpleMemobj, "Got response for addr %#x\n", pkt->getAddr());
    insert(pkt);

    if (outstandingPacket != nullptr) {
        accessFunctional(outstandingPacket);
        outstandingPacket->makeResponse();
        delete pkt;
        pkt = outstandingPacket;
        outstandingPacket = nullptr;
    } // else, pkt contains the data it needs

    sendResponse(pkt);

    return true;
}

---

• Implementing the functional cache logic, now.

simple_memobj.hh

void insert(PacketPtr pkt);
bool accessFunctional(PacketPtr pkt);
std::unordered_map<Addr, uint8_t*> cacheStore;

• Implement the access logic

simple_memobj.cc

bool
SimpleMemobj::accessFunctional(PacketPtr pkt)
{
    Addr block_addr = pkt->getBlockAddr(blockSize);
    auto it = cacheStore.find(block_addr);
    if (it != cacheStore.end()) {
        if (pkt->isWrite()) {
            pkt->writeDataToBlock(it->second, blockSize);
        } else if (pkt->isRead()) {
            pkt->setDataFromBlock(it->second, blockSize);
        } else {
            panic("Unknown packet type!");
        }
        return true;
    }
    return false;
}

• Implement the insert logic

simple_memobj.cc

void
SimpleMemobj::insert(PacketPtr pkt)
{
    if (cacheStore.size() >= capacity) {
        // Select random thing to evict. This is a little convoluted since we
        // are using a std::unordered_map. See http://bit.ly/2hrnLP2
        int bucket, bucket_size;
        do {
            bucket = random_mt.random(0, (int)cacheStore.bucket_count() - 1);
        } while ( (bucket_size = cacheStore.bucket_size(bucket)) == 0 );
        auto block = std::next(cacheStore.begin(bucket),
                               random_mt.random(0, bucket_size - 1));

        RequestPtr req = new Request(block->first, blockSize, 0, 0);
        PacketPtr new_pkt = new Packet(req, MemCmd::WritebackDirty, blockSize);
        new_pkt->dataDynamic(block->second); // This will be deleted later

        DPRINTF(SimpleMemobj, "Writing packet back %s\n", pkt->print());
        memPort.sendTimingReq(new_pkt);

        cacheStore.erase(block->first);
    }
    uint8_t *data = new uint8_t[blockSize];
    cacheStore[pkt->getAddr()] = data;

    pkt->writeDataToBlock(data, blockSize);
}

---

• update the config file

simple.py

system.memobj = SimpleMemobj(size='1kB')

• Run it!