risc-v privileged architecture · risc-v virtual memory control by default, s-mode can’t access...

RISC-VPrivilegedArchitectureAndrewWaterman

SiFive Inc.andrew@sifive.com

6th RISC-VWorkshopShanghaiJiaoTongUniversity

May9,2017

NewDraftSpecificationsareReleased

§ UserISAv2.2- Improvementstodocumentation-Defineshownarrow-precisionFPtypesarerepresentedinwiderFPregisters-DefinesbehaviorusingwiderFPoperationsonnarrowoperands

§ PrivilegedISAv1.10-Subjectofthistalk

§ SpecsavailableonGithub (source&PDF)§ https://github.com/riscv/riscv-isa-manual§ Movedtoeditorshipmodel

PrivilegedArchitectureisStable

§ 1.10keepscompatibilitywith1.9.1formachine-mode-onlyimplementations

§ Futurereleasesshouldbecompatiblewith1.10forsupervisorISA,too

§ Caveat:theseareproposals;notyetratifiedbyFoundation

RISC-VPrivilegedArchitecture

§ Providecleansplitbetweenlayersofthesoftwarestack§ ApplicationcommunicateswithApplicationExecutionEnvironment(AEE)viaApplicationBinaryInterface(ABI)

§ OScommunicatesviaSupervisorExecutionEnvironment(SEE)viaSystemBinaryInterface(SBI)

§ HypervisorcommunicatesviaHypervisorBinaryInterfacetoHypervisorExecutionEnvironment

§ AlllevelsofISAdesignedtosupportvirtualization4

2 1.1draft: Volume II: RISC-V Privileged Architectures

ApplicationABIAEE

ApplicationABI

OSSBISEE

ApplicationABI

SBIHypervisor

ApplicationABI

OS

ApplicationABI

ApplicationABI

OS

ApplicationABI

SBI

HBIHEE

Figure 1.1: Di↵erent implementation stacks supporting various forms of privileged execution.

the OS, which provides the AEE. Just as applications interface with an AEE via an ABI, RISC-Voperating systems interface with a supervisor execution environment (SEE) via a supervisor binaryinterface (SBI). An SBI comprises the user-level and supervisor-level ISA together with a set ofSBI function calls. Using a single SBI across all SEE implementations allows a single OS binaryimage to run on any SEE. The SEE can be a simple boot loader and BIOS-style IO system in alow-end hardware platform, or a hypervisor-provided virtual machine in a high-end server, or athin translation layer over a host operating system in an architecture simulation environment.

The rightmost configuration shows a virtual machine monitor configuration where multiple multi-programmed OSs are supported by a single hypervisor. Each OS communicates via an SBI with thehypervisor, which provides the SEE. The hypervisor communicates with the hypervisor executionenvironment (HEE) using a hypervisor binary interface, to isolate the hypervisor from details ofthe hardware platform.

Our graphical convention represents abstract interfaces using black boxes with white text, toseparate them from actual components.

The various ABI, SBI, and HBIs are still a work-in-progress, but we anticipate the SBI and HBIto support devices via virtualized device interfaces similar to virtio [2], and to support devicediscovery. In this manner, only one set of device drivers need be written that can support anyOS or hypervisor, and which can also be shared with the boot environment.

Hardware implementations of the RISC-V ISA will generally require additional features beyond theprivileged ISA to support the various execution environments (AEE, SEE, or HEE), but these weconsider separately as part of a hardware abstraction layer (HAL), as shown in Figure 1.2. Note

ApplicationABIAEEHAL

Hardware

ApplicationABI

OSSBISEE

ApplicationABI

HALHardware

SBIHypervisor

ApplicationABI

OS

ApplicationABI

ApplicationABI

OS

ApplicationABI

SBI

HBIHEE

HardwareHAL

Figure 1.2: Hardware abstraction layers (HALs) abstract underlying hardware platforms from theexecution environments.

RISC-VPrivilegeModes

§ Threeprivilegemodes-User(U-mode)- Supervisor(S-mode)-Machine(M-mode)

§ Supportedcombinationsofmodes:-M (simpleembeddedsystems)-M,U (embeddedsystemswithprotection)-M,S,U (systemsrunningUnix-likeoperatingsystems)

§ Plannedsupportforhypervisors

5

SimpleEmbeddedSystems

§ SimplestimplementationneedsonlyM-mode§ Noaddresstranslation§ Minimalmemoryprotection- Trapbadphysicaladdressesprecisely

§ Applicationcodeistrusted

§ Lowimplementationcost- 27 bitsofarchitecturalstate(inadditiontouserISA)- +27 morebitsfortimers- +27 moreforbasicperformancecounters

6

EmbeddedSystemswithProtection

§ Applicationcodeisnottrusted§ AddU-mode;runappcodeinU-modeandtrustedcodeinM-mode

§ PossiblyaddN extensionforuser-levelinterrupts§ Stillnoaddresstranslation§ Needmechanismtoprotectphysicalmemory

7

PhysicalMemoryProtectionUnit

§ Optionalnewfeatureinv1.10§ WhenPMPisimplemented,modesbelowM-modehavenomemorypermissionsbydefault

§ CangrantR/W/Xpermissionson≥4-bytegranularity§ Upto16PMPregions§ EachPMPregionisanynaturally alignedpower-of-2numberofbytes

§ CanconfigureadjacentPMPregisterstoformanarbitrarybase-and-boundsregioninstead

§ PMPscanbelocked (can’tberewrittenuntilreset),inwhichcasetheyaffectM-mode,too

8

SupportforUnix-likeOperatingSystems

§ AddS-modetoprovidevirtualmemory§ Memorydividedinto4KiB basepages§ Radix-treepagetables-2levelsforRV32(Sv32)-3or4levelsforRV64(Sv39,Sv48)-EncodingspacereservedforSv57/Sv64

§ Superpages possibleatalllevelsofpagetable-e.g.2MiB and1GiB forSv39

§ HardwarePTwalksspecifiedinsupervisorISA-CantraptoM-modeforsoftwareTLBrefill

InteractionbetweenPMPandVM

§ PhysicalMemoryProtectionandpage-basedvirtualmemoryarecomposable

§ Addresstranslationhappensfirst,possiblygeneratingpage-faultexceptions

§ PMPunitcheckstranslatedaddress,possiblygeneratingaccessexceptions

§ UsefulwhenS-modecodeisuntrusted

RISC-VPageTableEntries

§ SeparatelycontrolledR,W,Xpermissions- SupportsX-onlypages-W&~Rcombinationreserved

§ Supervisorcan’taccessuserpagesbydefault§ Globalbitindicatesthemappingbelongstoalladdressspaces(e.g.kernelpagesinaUnixsystem)

§ Accessed/DirtybitsoptionallymanagedbyHW-Updatesmustbeatomicw.r.t.permissionscheck- Complextoimplement,sopermittrappingwhenA/Dnotsetinstead

RISC-VVirtualMemoryControl

§ Bydefault,S-modecan’taccessuserpages-HelpsdetectOS/driverbugs-Stillneedabilitytoreadusermemory,e.g.onsystemcall-Set“SupervisorAccesstoUserMemory”bitinsstatus toreadusermemory,thenturnitoffagain

§ Similarly,S-modecan’treadexecute-onlypages-Set“MakeExecutableReadable”bitinsstatus tooverride-Usefulforillegal-instructiontraphandlers

§ S-modecanenable/disableVMandchoosepage-tabledepthinsatp register

RISC-VInterruptDesignGoals

§ Simplicity§ Supportallkindsofplatformsfrommicrocontrollerstovirtualizedservers

§ Enabletradeoffsbetweenperformanceandimplementationcost

§ Flexibilitytosupportspecializedneeds

13

InterruptUsesinDifferentApplications

§ High-performanceUnix-likesystems- Interrupthandlingsmallfractionofprocessingtime- Fastcores,smartdevices

-Minimalinterrupthandler- Schedulinginsoftware

§ Low/midembeddedsystems- Interrupthandlingsignificantfractionofprocessortime- Slowcores,dumbdevices

- Significantfractionofcodeinhandlers- Interruptcontrolleractsastaskscheduler

§ High-performancereal-timesystems- Can’twastetimeoninterruptoverhead- HandlerspollI/Odeviceswithregularheartbeat

§ Andeverythinginbetween

CategorizingSourcesofRISC-VInterrupts

§ LocalInterrupts- Directlyconnectedtoonehart-Noarbitrationbetweenhartstoservice- Determinesourcedirectlythroughxcause CSR-Onlytwostandardlocalinterrupts(software,timer)

§ Global(External)Interrupts- RoutedviaPlatform-LevelInterruptController(PLIC)- PLICarbitratesbetweenmultiplehartsclaiminginterrupt- Readofmemory-mappedregisterreturnssource

15

MachineInterruptPendingCSR(mip)

§ mip reflectspendingstatusofinterruptsforhart§ Separateinterruptsforeachsupportedprivilegelevel(M/S/U)

§ User-levelinterrupthandling(“N”)optionalfeaturewhenU-modepresent(discussedlater)

16

ExternalfromPLIC LocalTimer LocalSoftware

(AddNon-StandardLocalInterruptsHere)

SoftwareInterrupts

§ MSIP-Onlywriteableinmachine-modeviamemory-mappedcontrolregister(mappingisplatform-specific)

-Onehartcanwritetodifferenthart’sMSIPregister-Mechanismforinter-hartinterrupts

§ SSIP andUSIP- HartcanonlywritebitxSIP inownmip registerwhenrunningatprivilegemode xorgreater

§ App/OS canonlyperforminter-hartinterruptsviaABI/SBIcalls- Destinationvirtualhartmightbedescheduled- InterruptsvirtualizedbyM-modesoftwareusingMSIP

17

TimerInterrupts

§ MTIP- Single64-bitreal-timehardwaretimerandcomparatorinM-mode

-MTIPsetwhenmtime >=mtimecmp-MTIPclearedbywritingnewmtimecmp value

§ STIP andUTIP-M-modemultiplexessinglehardwaretimerandcomparatorforlower-privilegemodesonsamehart

- ABI/SBI callstosetuptimer-M-modesoftwarewrites/clearsSTIP/UTIP

§ MostsystemswillalsohaveotherhardwaretimersattachedviaPLICetc.

18

ExternalInterrupts

§ MEIP,SEIP,UEIP- InputsfromaPlatform-LevelInterruptController(PLIC)- EachprivilegemodehasitsowninputfromPLIC- Interruptsclearedwithloads/storestoPLIC- SoftwarecaninjectSEIPandUEIPinterruptstosupportvirtualizingthePLIC

19

Platform-LevelInterruptController(PLIC)

20

PLIC

U TimerSoftware

S TimerSoftware

M TimerSoftware

ExternalInterrupt1

ExternalInterrupt2

LocalInterruptsGlobalInterrupts

ExternalHart0

Hart1U Timer

Software

S TimerSoftware

M TimerSoftware

Gateway

PriorityIP

Interrupt Request

Interrupt 2 Signals

Gateway

PriorityIP

Interrupt Request

Interrupt 1 Signals

PLIC Core

PLIC Gateways

0

0 Max IDThresholdMax Pri.

EIP>? EIP Interrupt Notification

Interrupt ID}To Target

0

IE >?

101

01

IE >?

201

01

0

0 Max IDThresholdMax Pri.

EIP>? EIP Interrupt Notification

Interrupt ID}To Target

1

IE >?

101

01

IE >?

201

01

Platform-LevelInterruptController

21

MachineInterruptEnableCSR(mie)

§ miemirrorslayoutofmip§ providesper-interruptenables

§ Also,globalinterruptenablesinmstatus foreachprivilegemode

§ Interruptsalwaysdisabledforlowerprivilegemodes;alwaysenabledforhigherprivilegemodes

22

ExternalfromPLIC LocalTimer LocalSoftware

AllinterruptstraptoM-modebydefault

§ mcause registerindicateswhichinterruptoccured§ M-modecanredirecttoappropriateprivilegelevelusingMRETinstruction

23

OptionalInterruptHandlerDelegation

§ Candelegateinterrupt(andexception)handlingtolowerprivilegeleveltoreduceoverhead

§ mideleg hassamelayoutasmip§ Ifabitissetinmideleg thencorrespondinginterruptdelegatedtonextlowestprivilegelevel(SorU)

§ Canbedelegatedagainusingsideleg§ Oncedelegated,theinterruptwillnotaffectcurrentprivilegelevel(mie settingignored)

24

HypervisorStatus

§ Previousspecsketchedafourthprivilegemode,H,aboveS(M/H/S/U)

§ DesignedforType-1hypervisorsupport§ FeedbackfromcommunityledustopursueHWsupportforType-2hypervisors(likeKVM)instead-StillworkswellforType-1hypervisors

§ PlanistomakeafullproposalbySeptember

25

ImplementationStatus

§ SpikeandUCBRocket-Chipconformtov1.10§ Linuxporttov1.10workswithSpike/Rocket-Workingonupstreaming theLinuxkernel

§ UpstreamGCCandbinutils portsarecompatible

26

Questions?

27

Specsavailableathttps://github.com/riscv/riscv-isa-manual