design and implementaion of a web-based rc3e user frontend
TRANSCRIPT
Fredo Erxleben
Design and Implementaion of aWeb-Based RC3E User FrontendStatus Report // Dresden, April 12, 2018
Contents of this Talk
IntroductionRC3E
Django
REST
Data Model
Selected IssuesUser Interaction ConceptSupporting Multiple APIsReservation Scheduling
Results and Future Work
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 2 of 30
Section 1
Introduction
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 3 of 30
The Core Issue
RC3E ( Reconfigurable Common Cloud Computing Environment )
• System for distribution of FPGA ressources to remote users
• Actual ressources get abstracted as virtual FPGAs
• Controlled by a human administrator, who. . .
– . . . runs selected scripts on the host– . . . manages users by hand– . . . has to schedule ressources– . . . does the programming of vFPGAs and relays the results
Project Aim
Create a web-frontend that can automatize these tasks.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 4 of 30
What is Django?
• Python-based framework for web applications
• Architecture is Model Template View
– Variation of MVC pattern
• Focus on reuseability, rapid prototyping and the DRY-principle1
• Huge amount of out-of-the-box features
1DRY = Don’t repeat yourself
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 5 of 30
Architecture of a Django Project
Project
manage.py
settings.py
urls.py
App1
forms
models
templates
urls
views
App2
. . .
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 6 of 30
Operation Principle of a Django-App
Browser
URL Dispatcher
HTML TemplateRendering Engine
View ModelDatabaseConnector
MySQL
DatabaseBackend
Django Controller
Web Server
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 7 of 30
What is REST?
REpresentational State Transfer
• Architecture principle for distributed web applications
• Allow clients to access a textual representation of a ressource on thehost
• Set of available operations on the ressources are well-defined by asimple interface
Benefit
Easy to parse for machine-to-macine communication.Useful for interaction with the RC3E-server.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 8 of 30
Example for a RESTful API Call
Request:
fredo in ~ > http GET localhost :8000/ rest_api/producers/
Response:
HTTP /1.1 200 OK
Content -Type: application/json
[More http header stuff]
[
{
"id": 1,
"name": "Xilinx"
},
{
"id": 2,
"name": "Altera"
},
{
"id": 3,
"name": "Intel"
}
]
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 9 of 30
Section 2
Data Model
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 10 of 30
Preface
The data model is the backbone of the project.
• Must be represented by a relational DB
• Fine granularity to improve changeability
Important!
FPGAs are assumed to be homogenous.
Note
Multiple development iterations introduced some technical debt.But: It can be purged by refactoring cycles.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 11 of 30
Data Model Details I
fpganode : nodefpga model : fpga modelnode pci : pci addressdevice pci : pci address
pci addresssystem : character[4]bus : character[2]device : character[2]function : character[1]
nodename : character[255]ip address : generic ip addresscomment : text field
fpga modelproducer : producerdesignation : character[255]region type : region typeregion count : integer
regionin fpga : fpgamemory device path : file pathregion type : region typeindex : integer
region typelut count : integerdsp count : integerregister count : integerblockram size : integer
producername : character[255]
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 12 of 30
Data Model Details II
region reservationregion : regionreserved by : reservation
regionin fpga : fpgamemory device path : file pathregion type : region typeindex : integer
reservationby user : django.auth usercreation date : datetimestart date : datetimeend date : datetime
Formerly known as vfpga
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 13 of 30
Data Model Details III
fpganode : nodefpga model : fpga modelnode pci : pci addressdevice pci : pci address
fpga modelproducer : producerdesignation : character[255]region type : region typeregion count : integer
device variablefpga : fpganame : character[255]value : text field
scriptprogrammer : programmerfpga model : fpga modeltemplate : text field
runtime variablename : character[255]value : text field
programmername : character[255]device file path : file path
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 14 of 30
Section 3
Selected Issues
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 15 of 30
Basic Principle of User Interaction I
Boil down all interaction to the following operations:
• List objects of a type
• Show object details
• Create objects
• Delete objects
Note
Allowing to modify existing objects has severe implications!(What happens when you change the region type/ count of an FPGA?)
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 16 of 30
Basic Principle of User Interaction II
Issue
URLs and internal project structure have to consistently represent thisprinciple.
Issue
Look and feel of the HTML pages should be uniform.
Solution
Use Djangos HTML template inheritance.
Note
Even special pages can profit from inheriting layout templates.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 17 of 30
HTML Template Structure
layout general.html
layout create.html
fpgas/create.html
. . .
layout delete.html
fpgas/delete.html
. . .
layout list.html
layout show.html
welcome.html
registration/login.html
registration/logged out.html
css/layout.css
WIP
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 18 of 30
Automating URL-Binding Generation
Issue
Generating all the URL-bindings for the List/Show/Create/Delete pagesby hand is tedious and error-prone.
Solution
Write generator functions for the common case and only deal with thespecial cases by hand.
Note
Generator functions also check if the required templates are present.
(PS: https://www.xkcd.com/917/)
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 19 of 30
Supporting Multiple APIs I
Issue
Need to support a REST and a web API
• Both use the same protocol (HTTP/ HTTPS)
• HTTP request types GET, POST used by web API
• REST will use these as well
How to figure out what to do with an HTTP request?
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 20 of 30
Supporting Multiple APIs II
Idea
Give each API a different URL namespace.
Web-API:GET /web_api/producers/create/
POST /web_api/producers/create/
REST-API:GET /rest_api/producers/
POST /rest_api/producers/
GET /rest_api/producers /1/
DELETE /rest_api/producers /1/
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 21 of 30
Supporting Multiple APIs III
Issue
The REST-framework alters the Django-apps structure.
Solution
Split the project into one app per API.
Next Issue
Both apps have to operate on the exact same model.
Solution
Split off the model into a separate app as well.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 22 of 30
Supporting Multiple APIs IVrc3e manager
. . .
backend
models
rest api
serializers
views
web api
forms
templates
views
import
Same name,Different stuff
!
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 23 of 30
Scheduling of Reservations I
User Input
• Region type
• Region count (consecutive)
• Desired start and end of reservation
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 24 of 30
Scheduling of Reservations II
View generates and issues SQL query from user input.
Database Action
1. Filter for FPGAs with correct region type
2. Filter regions without reservation in desired timeframe
3. Sort remaining regions by index
Result type: {FPGA: [Region]}
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 25 of 30
Scheduling of Reservations III
View then processes the query result.
Pseudocode
foreach fpga:
| foreach region:
| | if region is not consecutive:
| | | reset markers // deal with fragmentation
| |
| | mark region
| |
| | if enough regions marked:
| | | break all // Got enough reservation candidates
|
| reset markers // Reservations across FPGAs are not allowed
reserve candidates // If there are any
// Inform User
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 26 of 30
Scheduling of Reservations IV
Notes
• First fit scheduling does not use ressources equally
• Fragmentation may become an issue in real-life use
• Reservations are not supposed to be deleted (accounting) which mayslow down the DB after many reservation cycles
It works for the scale of RC3E but has optimization potential.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 27 of 30
Section 4
Results and Future Work
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 28 of 30
Summary of Current State
• Data model has been developed
• Technology research is completed
• Model is implemented
• Web-API is implemented
• Currently known issues have been solved
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 29 of 30
Future Work
Immediate tasks:
• Finish transition to new HTML template structure
• Finish implementation of REST-API
Whishlist:
• Subsystem for uploading designs, running the FPGA programmingscripts and operation feedback
• Subsystem for statistics and accounting
• Logging RC3E interactions to DB
Ultimative Goal
Connect the frontend with the live RC3E-system.
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 30 of 30
Section 5
Appendix
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 31 of 30
Further Reading I
RC3E
RC3E: Reconfigurable Accelerators in Data Centres and Their Provisionby Adapted Service Models, Oliver Knodel, Patrick Lehmann, Rainer G.Spallek, 2016https://ieeexplore.ieee.org/abstract/document/7820030/
Rekonfigurierbare Hardwarekomponenten im Kontext vonCloud-Architekturen, Dissertation of Oliver Knodel, 2018Publication pending
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 32 of 30
Further Reading II
REST
Architectural Styles and the Design of Network-based SoftwareArchitectures, Dissertation of Roy Fielding, 2000https://www.ics.uci.edu/~fielding/pubs/dissertation/
fielding_dissertation.pdf
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 33 of 30
Further Reading III
Django
Official Documentationhttps://docs.djangoproject.com/en/2.0/
Django-RESThttp://www.django-rest-framework.org/
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 34 of 30
Last Slide
Thank you for your attention.Questions?
Office: TUD, APB 1095Phone: TUD-38456E-Mail: [email protected] Git: http://github.com/VLSI-EDA/rc3e-manager
PS: We are looking for students to continue working on this project.
Bachelor thesis, anyone? ;)
Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018
Slide 35 of 30