© 2005 altera corporation © 2006 altera corporation batch computing at altera condor, quill and...
TRANSCRIPT
© 2005 Altera Corporation© 2006 Altera Corporation
Batch Computing at Altera
Condor, Quill and The Enterprise
2© 2006 Altera Corporation
About AlteraAbout Altera “The Programmable
Solutions Company” Pioneer of SOPC
technology Founded in 1983 $1.02 billion in
2004 sales 2,300 employees 14,000+ worldwide
customers
3© 2006 Altera Corporation
Programmable LogicDevices (PLDs)
Intellectual Property (IP) Development Software
About Programmable SolutionsAbout Programmable Solutions
4© 2006 Altera Corporation
About MeAbout Me
Senior Software Engineer at the Toronto Technology Center
B.A.Sc. in Engineering Science from the University of Toronto
Joined Altera in 2001 Focus on distributed computing
5© 2006 Altera Corporation
Where It All BeganWhere It All Began
Developed in Toronto Centralized scheduling system Multiple queues Priority/FIFO execution No limit on resource claims Engineer-designed, custom API
6© 2006 Altera Corporation
Change Is Good, Right?Change Is Good, Right?
Multi-OS support Redundancy and fault tolerance Easy expansion beyond Toronto Easy-to-use API New features Improve matchmaking Capacity planning Really Important!
7© 2006 Altera Corporation
Pain Free MigrationPain Free Migration
TTCPOOL
USERTOOLS
CONDORPOOL
META SCHEDULER
PriorityEngine
CO
ND
OR
TT
C
SO
AP
DB
8© 2006 Altera Corporation
Time Stands StillTime Stands Still
Nice-style priorities [1:N] Use priority factor to ensure PN negotiates
before PN+1, PN+2, etc. RUP(PN) = 0.5
EUP(PN)/EUP(PN+1) = ½
Freeze RUP values in time PRIORITY_HALFLIFE = 100000000000000000000
Let jobs at PN get all VMs in the system NEGOTIATOR_IGNORE_USER_PRIORITIES = True
9© 2006 Altera Corporation
Translation ServicesTranslation Services
<cluster><id>1</id><priority>2</priority><os>windows</os><group>fitter</group><job>
<id>1</id>...
</job><job>
<id>2</id>...
</job>...
</cluster>
METASCHEDULER
+AlteraClusterID = 1+AlteraGroup = fitterrequirements = OpSys
= ...+AccountingGroup = P1AlteraTargetOs =
windows...
+AlteraJobID =1...queue
+AlteraJobID = 2...
10© 2006 Altera Corporation
SQL! SQL! Everywhere!SQL! SQL! Everywhere!
POSTGRESQLDBMS
METASCHEDULER
CONDORQUILL
CONDORCOLLECTOR
USAGEHISTORY
STATUSINFO
SYSTEMAUDITS
11© 2006 Altera Corporation
Better Planning Through PicturesBetter Planning Through Pictures
12© 2006 Altera Corporation
From Here, Where?From Here, Where?
Roll out across the enterprise Scaling with multiple schedds Quill++ DBMS for configuration management (with
R. Nordlund & J. Stowe from The Hartford)