Parallel Computing using Condor on Windows PCs

Peng Wang and Corey ShieldsResearch and Academic Computing DivisionUniversity Information Technology Services

Indiana University

Problem Description

Turn Windows desktop systems in STC labs (around 2000) into a parallel scientific computer

DiscussionParallel applications need coordination through message passingMPI does not handle ephemeral processes wellMultiplexing communication among processesPorts brokered among multiple parallel sessions

What do we have ?Condor NT, vanilla universe

match-making, file transfer, fair sharing, job submission, suspension, preemption, restart, securityTest application – fastDNAml-p

Parallel application, master-worker model, small granularity of work

How we did itSimple Message Brokering Library (SMBL)Process and Port Manager (PPM)A mechanism for users to submit jobs (web portal)

SMBLAn IO multiplexing server in charge of message delivery for each parallel session (serialize communication)SMBL client library implements selected MPI-like callsBoth the server and the client library are based on a TCP socket abstraction library

Process and Port ManagerAssigns port to each of the SMBL server processstart the SMBL server and application processes on demanddirect workers to their servers

The PortalApache basedPHP web interfaceCreates and submits the condor submit files

The Big Picture

The shaded box indicates components hosted on multiple desktop computers

Statistics

Red: total owner Blue: total idle Green: total Condor

Scalability Issues

Needed big serverAdjusted condor_config

MAX_JOBS_RUNNING 1000

SHADOW_SIZE_ESTIMATE 900KB MAX_STARTD_LOG 640KB

Lost workers because of per-process file descriptor limit (1024)

SummaryBuilt a large parallel scientific computing facility using CondorBuilt parallel message passing library to deal with ephemeral resources Built port broker to handle multiple parallel sessionsBuilt web portalIt is open source, visit: http://smbl.sourceforge.net