pushing the rule engine to its limits with drools planner (parisjug 2010-11-09)

drools

Geoffrey De Smet

Pushing the rule engine
to its limits
with Drools Planner

Agenda

Drools Platform overview

Use casesBin packagingWhat is NP complete?

Employee shift rosteringHard and soft constraints

Patient admission scheduleHow many possible solutions?

AlgorithmsMeta-heuristics

Benchmarking

Drools Platform

Overview

Business Logic Integration

BusinessLogicIntegrationPlatform

Rule
engine

Workflow

Complex event
processing
(CEP)

Business Rule
Management System(BRMS)

Automated
planning

(jBPM 5)

Use cases

What are planning problems?

New office furniture... 1 car

Half hour later...

Wasted space

Hard constraint implementation

// a nurse can only work one shift per dayrule "oneShiftPerDay" when $left : EmployeeAssignment( $employee : employee, $shiftDate : shiftDate, $leftId : id ); $right : EmployeeAssignment( employee == $employee, shiftDate == $shiftDate, id > $leftId); then // Lower the hard score with a weight ...end

Soft constraint implementation

rule "dayOffRequest" when $dayOffRequest : DayOffRequest( $employee : employee, $shiftDate : shiftDate, $weight : weight ); $employeeAssignment : EmployeeAssignment( employee == $employee, shiftDate == $shiftDate ); then // Lower the soft score with the weight $weight ...end

Patient admission schedule

Hard constraintsNo 2 patients in same bed in same nightRoom gender limitationDepartment minimum or maximum agePatient requires specific room equipment(s)Soft constraintsPatient prefers maximum room sizeDepartment specializationRoom specializationPatient prefers specific room equipment(s)

Needle in a haystack

How many possible solutions?310 bedsin 105 roomsin 4 departments84 nights2750 patients (admissions)Numbers from a real dataset


How many possible solutions?310 bedsin 105 roomsin 4 departments84 nights2750 patients (admissions)> works of art in the Louvre?35000 works of art

Source: wikipedia


How many possible solutions?310 bedsin 105 roomsin 4 departments84 nights2750 patients (admissions)> humans?7000000000 humans

Source: NASA (wikipedia)


How many possible solutions?310 bedsin 105 roomsin 4 departments84 nights2750 patients (admissions)> minimum atoms in the observable universe?10^80

Source: NASA and ESA (wikipedia)


How many possible solutions?310 bedsin 105 roomsin 4 departments84 nights2750 patients (admissions)> atoms in the universe
if every atom is a universe
of atoms?(10^80)^80 = 10^6400

Source: NASA and ESA (wikipedia)


How many possible solutions?310 bedsin 105 roomsin 4 departments84 nights2750 patients (admissions)A little over 10^6851

Do the math

1 patient310 beds310 ways to schedule 1 patient2 patients310 * 310 = 961003 patients310 * 310 * 310 = 297910002750 patients310 * 310 * ... * 310310^2750= a little over 10^6851

A little over 10^6851

17565400009613647714189309909847019528176031676612802408947467896773536824694320253771058944295328388965847321421820520338269384218763243353186553203291774156243401969717704429199723195574019731357439221586010900901318775661618988149185744289903563474940721372704649669447515706252843900344472960295273324597346780837674886972275338744554737771986776337765496773882810395955542351928331414378439283405132892358744224215499792287586474426914111457027335258269167103194692790665217766902628732489519488236496931246157419949856522543510386164584826962224171420152255658502323853504643497053252922727924406406179159327025274968691052425818270121710670552641837503435108794481961062422002729288737980404185875233912428178009558605680683578864680145557998942327371019832139824665975180911386722774004539981342785523851686636374432671601485496423110930175953308580416767966832068095365964656939583089443708945844323888227816296382464122209980716936990539499472027590738400791514217875461942733015467480308665074008529461146577114465977072581447925882122906827160570007228017056499674188148507908716786164922536467490587163626945689452927061532150637454615233641645612791027410608416476380642469085143980464067453971429400369608313929289399595696359958354101721624055729520838609453039985592726289376243856941429063767903919977138724432513602703448146045970566585076809576476984036923221553270878279574239866657156729051295085970100212856025787345034666683590797377984104207041334053480226788236770435009014979334517769826461063281178894554527012859966520606253098788699367180806367012372895824963357992764969799123361044456461687410981522493093316910464937089299655880447014074876397812210684054337706530175130912335567383560436635091489083375525519539844805718811296858076507685112192499405286337668107046095023999871224655107877174220679360212410573001491104381221662138764756898834588381340410892158544837200229008533930816794663631470201197595487045022087615873490295940409113638894083753062801416644858


70757942487218045035508810158461046502812782292385633846174494690914238485407798379765738528402484632449685642401410897637632172694954465509230907381501728706686840208173164426348468614134650930610728341876292346711310677332648553951422991989751468506508069513397904821612697383659788320905691994864296149528670873271380180066507702490525594196383327289726362280241278856579591895744202499646581373849829964867870738964842426372580420928473929652466453066089306581572745139056256181505240205578741292314133858985615181677968313683876880079649763914095651402272047776108451445066888366968448972791816669030945790810396895725248399188822034667566483522327685006195080178525107491255245054238976705620547582329759857450557583834141364747838395082871684183199560673441708538602779816347949276957201268066627372831370768073558934679410276824283049183299518869516908654179971718550810202675628457097617280232889096038128616543128200089047813223519902714196694639844273986565523012106816309974964021700560537928432528631741787455155275823033005847637107210747721372066369346754152090839849611389908167353907349234368276522287410730637555342957452428266968002573227849933691449063418286501311036314048960528249465982665132492491072491788618403253474529440348798670244615185355092357283764936387607076234181916670755269421546530337284689838773122323053171794271444358533638806848969871884168582847613016398013006633016140503743175611255484273419291435502775849577615159921009571496639402549077872124227731739936370100132762333353473908080215719004335487157010620020523763648856692728699459471603410776592535816520745995861336577877431293776796124264697049418795186010546056975226424227455446011031581123438684685838655333776830823720356749120051129394691394743488410658613543537795455760655157849602214428915084056181314465895075924498263846040474495622654552157933867572542745838370889391243702358459244386561081479905545570084491443709439642235090455604548030317754849613813010298858282615659336373785985294


58667337266411706925088329158776619790296442059252886388879455197093987039774900087293989610103172000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000


000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

The search space is big!Compare with WWW size 22020000000 pagesEach possible solution2750 patients scheduled into 310 bedsStill need to calculate the score! => Drools Expert

Algorithms

Operational research is fun.

Brute force?
Throw hardware at it?

Calculate 10^9 scores per msImpossible today!31579200000 ms in 1 year< 10^11 ms in 1 year10^9 * 10^11 scores per year= 10^20 scores per yearHow many years? 10^6851 / 10^20= 10^6831 yearsCPU 1000 times fasterIt becomes 10^6828 years

Smarter brute force?

Eliminate subtreesBranch and bound

Still too many for loops

Still takes forever

for (bedOfPatient1 : bedList) { patient1.setBed(bedOfPatient1);

for (bedOfPatient2 : bedList) { patient2.setBed(bedOfPatient2);

if (patient1.shareNightWith(patient2) && bedOfPatient1.equals(bedOfPatient2)) { continue; // bug: best solution might break a hard constraint } for (bedOfPatient3 : bedList) { ...

2 patients in the same bed

1 patient0 of 310 (no chance)2 patients310 of 96100= 1 of 3103 patients620 of 29791000= 1 of 48 0502750 patients310*2750*2749/2 of 310^2750< 1 of 310^2740

Imperfect algorithms
(mimic a human)

DeterministicFirst in, first assigned, never changed

Easy to implementDrools Planner score support

Fixed time (for example 18 seconds)

Meta-heuristicMove things aroundStart from result of deterministic algorithm

Drools Planner implementations

More time = better score

N Queens: use case

Place n queens on a n-sized chess board

No 2 queens can attack each otherScore -1 for every 2 queens that can attack each other

Score = -2

Score = 0

Move things around

Move = from solution A to solution BChange the row of 1 queen

Give 2 queens each others rows

...

Score = -6

Score = -4

Thank you statefull rule engine!

All moves from one solution

Number of moves < number of solutionsN queensn*n < n^n

4 queens16 < 256

8 queens64 < 16777216

64 queens4096 < 10^116

Local search 1/2

Local search
2/2

Search pathNot a tree

Local optima

1) Deterministic StartingSolutionInitializer

2) Simple local search

3) Stuck in local optimum!

Source: Wikipedia

Score function

Local search++

Tabu SearchSolution tabu (high tabu size)Been there, no need to go there again

Move tabu (low tabu size)Done that recently, no need to do that again

Property tabu (low tabu size)Changed that recently,
no need to change that again

Simulated annealing

Great deluge, late acceptance,

Hyper heuristics

Benchmarker

Measure, don't guess.

Benchmarker utility

CPU power VS algorithms

more CPU power

better
algorithm

deterministic algorithm

Free speed upgrades
from the rule engine

Differential update (AKA true modify)Drools 5.0: update = retract (remove) + assert (insert)

Drools 5.1: real update (released in Q3 2010)Uses less memory and reduces garbage collector stress

Improves performance

Update is mostly used in statefull environments

Statefull memory drools 5.0with Drools Planner 5.1

Statefull memory drools 5.1
with Drools Planner 5.1

Summary

Summary

Drools Planner solves planning problems

Adding constraints is easy and scalable

Switching/combining algorithms is easy

Q & A

Questions?Useful linksWebsitehttp://www.jboss.org/drools/Reference manualhttp://www.jboss.org/drools/documentation.htmlBloghttp://blog.athico.com/Mailing lists (forum interface through nabble.com)http://www.jboss.org/drools/lists.html

This is the title
Second line of the title

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelEighth Outline LevelNinth Outline Level

This is the chapter title

This is the subtitle

pushing the rule engine to its limits with drools planner (parisjug 2010-11-09)

Technology