heavy duty primavera usage

Heavy duty Oracle Primavera usage in

enterprise environments: maximising

ROI

A paper presented at the InSync10 Conference, Melbourne, Australia, 16 ‐17 August 2010.

Anthony Spierings

Strategy and Information Manager, ENERGEX

[email protected]

Heavy duty Oracle Primavera usage in enterprise environments

Page | 2

1. Contents

1. Contents .......................................................................................................................................... 2

2. Table of Figures ............................................................................................................................... 3

3. Executive Summary ......................................................................................................................... 4

4. Purpose ........................................................................................................................................... 5

5. Oracle Primavera @ ENERGEX in 2010 ........................................................................................... 5

5.1. About ENERGEX ...................................................................................................................... 5

5.2. How Oracle Primavera is used ................................................................................................ 5

5.3. Oracle Primavera Configuration ............................................................................................. 8

6. ENERGEX’s journey with Oracle Primavera .................................................................................... 9

7. Statistics about how Oracle Primavera is used ............................................................................... 9

8. Interfaces to other applications ...................................................................................................... 9

9. The Limits of 32‐bit computing ..................................................................................................... 10

9.1. How much memory are you using ........................................................................................ 10

9.2. Issues for ENERGEX due to 32‐bit limitations (desktop client) ............................................. 12

9.3. Options for Five and Ten year PoW ...................................................................................... 13

10. A day in the life of XBNEWAS18 ................................................................................................ 14

10.1. Job Services 32‐bit limits ................................................................................................... 15

10.2. Using Job Services 32‐bit platforms .................................................................................. 15

10.2.1. Get the required data directly from the database ........................................................ 16

10.2.2. Oracle Primavera Enterprise Reporting Database ........................................................ 16

10.2.3. Split the jobs into smaller chunks and recombine ........................................................ 17

10.2.4. Go to a 64‐bit solution .................................................................................................. 18

10.3. Other Issues with Job Services .......................................................................................... 18

11. Database Performance ............................................................................................................. 20

12. Conclusions ............................................................................................................................... 21

13. Appendix 1 – How to Split Job Services .................................................................................... 22


Page | 3

2. TableofFigures

Figure 1 ‐ High level data flows around EPPM in ENERGEX .................................................................... 7

Figure 2 ‐ Desktop Application Out of Memory Warning ..................................................................... 10

Figure 3 ‐ Primavera Memory Usage at project screen (no projects open) ......................................... 11

Figure 4 ‐ Primavera Memory Usage with 100 projects opened (display at activity level). ................. 11

Figure 5 ‐ Primavera EPPM memory usage per project inside ENERGEX ............................................. 12

Figure 6 ‐ Primavera resource pool view when preparing 18‐month PoW .......................................... 13

Figure 7 ‐ XBNEWAS18 CPU Load 23 February 2009 ............................................................................ 14

Figure 8 ‐ XBNEWAS18 free memory 23 February 2009 ....................................................................... 14

Figure 9 ‐ XBNEWAS18 free memory 7 March 2010 ............................................................................. 15

Figure 10 – XBNEWAS18 free memory 1 July 2009 .............................................................................. 15

Figure 11 ‐ Screen capture of Job Services running the same report multiple times ........................... 22

Figure 12 ‐ ENERGEX EPS that Job # 440 is being run against............................................................... 23

Figure 13 Job Services CPU Load from 11 July 2010 to 18 July 2010 .................................................... 23

Figure 14 ‐ Server Free Memory from 11 July 2010 to 18 July 2010 .................................................... 24

Figure 15 ‐ Screen capture of the P6 report list .................................................................................... 25

Figure 16 – Report editor, reports have a header row ......................................................................... 26

Figure 17 ‐ Report editor, reports have header row removed ............................................................. 26


Page | 4

3. ExecutiveSummary

ENERGEX uses Oracle Primavera as integral part of its Enterprise Portfolio Project Management

methodologies. ENERGEX works under EPPM are valued at AUD$8 billion FY2010 to FY2015.

ENERGEX has two rolling time horizons of interest; now to +18 months (Tactical Program of Work)

and now to +5 years (Regulatory Program of Work). An emerging requirement is for a ten year view

(Strategic Program of Work). Presently, Primavera is the core system for preparing time‐series

forecasts for; cash consumption, internal resource consumption, material delivery, and desired

external resource capability.

At any one point in time, there are approximately 1 000 projects under active management in the

Tactical Program of Work, with a further 4 000 projects in the Regulatory Program of Work. All up,

the database contains over 35 000 projects (including baselines), with over 1.4M activities.

Enterprise users make heavy demands on the information systems infrastructure. When using

Oracle Primavera on top Windows 32‐bit Operation Systems users may reach the limits of 32‐bit

computing. By observing the application’s memory consumption, End Users can develop simple

“rules of thumbs” and operational techniques to maximise their ROI in computing resources.

There are a number of options available to heavy duty enterprise users when conducting corporate

reporting. Option one; extract the data directly from the back end database. However, due to the

sophisticated use of logical data we cannot recommend this approach. Option two; use the Oracle

Primavera Enterprise Reporting Database. Option three; split large reporting jobs into smaller

chunks. Then recombine the output files. Option four; upgrade to P6V7 version that is packaged for

64‐bit environments. Make provision in the budget for a full release 64‐bit version.

We also note in passing that the ‘out of the box” settings for the back end database are optimised

for standard installations. ENERGEX strongly recommends that large Oracle Primavera EPPM

installations engage the services of an Oracle database application tuning expert.

The Oracle Primavera EPPM product suite has been integral part in assisting ENERGEX efficiently and

effectively plan, deploy, manage, and deliver a $1B annual Program of Work.


Page | 5

4. Purpose

Enterprise Portfolio Project Management (PPM) is a rapidly evolving methodology for many

organisations. Oracle Primavera provides a suite of information system tools to assist organisations

optimise their program management. EPPM makes heavy demands on the information technology,

particularly in large enterprise environments.

This paper discusses how ENERGEX uses Oracle Primavera EPPM to develop Tactical, Regulatory, and

Strategic Program of Work. It then provides a brief tutorial on 32‐bit computing to assist End Users

understand how Oracle Primavera uses computer memory, particularly how the product response to

their projects in their environment.

5. OraclePrimavera@ENERGEXin2010

5.1. AboutENERGEX

ENERGEX is one of the Australia's largest and fastest‐growing organisations with more than 3500

staff working in a range of roles to supply electricity to a population of around 2.8 million people in

South East Queensland.

A top 100 Australian company with more than $7 billion in total assets, ENERGEX has more than 80

years industry experience and has a strong focus on customer communications, positive

environmental outcomes and community safety.

The company’s South East Queensland distribution area of 25,000 square kilometres includes more

than 50,000km of underground and overhead electricity lines and cables, over 600,000 power poles,

some 43,000 transformers and almost 300,000 street lights.

5.2. HowOraclePrimaveraisused

ENERGEX uses Oracle Primavera in a number of different ways.

First, it is the core application for forecasting across a Program of Works that has a $1B annual

expenditure. Oracle Primavera is used to provide time‐series (temporal) data on expected cash

consumption, internal labour requirements, major material delivery dates, and contractor capability.

There are number of time horizons that interests ENERGEX;

Time Horizon 1. now to eighteen months (Tactical Program of Work)

Time Horizon 2. now to five years (Regulatory Program of Work)

Time Horizon 3. now to ten years (Strategic Program of Work)

The now to eighteen time horizon is used to prepare the rolling Tactical Program of Work. To ensure

that ENERGEX’s “internal production line” is running in an optimal manner. Cash consumption

forecasts, internal resource usage, and synchronising the program with the delivery of major

electrical plant items are front of mind.


Page | 6

The now to five year time horizon (Regulatory Program of Work) is partly driven by the regulatory

environment that ENERGEX operates within. Indeed, much of the core financial modelling for

project related work that ENERGEX prepared for its recent submission to the Australian Energy

Regulator was conducted in the Oracle Primavera environment.

The ENERGEX Executive is now challenging the Program Management Office to explore “over the

horizon”, the now to ten year time frame (Strategic Program of Work).

Oracle Primavera is a link in ENERGEX’s sophisticated end‐to‐end methodology for prudently

managing a growing network. Oracle Primavera accepts upstream data from the electricity planners

(e.g. project scope, estimates, delivery dates) and turns this into forecasting outputs for downstream

service providers (e.g. when are major materials required). This is achieved by using the Primavera

API to; import data from the Network Planner’s database, cross‐match the project to an existing

template project, and then build the project in Primavera1. We believe that our methodologies for

preparing Programs of Work place ENERGEX ahead of the pack. It is a significant step ahead of the

plethora of spreadsheet that most other organisations still resort to.

Second, project managers use Oracle Primavera in a more traditional manner on a subset of projects

within the overall Program of Work. That is, to use Oracle Primavera’s capability to generate Gantt

and Pert Charts to develop the project delivery plan for important projects. These projects, like

building a new electricity substation, use Prince 2 methodologies. There are around 1 000 active

projects with approximately $0.6B in annual spend that are managed using traditional project

management techniques.

An overview of data flows into and out of Oracle Primavera EPPM is shown in Figure 1. Note that

while this schematic has EPPM at the “centre of the universe”, in practice, the EPPM is just one link

in a rather long value chain. Also note that most of the links are not automatic data flows, but a

representation of how data flows between systems.

1 Refer to Spierings’ presentation at Primavera Conference, Perth, 2008 for more information


Page | 7

Oracle Primaveradatabase

Build 18-month POW

Ellipse Estimates

ResourceAvailability

SIFT(Network Needs)

OrganisationalConstraints(e.g. budget, community obligations,

acceptable risk profile, ROI, etc)

Day to day project

management

Current project statusSkill sets

Unconstrained dates

Forecast; resource profile

Qld GovtState Budget

Project completion dates

Constrained dates

Controlled plantDelivery dates(Transformers,

modules, switchgear, relays,

etc)

Balance mix of internal and

external resources

Manufacturers source raw materials

Manufacturers stay or seek work

elsewhere

QLD Govt Infrastructure

plans

HR Human Capital Plans

AER Submission

ENERGEX Funding

Build 5 year POW

Regulatory Outcomes

Ellipse Works Orders

(dates, budget ..)

Future resource capability

Network StandardBuilding Blocks

DO Work

Real Time Network

Equipment Ratings

Economic forecasting

engine

Controlled PlantContracted Deliveries

Figure 1 ‐ High level data flows around EPPM in ENERGEX


Page | 8

5.3. OraclePrimaveraConfiguration

Primavera runs on an Oracle database. Power Users use the Primavera desktop client to connect

directly to the database. Regular users have their Primavera desktop client running on top of Citrix.

Table 1 lists the current hardware configuration.

Primary Data Centre Oracle Database Server. Running three applications: Primavera, plus two other document management systems. myPrimavera Web: Xeon Quad Core Virtualised (one production, one development, one UAT). Also configured as warm backup for job services. Citrix Farm: Four virtual servers for Primavera running on Citrix.

Backup Data Centre Oracle database server. Cold backup, normal duties as development server. Job Services: Xeon Quad Core Virtualised (one production, one development, one UAT). Also configured as warm backup for myPrimavera Web. Runs schedule task for API in command line mode. Citrix Farm: Two virtual servers for Primavera running on Citrix.

Desktop Client P6.1 (on the desktop): 10 P6.1 (via Citrix): 70 users

Primavera database disk size

55GB

Projects in database (including baselines)

35 000

Activities in database 1 400 000

Time span of projects 2007 – 2020

User numbers 87 named, 14 concurrent, 40 web, 53 portfolio

Table 1 ‐ Oracle Primavera configuration within ENERGEX


Page | 9

6. ENERGEX’sjourneywithOraclePrimavera

At some point in the seemingly distant past (03/03/2003 actually), the Primavera install disks were

first placed in the optical drive. Initially there were only a few users, connecting to a back end Oracle

database via the desktop client running on top of Windows XP (in a Standardised Operating

Environment).

Oracle Primavera’s usage inside ENERGEX was modest for the first few years. By early 2005 a

Program Office was established, with ENERGEX making more use of Oracle Primavera as an EPPM

tool to manage a $0.5B Program of Work. As the application’s usage increased, there was a gradual

decline, imperceptible at first, in the database’s performance. In many respects, poor database

performance (or our lack of ability to tune the application) remains one of the Oracle Primavera

ENERGEX custodian’s primary concerns.

The year 2008 was a watershed year for Oracle Primavera within ENERGEX. The Program

Management Office was heavily involved in the organisation’s preparation of an $8B regulatory

submission. Our methodologies were evolving and maturing at a rapid rate. We figured out ways of

using the Primavera API to shuffle extraordinary amounts of project data into and out of the product

suite.

The process was stressful in more ways than one. It was a period of intense creativity. It was also a

period when we discovered significant improvement opportunities in our organisational processes.

While we always envisioned that the process would be iterative, we never fully appreciated how the

wealth of larger and improved data sets would precipitate even more demands on people and

systems.

7. StatisticsabouthowOraclePrimaveraisused

The ENERGEX database contains over 35 000 projects with approximately 1.4M activities. Around

10 000 projects are visible at any one point in time, covering approximately 445 000 activities. The

value of works being modelled is approximately $30B over a time span from two years in the past

(concluding projects) to approximately 20 years into the future. Of this, there are 1 000 projects

under intensive project management. ENERGEX Project Managers are responsible for maintaining

their projects in P6.

Of the 35 000 projects in the database, 25 000 are baselines. Project managers are encouraged to

create baselines. On top of this, ENERGEX runs an API over the Program of Work to establish fixed

baselines at known points in the annual budget cycle. These baselines become fixed reference

points for determining changes to the PoW.

8. Interfacestootherapplications

ENERGEX runs P6.1 (the version before web services became available). Most of the upstream data

for future works is imported into Primavera using an API to connect to an in‐house database. This

allows us the ability to automatically build the outlying years automatically and with little effort.


Page | 10

Data is also imported from the ERP (Mincom Ellipse). Data is imported at the project and activity

level into UDF (meta‐data, actual expenditure, approval status, etc). The Primavera material and

contractor resource pool is also synchronised with the ERP on a weekly basis. Further, we do have

the ability to import actual from the ERP into the Primavera past period actual field.

9. TheLimitsof32‐bitcomputing

It would be reasonable to suggest that most Oracle Primavera EPPM installations that are in service

today are installed on top of a 32‐bit version of a Microsoft operation system (e.g. Windows XP,

Windows Vista, or Windows 7). While there are 64‐bit versions available of Windows XP (2005) and

Vista, it was not until Windows 7 64‐bit was released that organisations have generally started to

consider moving off 32‐bit operating systems.

32‐bit applications have a maximum address 2^32 bytes, which is approximately 4GB of memory.

Under the typical Windows install, each application is given virtual 4GB of address space. However,

the operating system reserves 2GB of this memory for the kernel and only gives 2GB for applications

(the default settings). Most 32‐bit applications written to run on Microsoft 32‐bit operating systems

(e.g. Oracle Primavera) have a maximum memory space of 2GB. Until recently, Oracle Primavera

EPPM2 was only available as a 32‐bit application.

Often, the first warning to the End User that they have reached the limits of 32‐bit computing is a

message like the one shown in Figure 2

Figure 2 ‐ Desktop Application Out of Memory Warning

9.1. Howmuchmemoryareyouusing

Below is a simple test that End Users can run on the desktop client. Open windows task manager

and select the processes tab. Identify the image name “PM.exe”. This is the Primavera application.

On the day this screen capture was taken in the ENERGEX environment, the application had a

memory footprint of 115MB (see Figure 1)

Next, open up a number of projects (at least 100). Note the memory footprint the application now

requires. In the example provided in Figure 4, the memory footprint had risen to 432MB.

2 See Knowledge article ID 910591.1


Page | 11

Figure 3 ‐ Primavera Memory Usage at project screen (no projects open)

Figure 4 ‐ Primavera Memory Usage with 100 projects opened (display at activity level).

As an exercise, using Windows Task Manager, I have plotted opened projects (displayed at the

activity view) verses reported memory consumed Figure 5. In the ENERGEX environment, Primavera

uses approximately 120MB of memory while having zero projects open. On average, each additional

project opened uses around 0.7MB of memory. In this particular test, the Primavera EPPM

application exhausted its available memory when 1 950 projects were opened.


Page | 12

Figure 5 ‐ Primavera EPPM memory usage per project inside ENERGEX

The graph in Figure 5 has a trend line superimposed on the data points to provide indicative average

memory consumption per opened project. We believe that the grouping of data points, first below

then above, the average trend line is nothing more than an artefact of ENERGEX’s Enterprise Project

Structure (EPS).

9.2. IssuesforENERGEXdueto32‐bitlimitations(desktopclient)

ENERGEX has around 600 large capital works projects with expenditure in the 2010/11 fiscal year3

currently being modelled in Oracle Primavera. In addition, there is another 600 sub‐programs

(power pole replacement programs, load break switch replacement programs, etc) modelled in

Oracle Primavera using “bucket” or “volumetric” approaches.

3 ENERGEX PoW Year is the same as the government’s fiscal year, starting 1 July.


Page | 13

The 32‐bit memory limit means that, given the way ENERGEX presently uses Oracle Primavera, we

can only open two years worth of work. This means that we can use the desktop client when

preparing the Program of Work for Time Horizon 1. However, we can no longer fully use the desktop

client when preparing longer terms Programs of Work for Time Horizon 2 (five year) and Time

Horizon 3 (ten year) as shown in Figure 6.

Rolling 18-month PoW

Five Year PoW

This area filtered out to fit within 32-bit memory limit

Projects in this area are hidden from analysis in the desktop client

Rolling 18-month PoW

Five Year PoW

This area filtered out to fit within 32-bit memory limit

Projects in this area are hidden from analysis in the desktop client

Figure 6 ‐ Primavera resource pool view when preparing 18‐month PoW

9.3. OptionsforFiveandTenyearPoW

There are approximately 4 000 projects in the five year time span4. This means that the limits of 32‐

bit computing (2GB application limit under Windows XP) are preventing us from using Oracle

Primavera for five year program management in our preferred manner. This paper will discuss our

workarounds later on.

4 Recall that ENERGEX’s larger projects can span up to four years. Hence, it is not unusual to see a long “tail” of

resources into future years for projects that are active in the current year.


Page | 14

10. AdayinthelifeofXBNEWAS18

For many years, a single server (designated XBNEWAS18) was used to provide Web and Job Service

functionality. This provided cost effective use of computing resources. During the day, the server

was primarily serving Web Clients. During the night, the server was busy running Job Services

activities.

By late 2008 we started to notice that the myPrimavera application suite was becoming more and

more erratic. By early‐2009, we had a string of Monday morning’s in a row where we would arrive

at work to find both Primavera Web and Job Services had stalled. Sometimes, the server itself was

completely locked up and unresponsive. The server’s log file was not exactly full of clues either.

The performance graphs provided the best hints as to what the problem could be. Figure 7 is the

CPU load on Monday, 23 February 2009 (a normal day in the office for XBNEWAS18). The CPU graph

is unremarkable.

Figure 7 ‐ XBNEWAS18 CPU Load 23 February 2009

Figure 8 is a graph of free memory on the server. This offers more clues as to what the problem was.

At one point, free memory was down to 44MB. This was coinciding with Job Service activities that

we had scheduled to run during the day.

Figure 8 ‐ XBNEWAS18 free memory 23 February 2009

By this stage, it was slowly dawning on us that we were up against the memory limitations of 32‐bit

computing. While we were aware of these low memory events and understood that it was Job

Services causing the issue, we could not understand exactly what we could do to prevent this from


Page | 15

reoccurring (see Figure 9 & Figure 10); particularly, if we wanted to keep running critical financial

reports out of the system.

Figure 9 ‐ XBNEWAS18 free memory 7 March 2010

Figure 10 – XBNEWAS18 free memory 1 July 20095

Concluding that Job Services was the culprit, we stole borrowed acquired another virtualized server

and divided myPrimavera’s functionality. One server (XBNEWAS18) would be primarily responsible

for P6 Web access, with the other (XBNEWAS06) primarily responsible for P6 Job Services.

Immediately, the server for web clients became stable and has remained so ever since.

10.1. JobServices32‐bitlimits

Job Services is a 32‐bit application. As such, it has the same technical restrictions as the 32‐bit

desktop client. There is a subtle difference in that Job Services has a smaller memory footprint than

the desktop client and appears to be more memory efficient. (We surmise that this is because Job

Services does not require a G.UI.)

10.2. UsingJobServices32‐bitplatforms

This section discusses some of the issues and workarounds with Job Services.

5 This graph has an error in its title. The graph is titled CPU load, however, it is actually a graph of free

memory.


Page | 16

10.2.1. Gettherequireddatadirectlyfromthedatabase

One solution is to bypass Job Services and to get the required information directly from the

database. There is, however, a small problem; particularly when dealing with time‐series data. The

Primavera application makes clever use of the concept of logical6 data. We define logical data as

information that can be reconstructed logically. This reconstruction is often done on the fly.

Consider the following equation;

321 nnn

If we wanted to store this equation inside the database, we only need to store n1 and n2 inside the

database. When we retrieve these two values, we can logically reconstruct n3 (as n1 + n2) on the fly.

As mentioned before, Primavera makes extensive use of logical data. Some of the time‐series7 data

that we were after simply does not exist inside the database. It is reconstructed by the desktop

client, web client, or Job Services when the project is loaded into memory. It is theoretically possible

to reverse engineer the calculations using the information inside the database, but this was beyond

our technical capability and resources available at the time.

Key Learning’s for ENERGEX

Important that data extractors understand the difference between persistent and non-persistent data.

Only extract persistent data directly from the database. And only then if you must.

10.2.2. OraclePrimaveraEnterpriseReportingDatabase

The obvious solution to the memory limitations is to install the Oracle Primavera Enterprise

Reporting Database (ERD) and extract the data required from the star‐schema. While we were (and

still are) keen to install the ERD, there are some issues.

First issue is the price. Our budget estimate to install the ERD was $500,000 AUD. While this may

appear somewhat extravagant for an ERD, consider the following. There is the ERD licence itself,

new database server (which requires both hardware and licensing), disk space to suit, and new

virtual servers for the ETL processing. Because this would form part of our core financial system,

times everything by three for the; Development, User Acceptance Testing, and Production

environments. On top of this, would be the report writing expenses. These costs are traded off

against an existing investment (DEV, UAT, PROD) that is working satisfactorily.

The next part of the problem is cost justifying this expense. As a general comment, Business

Intelligence solutions are a hard product to sell to senior management. Part of the reason is that

6 Logical data is sometimes called “virtual data” or “non‐persistent” data.

7 We define time‐series data as information that has temporal ordering. For example a monthly cash flow

forecast.


Page | 17

Business Intelligence is a paradigm shift for organisations. They provide answers to questions that

have not even been thought of yet. It is only after a BI solution is installed that organisations look

back with wonder at how they could have possibly survived with antiquated reporting solutions.

Another part of the reason for ENERGEX not proceeding with the ERD solution was a perceived clash

between the Oracle Primavera Enterprise Reporting Database functionality and an existing corporate

project to install an organisational wide Business Intelligence solution. This author failed in his

attempt to show that the Oracle Primavera ERD is a complementary product solution, not a

competitor.

That said, the Oracle Primavera ERD is a valid solution and is worthy of consideration when

developing integrated reporting solutions. It is presently on the internal development roadmap. The

circumstances of the time forced us to find another alternative to installing the Oracle Primavera

ERD.


ERD turns logical data into persistent data for integration into Business Intelligence solutions ERD will remove a lot of stress from the daily working life ERD is immune to 32-bit computing limitations ERD is a complementary product

10.2.3. Splitthejobsintosmallerchunksandrecombine

The Primavera knowledge base offers the following advice for circumstances when the 32‐bit

memory limit is exceeded.

“Workarounds for using Project Management in a non-LAA environment:

Using the PM client, open smaller EPS nodes/download less data”

Primavera Knowledge Article ID 910591.1


Existing Job Services tool suite is a cost effective use of existing functionality Splitting jobs across the Enterprise Project Structure is an effective technique to manage

memory requirements Job Services outputs are available in time-series data Tip: consider use TAB delimiters in the output


Page | 18

10.2.4. Gotoa64‐bitsolution

Note: At this time of writing, while it has been discussed at various conferences, Oracle Primavera

has yet to release a true 64‐bit client.

With the release of P6V7, the Primavera client is now Large Address Aware. When the /3GB switch

is set in the Boot.ini file on Windows XP Professional, the application can now access up to 3GB of

memory. When P6V7 is installed on a 64‐bit operating system, the application can address up to

4GB of memory. Referring back to Figure 5, memory testing in the ENERGEX environment provided

the formula;

This roughly equates to 5000 projects that can be opened simultaneously in the ENERGEX

environment. This would probably allow ENERGEX to simultaneously open all of the projects in the

five‐year Program of Work. However, the ten‐year Program of Work remains out of reach.

If it were available, a true 64‐bit solution running on top of Windows 7 64‐bit Enterprise, maxed out

with 192GB of RAM, would allow ENERGEX to simultaneously open somewhere around 250 000

projects. Such a package would certainly alleviate the current issues.

ENERGEX Position

P6 V7 64-bit packaged version offers immediate relief for memory limitations Eagerly await the release of Oracle Primavera EPPM full 64-bit desktop client

10.3. OtherIssueswithJobServices

As mentioned above, cash flow forecasting is a core function of the ENERGEX Program Management

Office. This requires nightly reports that are reliable and consistent.

As memory management is an issue for large Programs of Work, we recommend that particular care

is exercised to sequence the jobs to occur one after the other, lest two jobs overlap and exhaust

available memory. Remember to also coordinate daily jobs (e.g. cash forecasts) with weekly jobs

(e.g. XER backups).

Be aware that completion times for certain jobs are highly variable. For example, the summarisation

process will skip projects that it ascertains has not had any changes since last summarisation.

Conducting a project schedule (F9) is a flag to Summarisation that it must re‐summarise this project.

If a schedule as occurred across the large sections of the database (for example, we have completely

rebuilt the five year Strategic Program of Work) this can literately add hours to the daily

summarisation process. As there are no interfaces available to flag that Job Services is complete, we

rely on having buffer dead time between when we expect Job Services to be complete in the worst

case and the start time for other process to uplift the data into the ETL for insertion into the data

warehouse.


Page | 19

There are two tricks that we have learnt when dealing with Job Services;

a. Restart job services (from inside the Windows Server’s control panel) whenever a

change has occurred.

b. Regularly check that the outputs have a current date stamp.


Page | 20

11. DatabasePerformance

The performance of the database has been an ongoing concern. Over the last few years, we have

undertaken some analysis of the database. The author is not fully convinced that all of the ENERGEX

instigated changes have been for the best. So, the comments below are rather general in nature.

Our suggestions to improve database performance are as follows;

a. Strongly recommend that large Primavera installations engage a database application

tuning consultant as a starting point.

b. In the ENERGEX environment, the database appears I/O bound. Increasing cache sizes and

disk speed improves performance. Pay particular attention to how the database server

conducts I/O operations and the UNDO table during summarisation.

c. Check that network issues are not constraining performance. (This is not an issue with

ENERGEX and the geographic locality is small and the Primavera End Users are located on

fat pipes.)

d. Including User Defined Fields and user created Project Codes in the layouts slows down

initial opening of the application and projects. Consider providing a layout to uses that

sticks to the default Primavera fields to improve response times when opening the

product.

e. The administrator guide recommends turning off the loading of resources when opening to

improve performance. This is good advice when there are large resource pools.

f. The ENERGEX methodology of deleting and rebuilding (recall how the outer years of the

Program of Work are built automatically) knocks around the various indexes. Talk to the

DBA about the best methods to manage the internal indexes.


Page | 21

12. Conclusions

ENERGEX uses Oracle Primavera as an integral part of its EPPM methodologies. ENERGEX eagerly

awaits the release of a true 64‐bit Oracle Primavera desktop client (and job services).

AboutAnthonySpierings

Anthony has over 28 years experience in the energy industry. Anthony’s current work interests are

in the preparation and management of large, multi‐billion, multi‐year, Programs of Work.

Anthony is a qualified electrician. He also holds; an Associate Diploma in Electrical Engineering

(Power), Bachelor of Business (Economics, Banking and Finance), and a Masters in Project

Management (twice making the Dean's List) all from the Queensland University of Technology.

Anthony is presently a part‐time PhD Student at the University of the Sunshine Coast, researching

why End Users develop Feral Information Systems (like spreadsheets) in preference to the

sanctioned Enterprise System.


Page | 22

13. Appendix1–HowtoSplitJobServices

This section describes a technique to split large jobs to fit within the memory limits of 32‐bit.

It is relatively simple, if tedious, to replicate the reports so that each report only runs against subsets

of the Enterprise Project Structure. Figure 11 is a screen capture of this technique. Job # 440, 441,

and 442 run the same report against various parts of the Enterprise Project Structure.

Figure 11 ‐ Screen capture of Job Services running the same report multiple times

Figure 12 is a screen capture of part of the ENERGEX EPS that Job 440 et al is being run against.


Page | 23

Figure 12 ‐ ENERGEX EPS that Job # 440 is being run against

The impact of splitting up the jobs into smaller chunks is a reduction in CPU and memory usage.

Figure 13 shows the CPU load on the Job Server XBNEWAS06. This snapshot was taken during the

week from Sunday, 11 July 2010 to Sunday, 18 July 2010. The higher CPU usage is attributable to

improved database I/O performance (a separate exercise that is discussed later). Improving

database performance meant that CPU performance is now a constraint on performance.

Figure 13 Job Services CPU Load from 11 July 2010 to 18 July 2010

What is perhaps more instructive is the graph of available memory on the server8 as shown in Figure

14. This virtual server has 4GB of memory and 4GB page file. The assumption is that Primavera’s

memory usage is now down to 0.79G of memory (3.45G – 2.66G).

8 Recall that XBNEWAS18 has been replaced by XBNEWAS06 for Job Services.


Page | 24

Figure 14 ‐ Server Free Memory from 11 July 2010 to 18 July 2010

The problem with this solution is that output data is split across a number of files. Each file has its

own header, which is an annoyance when trying to merge the data. A simple workaround is to have

two sets of reports. The first report in the series has the column headers included. The second and

subsequent reports are identical, except the headers have been removed. See Figure 15 for an

example.


Page | 25

Figure 15 ‐ Screen capture of the P6 report list

Removing the report headers is a simple exercise. Copy the original report. Open the report up with

the editor. Figure 16 shows the report with a header row included (above the page header area). To

delete, click on each header and delete in turn. The end result should look like the screen capture in

Figure 17


Page | 26

Figure 16 – Report editor, reports have a header row

Figure 17 ‐ Report editor, reports have header row removed

Finally, write a small script to join the output files together. For example;

copy/b /y %INPATH%%INFILE1% + %INPATH%%INFILE2% + %INPATH%%INFILE3%

+ %INPATH%%INFILE4% %OUTPATH%%OUTFILE1%


Page | 27

Yes, it is a kludge. But it works surprisingly well. ENERGEX then imports this file into the corporate

ETL for further analysis. The only remaining issue is whether the data in the file requires a further

“group by” summarisation. If the text file is based on a unique ID (e.g. Project ID) in the Primavera

database, only one row will appear in the output. If the output is based on a non‐unique ID (e.g.

resource usage), then this information may appear multiple times in the report and require further

processing before it can be used.

heavy duty primavera usage

Documents