is your corporate data warehouse shrinking to a data mart

8/8/2019 Is Your Corporate Data Warehouse Shrinking to a Data Mart

1/4

Is Your Corporate Data Warehouse Shrinking

To a Data Mart

Since the past decade data warehouses have been gaining enormous ground in thebusiness intelligence (BI) domain. A corporate data warehouse was on every

organizations priority list. Organizations which were wary of making huge

investments but at the same time wanted to be in the race opted for data marts. Forbusiness leaders where innovation was the buzz word, a corporate data warehouse

and then consequently data mining (using complex algorithms and predicted trends

and the why of the trends) seemed to the long term strategy (long term in information

technology means a period of 3-5 years).

Projects were embarked on with great vigor and significant investments were

ploughed in. A giant server that would store terabytes or at least gigabytes of data was

brought installed. And after battling with issues like data quality, tools to be used forloading and analyzing data, granularity of data to be stored, years to data to be stored,

performance tuning for faster queries and so on and so forth, the corporate datawarehouse finally took shape. It was 15-18 months of persistent effort which had paid

off. Business users and analysts embarked on the journey of discovering the hidden

trends which could save their organization the much so important dollars and also

contribute towards the return on investment of the corporate data warehouse. Andindeed they did. Every issue of the organization newsletter had articles of trends

revealed and savings accrued.

Companies began to rely more and more on these BI systems. Critical business

decisions were based on the current and historical data available in the datawarehouse. Sophisticated OLAP tools which facilitate multidimensional analysiswere used. Business trends are identified using data mining tools and applying

complex business models.

Companies were battling each other out in terms of storing vast data which was being

analyzed day in and day out to make some sense out of it. Standard reports and views

from the sophisticated OLAP tools were generated on a regularly intervals basis for

analysis purposes.

As businesses grow from local to global, the complexities and parameters involved in

decision making and analysis became more complex. The corporate data warehouseserved as the perfect analysis tool.

But with businesses expanding in various business sectors and various locations itbecame increasing difficult for the corporate data warehouses to keep pace with the

humungous addition of data.

TCS Confidential


2/4

The corporate data warehouse which at one point of time housed the entire

organizations business data and provided everything from a holistic view of the

business to low level granular data suddenly ceased to be so. The organization foundthe data from a number of new locations and new business sectors missing in the

corporate data warehouse. Critical decision making and analysis work like product

introduction strategy, pricing strategy, marketing strategy and product enhancementstrategy for products in the new business sectors was all being done manually and the

sophisticated analysis tools that the data warehouse boasted of could not be employed

for help.

The organization which once so proudly stated that the corporate data warehouse was

a state-of-the-art system used for BI now began to wonder whether to call the data

warehouse a data warehouse or a data mart (data mart is a scaled-down version ofa data warehouse that is tailored to contain only information likely to be used by a

specific target group).

Critical business decisions which at one point to time where taken based on theinformation provided by the data warehouse and which were made effective globally

suddenly were being looked upon with doubt. Business analysts who took pride inrevealing and analyzing business trends found it difficult to justify their results and

providing global solutions.

Since the integration of data from new locations and business sectors into the datawarehouse needs proper planning and design and requires time, the organizations

found themselves dealing with a situation where the new locations and business

sectors had started developing their own set of home grown analytical systems. Theorganization thus ended up with a structure where each location and business sector

had their own system which made the integration process more complex. This further

delayed the process of making the data warehouse a true corporate data warehouse.The people maintaining the corporate data warehouse were unhappy with the fact that

inspite of adhering to the best data warehousing standards and norms like

organization wide conformed dimensions, etc. the data warehouse was being devoidof the status it deserved.

More and more companies today are facing the above situation. Most companies need

to look at global product sales, global product warranty, global product pricingstrategies, global product promotion strategy and global product trends. And there is

no one place where all the required information is available. The obvious choice is to

work on the available data set and extrapolate the results to the locations for whichdata is not available. Though this might be acceptable in a sizable number of cases,

extrapolation of data for critical decision making is a risky proposition and should be

avoided. It entirely defeats the purpose of having a corporate data warehouse.Models which use the data warehouse data for sales forecasting and resource

forecasting purposes need fairly accurate history data. Results of one country or

region cannot be extrapolated to other countries and if so done a significant degree of

fluctuation would be expected. The more the skew ness of data the more it renders

TCS Confidential


3/4

exercises like forecasting, trending, data modeling, etc ineffective. That is when the

big question of the return on investment on the corporate data warehouse crops up.

An amicable solution of such types of problems calls for close interaction between

the business users and the information technology team.

Rome was not built in a day, and so cant be corporate data warehouses.

Amidst the rapid changes taking place, the needs to be a team (a steering committee)who looks at the various jobs at hand and prioritizes them based on business needs

and requirements.

The project champion has a major role to play. But it is not only the project championwho can make them difference. Gone are the days when the business users and

analysts waited patiently for the information technology people to make the system

available to them. And gone are the days when the information technology people got

a ready-made business logic document which they just had to code. Today thebusiness users are expected to be techno-commercial. The information technology

people are no longer dumb coders but are expected to be business knowledge savvyand add value to the organization processes. Business analysts and the information

technology people have to work hand-in-hand and understand each other lingo to

churn out effective and optimized processes.

The information technology people should understand the critical business parameters

which are used for making decisions. The business users on the other hand have to

provide continual feedback on the gaps between the expected and actual. Prioritiesare to be set up in close coordination in such a way that each step towards integration

should be independently beneficial in some respect. Each step however small should

fit in the entire picture and be capable of providing the organization some benefit inits own way. This will add value incrementally to the integration work being done

creating a win-win situation.

The project charter is an important and wise way to start. All the intended activities

should be listed and the scope of each activity needs to be detailed out. The

requirement study should comprise brainstorming sessions to identify all possible

areas and bottlenecks that need due consideration. Business analysts and theinformation technology people should both participate in these sessions. Scope creep

is something the project champion should watch out for. The brain storming session

can prove to be a forum for throwing up a plethora of ideas. These thoughts andpossibilities need to be looked at carefully and the unnecessary ones should be

weeded out.

The next stage, feasibility analysis, then questions each of these user requirements,

looks at the requirements from the technical standpoint. The project champion could

decide the break the requirements into multiple phases or milestones if necessary.

What follows the feasibility analysis is the software development lifecycle which

TCS Confidential


4/4

comprises Design, Development, Integration Testing, Performance Testing, User

Acceptance Testing, Production Roll out and Maintenance and Enhancements. Its all

easier said than done and the whole team has to ensure than each requirement isthoroughly reviewed at all stages of avoid rework and effort overrun at later stages.

The review activity and defect logging though seem to be tedious activities; actually

add a lot of value in making the design scalable and robust.

A slow but sure step by step approach is a must for transforming the shrinking

corporate data warehouse back to the corporate data warehouse.

TCS Confidential

is your corporate data warehouse shrinking to a data mart

Documents