framework modeling guidelines

Cognos 8 – Best Practices Transformer

Framework Manager Modeling Guidelines

Saravanan Vajjiravel

24-Sept-2011

COGNOS P r a c t i c e Learn. Adapt. Belive. Succeed with Proven Solutions

COGNOS P r a c t i c e

Learn. Adapt. Belive. Succeed with Proven Solutions

REVISION HISTORY ................................................................................................................................. 3

SUMMARY: ................................................................................................................................................. 4

AUDIENCE: ................................................................................................................................................. 4

INPUTS: ........................................................................................................................................................ 4

DELIVERABLES: ....................................................................................................................................... 4

STANDARDIZE USER INTERACTIONS AND REUSE CALCULATIONS ....................................................... 6 LEVERAGE EXISTING MODELS ............................................................................................................ 6 ENFORCE SOURCE CONTROL ............................................................................................................... 6 EMPLOY A STANDARD NAMING CONVENTION .................................................................................... 6 USE AN APPROPRIATE NUMBER OF MODELS ....................................................................................... 7 DATA SOURCE CONNECTIONS ............................................................................................................. 7 EFFICIENT PROCESSING ....................................................................................................................... 8 LIMITED FUNCTION SET ...................................................................................................................... 8 LINKING AND SEGMENTING ................................................................................................................. 8 VALIDATE THE MODEL.......................................................................................................................15 MODEL MODIFICATIONS ....................................................................................................................15 SYNCHRONIZE THE PROJECT ..............................................................................................................16 TASK AUTOMATION ...........................................................................................................................16 MODEL REPORT .................................................................................................................................16

DESIGN TASKS: ........................................................................................................................................17

BUILD TASKS: ...........................................................................................................................................19

CHECKLIST: ..............................................................................................................................................20

ADDITIONAL REFERENCE DOCUMENTS: .......................................................................................20



Revision History

Release Date By Description

1.0 01/06/2011 Saravanan Vajjiravel Base Version



Summary: This document is intended to provide guidelines and processes that should be used when

implementing Cognos 8 models. Framework Manager (FM) is a desktop tool used to model the metadata for all ad-hoc and standard SQL reporting in Cognos8. Framework Manager is also

used to create IQD files that feed PowerPlay Transformer. The information within this document

applies to versions Cognos 8.x.x, and should be used in conjunction with the standard Cognos documentation provided with Framework Manager.

Be aware that the metadata will have a significant influence on the queries that are generated. To ensure that predictable queries and results are returned, it is necessary to correctly model the

metadata.

Audience:

Cognos Developer (Framework Manager Modeler), Cognos Center of Excellence (Cognos Application Architect)

Inputs:

Approved Project

Detailed Report Specifications Document (to include Service Level/Business Requirement

expectations) Data Model

Naming Standards as Approved by Data Governance

Source to Target Map

Deliverables:

Framework Manager Model

Store deliverables within corporate source control or at a minimum on a shared network

drive that is part of a formal back up process Naming conventions for deliverables

Tools: Cognos 8 installation on development server

Framework Manager installed on developer workstation (Please note: The version must

match the version installed on the Cognos 8 Server.)

Database connectivity software (Oracle Client, SQL Client, etc.) for all required data

sources Access to the development gateway server (your network admin must make provisions to

have you set up within the Corporate LDAP and the Cognos enterprise environment.)



Overall Considerations: Framework Manager is the foundation to a successful Cognos 8 application. Fully establish user

requirements prior to building the Framework Manager model. Listed below are several items that should be taken into consideration before modeling efforts commence:

Will this be a high availability system that requires user access on 7x24 basis? How will

database updates be handled - perhaps synonyms or a database alias can be used to ensure

that source data is always available.

What is the expectation for query performance?

Note: This is not the same as report performance and should have been established as part

of the project approval process.

Will this application be integrated with another application?

How will data security be controlled?

Examples: Row level security, database signons, objects access through the interface.

Will the project support authoring across all of the studios? If the project requires delivery of

ad-hoc functionality, then extra effort must be made to ensure that users are presented

information in a meaningful way, and appropriate precautions have been taken to prevent

run-away, poor performing queries. Consideration may be given to the following options:

o Incorporate a prompt against an indexed value in the query subject. The idea is to

incorporate a filtering mechanism that prevents the ad-hoc user from performing

queries that will perform full table scans or transferring large amounts of extraneous

data from the data source to the report server for processing. Several of these could

be created (recommend one for every key) and listed as separate Query Subjects.

Each of the Query Subject contains all of the columns required for ad-hoc processing

and the only difference being the filter applied.

o Set up a new user connection to the database and put governance on the data

source that limits the returned result sets. Governors could also be placed on the

Cognos8 side to limit returned output.

o If summary data is required, create summarized material views to support common

access paths.

Does an FM model exist that contains the desired metadata?

What are the data sources? Are the data sources using conformed dimensions?



Recommended Practices:

The Design and Build Tasks outline the specific steps to develop a model; however, execution of these steps should be completed using the principals below:

Standardize User Interactions and Reuse Calculations

As early as possible in the process, standardize naming conventions, identify filters, and determine query items that will be commonly used as prompts. Calculations that are used

across multiple report deliverables should be modeled within Framework Manager.

Caution: Overuse of this technique could result in excessively long report specifications

and/or poor report performance.

Recommendation: Work closely with the appropriate DBA to ensure that the data sources

are optimized for reporting with necessary columns to be indexed.

Design Approach

Determine the modeling approach: relational versus dimensional.

Recommended: Sketch out the model on paper before entering the model into Framework

Manager and identify all ambiguous joins and data blind spots. (A blind spot is where a join

is ignored based on the selected query data.)

Reminder: This is not an entity relationship model (ERM): The goal is to provide effective queries, presented in a business-centric format.

Leverage Existing Models

A high return on investment can be gained for FM work if your firm capitalizes on previous modeling efforts. One of the best ways to accomplish this is by developing a Common

Dimension Model that incorporates calculations and filters that are common across the firm.

Developers should check with the Cognos Competency Center to validate whether or not the models already exist for their functional areas.

Enforce Source Control

When starting project make sure appropriate precautions have been taken to protect the work. All files located within the model’’ project folders should be source controlled and

checked in at least once a day.

Employ a Standard Naming Convention

Prior to creating a data source within Framework Manager, determine an appropriate naming convention. The model source should be named in a manner that clearly defines the

application and packages should be named with the application and business functional area to be deployed. Once project approval has been granted, the Cognos Center of Excellence

will provide a prefix that should be used for all model and package source.



Use an Appropriate Number of Models

As a rule, there will be one model created for a single application. However, the following

factors should be considered when determining whether to create a new model:

o The redundancy of information when compared to currently developed model.

o The number of tables and attributes.

o Consistency in business logic and presentation.

o Size of currently deployed models.

Data Source Connections

Once the appropriate name has been determined, submit a request to the Cognos Center of Excellence to create the data source connection within Cognos Connection. If the data source

information does not exist within Cognos Connection, the metadata will not be available to

the Framework Manager modeler.

Below are options to ensure that the data source is recognizable regardless of the deployed environment:

o Create a generic name such as DB2 that could be used regardless of the

environment. The client connection is configured to the appropriate environment

such as DB2Test, DB2QA, DB2Prod.

o Create a parameter map for the catalog and schema name. This macro will read

defaultName session parameter and will substitute the appropriate variables at run

time. Choosing this option gives you the greatest flexibility for deployment.

Syntax for the Data Source Catalog parameter map:

#$Datasource_Catalog {$account.defaultName}#

Syntax for the Data Source Schema parameter map:

#$Datasource_Schema {$account.defaultName}#

It is also possible to specify a connection string to an XML data source to include a

parameterized XML. However, using parameterized XML introduces the following restrictions:

o If the URL component is a prompt, it cannot contain other data.

o Data cannot be imported through Framework Manager

o Query validation does not work through Report Studio



Efficient Processing

Determine if all data processing can be accomplished at the database level. Abstraction of

data processing to the report application server level typically results in slower processing and higher resource consumption on the report server(s). A decision to perform local

processing should be based on the following:

o A clearly identifiable need for specific processing not available from the database

such as cross database joins and unsupported SQL99 functions.

o Anticipated data volume in the production environment.

Limited Function Set

Limit the function list in accordance with the native source.

Linking and Segmenting

Recommended: Linking and segmenting are recommended for large models. These techniques are also valuable when multiple FM developers are required.

o Links are created to help organize work across large projects, to maintain

consistency, and to reuse information.

For example, the project named Inventory contains the folder named Products. You can create a link from the Sales Products to Inventory Products. If any changes or

additions are made to the Inventory Products folder, you will see them in the Sales

Products folder.

A linked project is shared by other projects. It should not be created in a sub-directory within the master project directory.

o A segment is a project within a main project. A segment is owned by its main project.

A project segment is a complete project and changes to that project impact all

projects to which it is linked. If you want to open a segment as a separate project, it must be structured as a complete project. There must be a physical layer in each

segment that contains a subset of the data source query subjects on which they are

based. These data source query subjects provide access to the data and metadata and must be included in the appropriate segments.

Note: When changing the project structure, do not open the segments as individual

projects. Instead, check the main project and make changes from within it.

Tips:

Do not change the physical layer in a segment. Any change will be reflected in

the linked parent model and will impact all model segments that share data source query subjects. Changes may not be apparent outside the model in which

they are made until the model is closed and reopened.

Before a project is segmented, ensure that the folder and namespace are named

correctly. You cannot rename the folder or namespace after it has been

segmented.



Create Multiple Layers

At a minimum, a data layer and presentation layer should be created.

Recommended: If you are modeling multiple fact tables that are joined to common

dimensional information, create multiple namespaces under the presentation layer.

A typical look & feel of FM project explorer;



Layer 1 - Data:

The data layer will consist of the content of those tables required to process the work. If

effective data warehousing modeling techniques are employed, most of the data may be retrieved from the tables. However, if there is data that will NEVER be used, then include a

filter for the query subject. For the off-clarity, organize the tables by name (to include alias

tables).

1. Detecting Cardinality from the Data Source

When importing from a relational data source, cardinality is detected based on a set of

rules that you specify. The available options are: Use primary and foreign keys.

Use matching query item names that represent uniquely indexed columns. Use matching query item names.

The most common situation is to use primary and foreign keys as well as matching query items that represent uniquely indexed columns. The information is used to set some

properties of query items as well as to generate relationships.

To view the index and key information that was imported, right-click a query subject and

click Edit Definition.

2. Create relationships and ensure cardinality rules are appropriate. This is an extremely

important component of the modeling process. Framework Manager uses the cardinality

rules to assist the query engine in generating the appropriate SQL statements.

Different types of cardinalities are:

1. One-to-one (1..1) - One-to-one (1..1) relationships occur when one instance

of data in a query subject relates to exactly one instance of another. For

example, each student has one student number.

2. One-to-many (1..n) or zero-to-many (0..n) relationships occur when one

instance of data in a query subject relates to many instances of another. For

example, each teacher has many students.

3. Many-to-many (1..n) - Many-to-many (n..n) relationships occur when many

instances of data in a query subject relate to many instances of another. For

example, many students have many teachers.

Use 1..n or 0..1 for dimensionally aware queries

Identifies fact tables in star schemas

Identifies multi-fact queries and generated ‘stitched’ SQL statements.

Use 1..1 or 0..1 for default intersection queries (Non-stitched SQL).



Screenshot reference

3. Do not create fact-to-fact joins as these can create problems (such as query

performance due to data volume, query performance with outer joins, different grains of

detail). Fact tables should be separated by a dimension.

4. Validate that keys and other identifiers are correctly identified. This is important because

as automatic aggregation occurs for numeric data that may really be an attribute as

opposed to a fact.

Tip: Make sure the fact data that will be used in calculations is set to properly handle null values.

5. Resolve loop joins or ambiguous relationships. The most common type of ambiguous

relationships are where multiple valid relationships exist between two tables, reflexive

relationships (table joins to itself). This can be resolved by creation of alias tables;

however, it is not recommended to build deep hierarchies to resolve reflexive

relationships. This should be accomplished by flattening out the table.

6. Create star schema groupings when they are opportunities to build multiple SQL paths or

there is a need to create “factless” queries.

7. Set the SQL Generation type. (Minimized means that the generated SQL contains the

minimum number of tables and joins required to obtain values for the selected query

items.)



8. Validate the data being returned in the Query Subject.

Tip 1: Remember that calculated values are treated as facts. As such, these values may

also need to be tested outside of the Framework Manager Query Subject. Recommend

testing through one of the studios.

Tip 2: Frequently, development is performed against a database that represents a very

small percentage of the data that will later be queried. During the project, be sure to test

against data that is representative of production environment data.

Tip 3: Make sure that fact data that will be used in calculations is set to properly handle

null values.

9. If row-level data security has been applied using session variables, test by overriding the

session parameters.

10. Use parameterized SQL to provide dynamic filtering and grouping capability to the report

authors.

11. Add dimensional information to the Query Subject if multiple levels exist within a single

physical table. For instance, if a table has been de-normalized to contain both the

product and product type information (not just the key) data. Levels should be set to

ensure Framework Manager understands the appropriate grouping algorithms.

12. Use determinants to identify unique data within a de-normalized table.

Determinants reflect granularity by representing subsets or groups of data in a query

subject. They are used to ensure correct aggregation of this repeated data. Determinants are most closely related to the concept of keys and indexes in the data source; they are

imported based on unique key and index information in the data source.

An example of a unique determinant is Day in Time dimension. An example of a non

unique determinant is Month; the key in Month is repeated for the number of days in a particular month.

When you define a non-unique determinant, you should specify group by. The group by indicates that when the determinant’s keys or attributes are repeated in the data, Cognos

8 should apply aggregate functions and grouping to avoid double-counting.

If you have multiple determinants with Uniquely Identified check box selected, only the Group By setting of the first determinant is used. This is because the processing of

determinants stops at the first determinant with Uniquely Identified check box selected.

Below example illustrates some determinants and how you would define them as either

unique or non-unique with a group-by.

Year Key Month Key Month Name Day Key Day Name

2006 200601 January 06 20060101 Sunday January 1, 2006

2006 200601 January 06 20060102 Monday January 2, 2006

For this data set, you can define three determinants; two group-by determinants (Year

and Month) and one unique determinant (Day).



Determinants and Attributes

If a determinant defines attributes, all query items in the query subject must be

associated with the determinant and defined as either a key or an attribute. You can have a determinant with no attributes defined for it. Framework Manager uses this type

of determinant to indicate which query items are indexed.

In the snapshot below, you can see 3 determinants for year, month and day created for

the Time Dimension



When to Use Determinants

While determinants can be used to solve a variety of problems related to data

granularity, you should always use them in the following primary cases:

o A query subject that behaves as a dimension has multiple levels of granularity and

will be joined on different sets of keys to fact data. For example, Time has multiple

levels, and it is joined to Inventory Fact on the Month Key and to Sales Fact on the

Day Key.

o There is a need to count or perform other aggregate functions on a key or attribute

that is repeated. For example, Time has a Month Key and an attribute, Days in the

month that is repeated for each day. If you want to use Days in the month in a

report, you do not want the sum of Days in the month for each day in the month.

Instead, you want the unique value of Days in the month for the chosen Month Key.

In SQL, that is XMIN (Days in the month for Month Key).

13. Collapse hierarchical information into a single Query Subject.

14. Avoid incorporating query subjects that act as both facts and dimensions.

15. By default, Cognos 8 will aggregate all query items that are identified as a fact so ensure

that numeric items are properly identified.



Layer 2 - Presentation

The arrangement of the presentation layer will be largely driven by the required deliverables.

1. Create a layer for the Report Studio developers. In creating the developer view, include

the primary content from the data view. This provides the greatest flexibility for report

authors. They can look up reference information, validate their data, create calculations,

etc. Within this layer, create additional calculations, commonly used data filters, and (as

appropriate) additional query subjects that may be used for more complex authoring

needs.

Tip 1: Where a filter will be used exclusively against a query subject, test it through

Report/Query Studio to ensure that you are achieving the desired results.

Tip 2: If possible, use shortcuts.

2. Create a layer for Query Studio. This will be a simplified view intended for case-by-case

use by the end user. The layer intended for report authors will not be visible in Query

Studio. This layer must be organized and presented in a manner that is easily

understood.

Create a Package for Each Functional Area

Recommended: Create a separate package for each functional area. This ensures that the

package stays as trim as possible and limits the demand for regression testing. Coordinate the publication of all packages with the Cognos Center of Excellence to ensure appropriate

access has been granted.

Validate the Model

During the development of the Data Model, the author should be testing their query subjects to ensure that the appropriate data is being returned. Always validate the project before

publishing to the server. The tool requires that all errors be repaired prior to publication; however, warnings should be closely examined to ensure the future integrity of the model.

Tip: Development is frequently performed against a database that represents a very small percentage production data. At some point in time during the project, these tests should be

performed against data that is more representative of the production environment.

Model Modifications

Once the report authors begin development, modifications to the structure of the Framework Manager model may negatively impact the report authoring process.

Recommended: Through the Framework Manager model, find all report dependencies. Create a validation package using the Cognos Center of Excellence assigned naming

convention. The report authors will be notified that changes have been published to the

validation package and the dependent reports should be tested against the validation package. After the successful validation of the reports, the report authors will notify the

Framework Manager that the package can be republished.

Tip: Ensure that only one copy of the validation package is saved to Content Manager.



Synchronize the Project

Frequently, changes are made to the underlying data sources that need to be propagated to

the model. The preferred method of doing this is to synchronize the project. However, if the project is large and complex, this may be fairly time-consuming. You may choose to update

individual query subjects, by selecting the update option under the tools menu.

Task Automation

If there is a need to perform identical tasks against multiple models, perform repetitive tasks

or fully reconstitute a model, convert the log files to scripts to perform repetitive tasks or

fully reconstitute a new model, save the transaction logs as scripts and execute the script as appropriate.

Model Report

Create a model report to serve as modeling artifact.



Design Tasks:

1. Naming Conventions

a. Propose Data Source Naming Convention

b. Propose Model Naming Conventions

c. Propose Model Namespace and Folder Naming Conventions Folders are used to organize objects by subject or functional area. This makes it

easier for you to locate metadata, particularly in large projects.

Drawback: The main drawback of folders is that they require unique names for all query subjects, dimensions, and shortcuts. Therefore, they are not ideal for containing shared objects such as conformed dimensions.

d. Propose Query Subject/Query Item Naming Conventions

2. Data Preparation

a. Identify Source Data

b. Identify Method of Accessing Data (e.g., native, ODBC)

3. Model Design

a. Identify location to be used for repository control

b. Design Model Namespace/Folder Structure (e.g., star, transaction)

c. Design Model Join Strategies

d. Design Model Filters

e. Design Parameter Maps

f. Design Model Calculations

g. Develop Package Strategy

h. Identify Alias & Synonyms Issues



i. Identify attribute usage

Usage property identifies the intended use for the data represented by each query item. During importing, the Usage property is set according to the type of data that

the query items represent in the data source.

You need to verify that this property is set correctly. For example, if you import a numeric column that participates in a relationship, the Usage property is set to

identifier. You can change the property.

For relational query items, the value of the Usage property depends on the type of

database object the query item is based on.

Usage Property

Database Object Description

Identifier key, index, date, datetime

Represents a column that is used to group or summarize the data in a Fact column with which it has a relationship. Also represents an indexed column. Also represents a column that is of the date or time type.

Fact numeric, time interval

Represents a column that contains numeric data that

Attribute String Represents a column that is neither an identifier not fact such has descriptions

j. Identify formatting requirements

4. Validate, Create, and Review Prototype Model

5. Deliver Proposed Physical Cognos8 Data Flow Schematic

6. Deliver Proposed Model Table, Join, & Namespace/Folder Document



Build Tasks:

1. Create Physical View

a. Create Data source b. Add Tables

c. Setup Joins

d. Alias Tables (Synonyms) e. Modify attribute usage (i.e. differentiate attribute, facts and identifiers)

f. Modify formatting as needed ($, % …) g. Analyze Joins (minimize outer joins for performance and functionality)

h. Setup Namespaces, Folders & Query Subjects i. Identify Table Filters

j. Test Joins

2. Create Performance Views

a. Resolve ambiguous joins b. Set dimensional information

3. Create Business Views (Phase 1 – Simple) a. Organize Folders & Query Subjects (use Star Schema Groupings when possible)

b. Rename Columns with Business Names c. Validate Demo Phase 1 Package (Usability Test)

4. Create User Views (Phase 2 - Enhanced)

a. Re-Organize Folders & Query Subjects

b. Adjust Columns Based on Feedback c. Construct Model Calculations

d. Construct Conditions e. Validate Demo Phase 2 Package (Usability Test)

5. Create User Views (Phase 3 - Optimized) a. Finalize Folder, Query Subject, Calculation, Filter Changes

b. Test Joins With Representative Ad-Hoc Queries c. Validate Demo Phase 3 Package (Usability Test)

6. Apply Security to Model a. Create Object Security

b. Create Data Security

7. Cognos Connection a. Model Packaging & Publishing

b. Testing

i. Security ii. Package Functionality

c. Load Testing i. Database

ii. Server Performance

iii. Network Communications iv. Web

v. Determine Critical Mass



Checklist:

Complete model design

Complete model testing (including functional and performance unit testing)

Publish model and apply security to package, objects, and data

Test Query Subjects changing sessions parameters

Perform a Checkpoint Review with appropriate Cognos Center of Excellence

representative

Create FM artifacts

Additional Reference Documents:

1. Framework Manager – User Guide (i.e. shipped along-with Cognos8 FM installation)

2. Framework Manager – Best Practices

For your quick reference, please find below embedded document

Framework_Manager_Best_Practices.pdf

framework modeling guidelines

Documents

c t i c e

proven solutions cognos

model modifications

model report

proven solutions page

source control

design tasks

build tasks