1209 web intelligence report validation tips and techniques.pdf

September911,2013Anaheim,California

WebIntelligenceReportValidationTipsandTechniques

MichaelWelter

2LearningPoints

First,becomefamiliarwithwhatisright Carefullyobservehowyourreportdivergencesfromright Categorizethedivergence Systematicallyfixtheproblem

3 Timesavedinvalidatingandfixingreports ConsumerconfidenceinreliableBIreports Properbusinessdecisionsbasedoncorrectanalysis Reducedwearandtearonreportdeveloper(s)

ReturnonInvestment

4 WebIntelligenceReportValidationTipsandTechniques HaveyoueverhadunexpectedresultswhendevelopingacomplexWeb

Intelligencereport?ThissessionisforWebIntelligencereportdesignersandITstaffwhosupportthem,lookingforsometimesavingpointersandfrustrationalleviatingtoolsfortroubleshootingandvalidatingreports.Seehowtoidentifyproblemsinthesourcedata,intheuniverse,andinthereportitself.Learntechniquesforverifyingaccuracy,andexplorethemysteriesofcalculationcontextsinWebIntelligence.Seethesemethodsinactionwithliveexamples.

BreakoutAbstract

51. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A

Agenda

6Afancy,beautifulreportthatswrongisuseless(orworse)

WhyAreWeTalkingAboutValidation?


BusinessObjectsGivesYouthePowerofPerspective

Turnstoreddataintodecisions Decisionmakingisamaze

Wehaveagroundlevelperspective

Needanaerialperspective Businessdecisionsrequireaccess

toinformation Flexible Dependable Accurate

7

8 Queryingadatabaseisnotatrivialmatter SQLcanbecomplex Databasedesignsareoftencomplicated

BusinessObjectsWebIntelligenceisabiglever Draganddroptobuildcomplexqueries Navigatesthelabyrinthforyou

Knowstherulesofconnectingdatabasetables Knowswhentoseparatestatementsforyou

Youdonthavetounderstandwhathappensbehindthescenes Consistentlycorrectresultsamazinglyoften!

ABigJobThatLooksEasy

9 Stillplentyofroomforerror! Onedaytobuild;Twoweeksto

debug Errorscanbeelusive Oftentakesmuchlongertofindand

fixthanoriginaldevelopment Generallyfrustrating

Weneedawaytofindandfixtheseerrors Asystematicapproachtodetecting

problems Amapforwhentofixandwhento

gethelp Asetoftoolsforwhenitstimeto

divein

DetectandFixErrors

10

YouhavebasictointermediateWebIntelligenceskills YouareusingWebIntelligencetoreportfromdatathat

Isbelievedtobecorrect Hassomebenchmarks(perhapsapreviousreport)

You(oracolleague)knowthedataandthebusinessitrepresents Thedatabaseyouarereportingfromiseither

Thesameastheoriginalwork,or Adatawarehouse/martbasedontheoriginaldatabase(andisvalid)

YouhaveITsupport(orskills) Youhaveextract/transform/loadsupport(ifwarehouse) Youhaveuniversesupport(orskills) YouhavesomeideahowtoreadSQL

Assumptions

11

Whattargetareyoutryingtohit?

WhatsRight?


12

Definebenchmarksfirst Youllsavetimeifyouknowthe

answer Ifyoudontknowwhatcorrectis,

youwontknowwhenyouhitormissit

Thenanalyzeyourreportsdeviationfromthebenchmark

Thefollowingisastrategyfordefiningbenchmarks

StartWithWhatisCorrect

13

Collectbenchmarkdatapointsfrombusinessexperts Perhapsthepersonwhowasresponsibleforthepreviousversionofa

reportyourereplacing Someonewhoknowsthesubjectandthedata Someonewhoknowshowitwasdone

Collectdatapointsatthelowestgrain Mostspecific Example:Sales_Qty,Sales_AmtofProduct1059on8/1/2007atStore21by

Employee99 andatvariouslevelsofaggregation

Increasinglymoresummarized Acrossalldimensionsinvolvedinthereport Example:Sales_Qty,Sales_AmtofallProductson8/1/2007atStore21by

allemployees

CollectDataPointsFromBusinessExpert(s)

14

Whatelementswereused? Whatcalculationswereused? Howwastimerelated? Anyoutsidedatasources?

Isanydatasourcerequiredoutsidethedatabasefromwhichwerereporting?

Ifso,howwasthatdatarelatedtotherest? GettheSQL

Ifpossible,getacopyoftheSQLused Softcopyisbest

DiscoverHowTheseDataPointsWereProduced

15

Interview Theresnosubstitutefortalking Dothisinpersonifpossible

Observation Example1:reportscurrentlyproducedfromAccess

Notewhatdatabase,queries,macros(ifany)theyused GetacopyoftheSQLused Notetheprocessingflow:Whatstepsandinwhatorder? Noteanytemporarytablesused

Example2:reportscurrentlycomingfromERPsystem Notetheprocessingflow:Whatutilities/commands,inwhatorder? AskITtoexplainhowthatappgetsitsreport AskITforcopyofSQLtheappgeneratesifpossible

DiscoveryMethods

16

VerifyyouorITisabletoduplicateresultsofbenchmarkSQL Atalllevelsofaggregationprovidedinbenchmarkdatapoints Insamesystemasoriginalreportcamefrom IndatawarehousewithanalogousSQL

VerifyrequiredSQLispresentinuniverse Askuniversedesignerforhelphere Fragmentssimilartobenchmarkexamples,or Analogousfragmentsbuilttoworkagainstdatawarehouse

Duplicatebenchmarknumberswithsimplereports MakebarebonesWebIntelligencereport CheckSQLagainstbenchmarkSQL GetIT/designerhelpasnecessary

VerifyResults

17

Howdoesyourreportdeviatefromthebenchmarks?

WhatsWrong?


18

Canadianbanktrainingtechnique Plumblineexample Nowwecanstarttoseewhats

wrong Wehavesomeknowngooddata

points Themoreyouknowthesourcedata,

themoreintuitiveyoullbeatspottingdeviation

NowYouCanStarttoSeeWhatsWrong

19

DatabaseproblemsgotoIT Universeproblemsgotoyouruniversedesigner Asermon/wordofcautionregardingyourteam

Whenaproblemisinanothersareaofexpertise Itsoutofyourhands(relief) Whydidnttheycatchthatproblem(frustration) Bytheway,its,um,outofyourhands(morefrustration)

Youneedthewholeteam Dataproblemsarearealityinyoungdatawarehouses Avoidblame

Resentmentanddefensivenessdegradeteamworkquickly Youworkhalfspeedwithaswordinonehand

YourBusinessObjectsTeam

20

Thecureisofteneasyonceyouknowwhatswrong

Thetools Thefollowingstrategyhelpsfocus

yourefforts Thefollowingtechniquesgiveyou

leverage Theapproach

Grouppossibleissuesbysymptom Workthroughthepossibilities

TheTools

21

Reportorblockfilter Theresafilterinplacethatisntobvious Demo Toolowconditiononly

Queryconditionisntright Youmayhavetoomanyortoofewwherestatements Youmaybefilteringonthewrongelements

Wronguniversemeasure(unitsinsteadofdollars?) Measureisdefinedincorrectlyorimprecisely

Ex:bookedordersvs.revisedbookings Semanticproblemneedstobeaddressedinthebusinessandinthe

universe

NumberValuesConsistentlyTooHighorTooLow

22

Showfilterpane

Viewstructure

CheckFilters

23

Selectfiltersonmapstab

CheckFilters

24

Clickthroughreportelementstofindfilters...

Aha...foundone!

CheckFilters

25

Lookforunintendedconditions

CheckQueryConditions

26

Shouldhave$insteadofunits

WrongUniverseMeasure?

27

Checkforwhatcorrectis...

ObjectDefinedIncorrectly

28

ChecktheSQLyourobjectisgenerating

ObjectDefinedIncorrectly

29

Wrongaggregationtypeonauniversemeasure Unusual,butpossible Useobjectinbarebonesreport Checkthemathonaggregation

Useobjectintableblockfooter(noagg function) Comparetoobjectusedinfooterw/aggregatefunction

Designeraddressesinuniverse

NumberValuesConsistentlyTooHighorTooLow

30

Valueseemstoolow

Especiallythetotal

WrongAggregationType

31

IDthemiscreantobject

Makeasimpledocwithit


32

Insertasumonyoursimpletable


33

...butwithnoaggregatefunction


34

Thatlooksmorelikeanaveragethanasum...


35

Ah,thebareobjectis projectingasanaverage

Askuniversedesignertofixthemeasuresaggregationfunction


36

Symptom:somevaluesaretoohigh,otherstoolow

Universeisconnectedtotestdata Youmayknowbydate,orsomeothertelltaledata Askuniversedesignertocheckthis

Wrongdatekeyinfacttablejoinedtodatetable Manydatawarehouseshaveadatelookup Facttablemayhavemanydates designerpickedone Example:Salesbydateshoulduseorderdate,notshipdate Whenviewbydate,somemonthsaretoohigh,otherstoolow ReviewSQLtoseewhatdateisbeingused

NumberValuesAreInconsistentlyWrong

37

Resultsaretoolowsomequarters,toohighothers

WrongDateKey

38

ExamineSQLstatement:itappearstouseashipdate

Askdesignertofixthedate

WrongDateKey

39

Symptom Somevaluestoohigh,othersmaybecorrect Whentoohigh,itsbyanevenmultipleofthecorrectvalue

ThissuggestsaSQL(fan/chasm)trapintheuniverse IDtheoffendingobject(s)bysystematicallypullingobjectsoutofthequery

untilthevaluesarecorrect Sharefindingswithyouruniversedesignersotheycanfix

SomeValuesTooHighbyEvenMultiples

40

Somevaluestoohigh Thosethataretoohigharesobyevenmultiples

($190,174*2=$380,348)


41

ToIDaSQLtrap,startpullingobjectsoutofthequery IsitShowroomTown?


42

ToIDaSQLtrap,startpullingobjectsoutofthequery IsitShowroomTown?

No.IsitShowroom?


43

ToIDaSQLtrap,startpullingobjectsoutofthequery IsitShowroom?

Nope.HowaboutSaleQty?


44

ToIDaSQLtrap,startpullingobjectsoutofthequery BINGO!

NowaddShowroomobjectsbackin


45

ToIDaSQLtrap,startpullingobjectsoutofthequery Revenuestilllooksgood

Nowgotellyouruniversedesigner: SalesRevenueistoohighforsomecustomers

Byevenmultiples WhenyouaddSalesQtytothequery

Yoususpectafanorchasmtraptheyllbeimpressed(orannoyed)


46

Example TY/LYsalesbylineiswrongforlastyear(buttotalisOK) WhenyoucompareTY/LYsalesbystate,everythingisfine

HistorybyCertainDimensionisWrong

47

Totalsareright,buttwolinesarewrongforlastyear


48

Maybeslowlychangingdimensionindatawarehouse DiscusswithbusinessandIT MaybesomeproductshavechangedLinessincelastyear Businessmustdecidetoviewbycurrentorhistoricalalignment Solutionisindatawarehouse(ifextract/transform/loadpersonis

abletochangebehavior)


49

Example Salesbyproductlacksnosaleproducts

MissingDimensionValuesWhereNothingHappened

50

Innerjoinusedinuniverse,butforgoodreason Weneedtoseeallcustomers,salesornosales Youcancheat:Demo

Forcereportsideouterjoin


51

Addquery:fulllistofdimensionvalues(listitemsonly)


52

Addtable:fullproductlist


53

Verifydimension(s)merged


54

Makemergedtable Usedimensionfrom2nd queryorthemergeddimension


55

1. Wrongcalculation(checkarguments)2. Objectsusedbyvariablerenderingwrongresults

Duplicatereportusingreferencedobjects Workthroughthetoolow/highsection

3. Calculationcontextisincorrect Example:calculatedcolumnvaluenotcomputedright Requiresspecificoutputcontext Demoofcalculationcontexts

CalculatedVariablesAreWrong

56

Well,THATsnotright


57

Clickcell noticeitcontainsavariable


58

Editvariable(HmmformulalooksOK)

Butletscheckhelpanyway


59

Help>FormatDate>IndextoDate&NumberFormats

ArgumentshouldbeMM/dd/yyyy(notmm/dd/yyyy)


60

Fixthevariablesformula


61

Thatsbetter!


62

Businessanalystsaysmargingoalcalculationistoolow

MargingoalcalculationlogiclooksOK

ObjectsUsedbyCalculationareWrong

63

Whatabouttheunderlyingobject? Margingoalsvariablesformulais=[Marginw]*1.1 Isthemarginwmeasuregivingustherightvalues? Letstestit:


64

Ah:marginwisnotgivingcorrectresults:

Calculationiswrongbecauseunderlyingobjectiswrong Followpreviousstepstodiagnoseproblemwiththisobject


65

Storesshouldbeclassifiedbasedonyearsopen Young,averageorold,relativetotheaverageforallstores Butallofourssayaverage somethingisamiss...

WrongCalculationContext

66

Storeclasscolumnisempty


67

Analerterpopulatesthecolumn


68

Thealerterslogicdependsonvariables


69

whichdependonothervariables


70

Letsseewhatsgoingonwith.Avg YearsOpen Duplicatethereport,andplaceanaverageonYearsOpen

Theaveragestorehasbeenopen14.6yearsatthistime


71

Letslookatour.AvgYearsOpenobjectinthetable:

.AvgYearsOpeniscalculatingtheaverageofeachrow


72

Whatwewantisforeachrowtohaveavalueof14.6 Theaverageofallrowsintheblock Justlikeourformulasittinginthefooteroftheblock

Bydefault,BusinessObjectsmeasures: Considerallthedimensionstheysee(intheirblockorsection) Aggregatetothelevelappropriatetowheretheyareplaced

Wewantour.AvgYearsOpenmeasureto: Consideralldimensionsforeachrow(default),but Aggregateasifitwereitplacedintheblocksfooter


73

ThisisCalculationContext: InputContext=thelistofdimensionsthecalculationconsiders OutputContext=theleveltowhichthecalculationaggregates Syntax:=AggregateFx([measure] input context) output context Inourcase,weonlywanttochangetheoutputcontext:


74

Now.AvgYearsOpenbehaveslikeafooterformula Butitdoessoforeachrow


75

Andnowthatourcontextforaverageiscorrect Ouralertercompareseachstoretotheaverageofallstores


76

Conclusion


77

Becomewellfamiliarwithwhatrightlookslike Carefullyobservehowyourreportdivergencesfromright Categorizethedivergence Worktheproblemsystematically

KeyLearnings

78

Indeterminingwhatisright",collectnotonlyexistingreportsandSQL,butalsoknownaccuratedatapointsatvariouslevelsofgranularity

Takethetroubletounderstandhowthebusinessworks VerifyresultsofsuppliedSQL Beoneofthoseplayersthatraisesyourteammates'game

BestPractices

79

Q&A


Thank you for participating.

Please provide feedback on this session by completing a short survey via the event

mobile application.

SESSION CODE: 1209

Learn more year-round at www.asug.com

1209 web intelligence report validation tips and techniques.pdf

Documents