Dashboard> PentahoDataIntegration> > 03.HelloWorldExample Search

ViewEditLogIn

03.HelloWorldExample

02.SpoonIntroduction PentahoDataIntegration(Kettle)Tutorial 04.RefiningHelloWorld

HelloWorldExampleAlthoughthiswillbeasimpleexample,itwillintroduceyoutosomeofthefundamentalsofPDI:

WorkingwiththeSpoontoolTransformationsStepsandHopsPredefinedvariablesPreviewingandExecutingfromSpoonExecutingTransformationsfromaterminalwindowwiththePantool.

OverviewLet'ssupposethatyouhaveaCSVfilecontainingalistofpeople,andwanttocreateanXMLfilecontaininggreetingsforeachofthem.

IfthiswerethecontentofyourCSVfile:

last_name,nameSuarez,MariaGuimaraes,JoaoRush,JenniferOrtiz,CamilaRodriguez,CarmendaSilva,Zoe

ThiswouldbetheoutputinyourXMLfile:

Hello,Maria!Hello,Joao!Hello,Jennifer!Hello,Camila!Hello,Carmen!Hello,Zoe!

ThecreationofthefilewithgreetingsfromtheflatfilewillbethegoalforyourfirstTransformation.

ATransformationismadeofStepslinkedbyHops.TheseStepsandHopsformpathsthroughwhichdataflows.Thereforeit'ssaidthataTransformationisdatafloworiented .

PreparingtheenvironmentBeforestartingaTransformation,createaTutorialfolderintheinstallationfolderorsomeotherconvenientplace.Thereyou'llsaveallthefilesforthistutorial.ThencreateaCSVfileliketheoneshownabove,andsaveitintheTutorialfolderaslist.csv.

TransformationwalkthroughTheproposedtaskwillbeaccomplishedinthreesubtasks:

1. CreatingtheTransformation2. ConstructingtheskeletonoftheTransformationusingStepsandHops3. ConfiguringtheStepsinordertospecifytheirbehavior

CreatingtheTransformation1. ClickNew,thenselectTransformation.AlternativelyyoucangototheFilemenu,thenselectNew,thenTransformation.YoucanalsojustpressCtrlN.2. IntheViewnavigator,clickTransformation1,thenclickSettings.OrrightclickthediagramandclickTransformationSettings.OrusetheCtrl+Tshortcut.3. AwindowappearswhereyoucanspecifyTransformationproperties.Inthiscase,justwriteanameandadescription,thenclickSave.4. SavetheTransformationintheTutorialfolderwiththenamehello.Thiswillcreateahello.ktrfile.

ConstructingtheskeletonoftheTransformationusingStepsandHops

AStep istheminimalunitinsideaTransformation.AwidevarietyofStepsareavailable,groupedintocategorieslikeInputandOutput,amongothers.EachStepisdesignedtoaccomplishaspecificfunction,suchasreadingaparameterornormalizingadataset.

AHop isagraphicalrepresentationofdataflowingbetweentwoSteps,withanoriginandadestination.ThedatathatflowsthroughthatHopconstitutestheOutputData oftheoriginStep,andtheInputData ofthedestinationStep.AHophasonlyoneoriginandonedestination,butmorethanoneHopcouldleaveaStep.Whenthathappens,theOutputDatacanbecopiedordistributedtoeverydestination.Likewise,morethanoneHopcanreachaStep.Inthoseinstances,theStephastohavetheabilitytomergetheInputfromthedifferentStepsinordertocreatetheOutput.

ATransformationhastodothefollowing:

ReadtheCSVfileBuildthegreetingsSavethegreetingsintheXMLfile

Foreachoftheseitemsyou'lluseadifferentStep,accordingtothenextdiagram:

Inthisexample,thecorrespondencebetweentasksandStepsisonetoonebecausetheTransformationisverysimple.Itisn'talwaysthatway,though.

Here'showtostarttheTransformation:

1. TotheleftoftheworkspaceistheStepsPalette.SelecttheInputcategory.2. DragtheCSVfileontotheworkspaceontheright.3. SelecttheScriptingcategory.4. DragtheModifiedJavaScriptValueicontotheworkspace.5. SelecttheOutputcategory.6. DragtheXMLOutputicontotheworkspace.

NowyouwilllinktheCSVfileinputwiththeModifiedJavaScriptValuebycreatingaHop:

1. SelectthefirstStep.2. HoldtheShiftkeyanddragtheiconontothesecondStep.3. LinktheModifiedJavaScriptValuewiththeXMLOutputviathissameprocess.

SpecifyingStepbehavior

EveryStephasaconfigurationwindow.ThesewindowsvaryaccordingtothefunctionalityoftheStepsandthecategorytowhichtheybelong.However,StepName isalwaysarepresentativenameinsidetheTransformationthisdoesn'tchangeamongStepconfigurations.StepDescrip tion allowsyoutoclarifythepurposeoftheStep.

ConfiguringtheCSVfileinputStep1. DoubleclickontheCSVfileinputStep.2. TheconfigurationwindowbelongingtothiskindofStepwillappear.Hereyou'llindicatethelocation,formatandcontentoftheinputfile.3. ReplacethedefaultnamewithonethatismorerepresentativeofthisStep'sfunction.Inthiscase,typeinnamelist.4. IntheFilenamefield,typethenameandlocationoftheinputfile.

Note: Justtotherightofthetextboxisasymbolwithareddollarsign.Thismeansthatyoucanusevariablesaswellasplaintextinthatfield.Avariablecanbewrittenmanuallyas${name_of_the_variable}orselectedfromthevariablewindow,whichyoucanaccessbypressingCtrlSpacebar.Thiswindowshowsbothpredefinedanduserdefinedvariables,butsinceyouhaven'tcreatedanyvariablesyet,rightnowyou'llonlyseethepredefinedones.Amongthose,select:

${Internal.Transformation.Filename.Directory}

Nextthenameofthevariable,typeaslashandthenameofthefileyoucreated:${Internal.Transformation.Filename.Directory}/list.csv

Atruntimethevariablewillbereplacedbyitsvalue,whichwillbethepathwheretheTransformationwassaved.TheTransformationwillsearchthefile

list.csvinthatlocation.

5. ClickGetFieldstoaddthelistofcolumnnamesoftheinputfiletothegrid.Bydefault,theStepassumesthatthefilehasheaders(theHeaderrowpresentcheckboxischecked).

Note: TheGetFieldsbuttonispresentinmostSteps'configurationwindows.ItspurposeistoloadagridwithdatafromexternalsourcesorpreviousSteps.Evenwhenthefieldscanbewrittenmanually,thisbuttongivesyouashortcutwhentherearemanyavailablefieldsandyouwanttousealloralmostallofthem.

6. Thegridhasnowthenamesofthecolumnsofyourfile:last_nameandname,andshouldlooklikethis:

7.Switchlazyconversionoff8. ClickPreviewtoensurethatthefilewillbereadasexpected.Awindowshowingdatafromthefilewillappear.9. ClickOKtofinishdefiningtheStepCSVfileinput.

ConfiguringtheModifiedJavaScriptValueStep1. DoubleclickontheModifiedJavaScriptValueStep.2. TheStepconfigurationwindowwillappear.ThisisdifferentfromthepreviousStepconfigwindowinthatitallowsyoutowriteJavaScriptcode.Youwilluseittobuildthe

message"Hello,"concatenatedwitheachofthenames.3. NamethisStepGreetings.4. Themainareaoftheconfigurationwindowisforcoding.Totheleft,thereisatreewithasetofavailablefunctionsthatyoucanuseinthecode.Inparticular,thelasttwo

brancheshavetheinputandoutputfields,readytouseinthecode.Inthisexampletherearetwofields:last_nameandname.Writethefollowingcode:varmsg='Hello,'+name.getString()+"!"

Note: Thetextname.getString()canbewrittenmanually,orbydoubleclickingonthetextinthefunctiontree.

5. Atthebottomyoucantypeanyvariablecreatedinthecode.Inthiscase,youhavecreatedavariablenamedmsg.Sinceyouneedtosendthismessagetotheoutputfile,youhavetowritethevariablenameinthegrid.Thisshouldbetheresult:

Warning: Don'tmixthesevariableswithPDIvariablestheyarenotthesame.

Note: ModifiedisnotanadjectiveforJavaScript,butfortheStep.YouarenotdealingwithavariantofJavaScriptitistheStepitselfthatismodified.ItisanenhancedversionoftheoriginalStep,whichyoufoundinpreviousversionsofPDI.

6. ClickOKtofinishconfiguringStepModifiedScriptValue.7. SelecttheStepyoujustconfigured.InordertocheckthatthenewfieldwillleavethisStep,youwillnowseetheInputandOutputFields.InputFie ldsarethedata

columnsthatreachaStep.OutputFie ldsarethedatacolumnsthatleaveaStep.ThereareStepsthatsimplytransformtheinputdata.Inthiscase,theinputandoutputfieldsareusuallythesame.ThereareSteps,however,thataddfieldstotheOutputCalculator,forexample.ThereareotherStepsthatfilterorcombinedatacausingthattheOutputhaslessfieldsthattheInputGroupby,forexample.

8. RightclicktheSteptobringupacontextmenu.9. SelectShowInputFields.You'llseethattheInputFieldsarelast_nameandname,whichcomefromtheCSVfileinputStep.

10. SelectShowOutputFields.You'llseethatnotonlydoyouhavetheexistingfields,butalsothenewmsgfield.

ConfiguringtheXMLOutputStep1. DoubleclicktheXMLOutputStep.TheconfigurationwindowforthiskindofStepwillappear.Hereyou'regoingtosetthenameandlocationoftheoutputfile,and

establishwhichofthefieldsyouwanttoinclude.YoumayincludeallorsomeofthefieldsthatreachtheStep.2. NametheStepFilewithGreetings.3. IntheFileboxwrite:

${Internal.Transformation.Filename.Directory}/Hello.xml

4. ClickGetFieldstofillthegridwiththethreeinputfields.Intheoutputfileyouonlywanttoincludethemessage,sodeletenameandlast_name.5. SavetheTransformationagain.

Howdoesitwork?WhenyouexecuteaTransformation,almostallStepsareexecutedsimultaneously.TheTransformationexecutesasynchronouslytherowsofdataflowthroughtheStepsattheirownpace.EachprocessedrowflowstothenextStepwithoutwaitingfortheothers.InrealworldTransformations,forgettingthischaracteristiccanbeasignificantsourceofunexpectedresults.

Atthispoint,HelloWorldisalmostcompletelyconfigured.ATransformationreadstheinputfile,thencreatesmessagesforeachrowviatheJavaScriptcode,andthenthemessageissenttotheoutputfile.Thisisasmallexamplewithveryfewrowsofnames,soitisdifficulttonoticetheasynchronousexecutioninaction.Keepinmind,however,thatit'spossiblethatatthesametimeanameisbeingwrittenintheoutputfile,anotherisleavingthefirstStepoftheTransformation.

Verify,previewandexecute1. BeforeexecutingtheTransformation,checkthateverythingisproperlyconfiguredbyclickingVerify.SpoonwillverifythattheTransformationissyntacticallycorrect,and

lookforunreachableStepsandnonexistentconnections.Ifeverythingisinorder(itshouldbeifyoufollowedtheinstructions),youarereadytopreviewtheoutput.

Jul29,2008

2. SelecttheJavaScriptStepandthenclickPreviewbutton.Thefollowingwindowwillappear:

3. Asyoucansee,SpoonsuggeststhatyoupreviewtheselectedStep.ClickQuickLaunch.Afterthat,youwillseeawindowwithasampleoftheoutputoftheJavaScriptStep.Iftheoutputiswhatyouexpected,you'rereadytoexecutetheTransformation.

4. ClickRun.5. Spoonwillshowawindowwhereyoucanset,amongotherinformation,theparametersfortheexecutionandthelogginglevel.ClickLaunch.6. AnewwindowtabwillappearintheJobwindow.Thisisthelogtab,whichcontainsalogofthecurrentexecution.

Thelogtabhastwosections:Anupperpartandalowerpart.

IntheuppersideyoucanseetheexecutedoperationsforeachStepoftheTransformation.Inparticular,payattentiontothese:

Read:thenumberofrowscomingfrompreviousSteps.Written:thenumberofrowsleavingfromthisSteptowardthenext.Input:thenumberofrowsreadfromafileortable.Output:thenumberofrowswrittentoafileortable.Errors:errorsintheexecution.Ifthereareerrors,thewholerowwillbecomered.

Inthelowerportionofthewindow,youwillseetheexecutionstepbystep.Thedetailwilldependontheloglevelestablished.Ifyoupayattentiontothisdetail,youwillseetheasynchronicityoftheexecution.Thelastlineofthetextwillbe:

SpoonThetransformationhasfinished!!

Ifthereweren'terrormessagesinthetext,openthenewlygeneratedHello.xmlfileandcheckitscontent.

PanPanallowsyoutoexecuteTransformationsfromaterminalwindow.Thescriptispan.batonWindows,orpan.shonotherplatforms,andit'slocatedintheinstallationfolder.Ifyourunthescriptwithoutanyoptions,you'llseeadescriptionpanwithalistofavailableoptions.

ToexecuteyourTransformation,trythesimplestcommand:

Pan/file/Hello.ktr/norep

/norepisacommandtoaskSpoonnottoconnecttotherepository./fileprecedesthenameofthefilethatcontainstheTransformation.isthefullpathtotheTutorialfolder,forexample:

C:/Pentaho/Tutorial

or

/home/PentahoUser/Tutorial

Theotheroptionsarerunwithdefaultvalues.

Afteryouenterthiscommand,theTransformationwillbeexecutedinthesamewayitdidinsideSpoon.Inthiscase,thelogwillbewrittentotheterminalunlessyouspecifyafiletowriteto.Theformatofthelogtextwillvaryalittle,buttheinformationwillbebasicallythesamethatyousawinthegraphicalenvironment.

02.SpoonIntroduction PentahoDataIntegration(Kettle)Tutorial 04.RefiningHelloWorld

Comments(12)HideComments CollapseAll

Anonymoussays:

Whenfollowingthistutorial,IgetthefollowingerrorswhendoingVerifyafterhookingeverythingup:

[4Error]ModifiedJavaScriptValueCouldn'taddInputfieldstoScript!Error:java.lang.RuntimeException:Unabletoverifyif[last_nameString(9)]isnullornotbecauseofanerror:java.lang.ClassCastException:

Jul29,2008

Aug03,2008

Aug04,2008

Sep11,2008

Jun16,2009

Feb13,2012

java.lang.Stringcannotbecastto[B

[4Error]ModifiedJavaScriptValueGeneralerrorexecutingscript:org.mozilla.javascript.EcmaError:ReferenceError:"name"isnotdefined.(script#3)

IfIrunthetransformation,however,itworksasexpected.I'musingpdiopen3.0.4GA

Anonymoussays:

Iwasalsogettingexactlythesameerrorspostedafterfollowingthetutorialbuteverythingworkedrightwhenexecutingthetransformation.IhaveKettle3.0.3.GA0569installed.

Anonymoussays:

Hiya,samehere,Imean,gettingerrorasstatedabovebutthetransformationrunsfineandoutputgotasexpected...

thanksforthewonderfulworkofcreatingthistutorial.

MariaCarinaRoldansays:

Ithinkthatit'sabugrelatedwiththe"csvinputstep".Either:

1.Replacethatstepwitha"textfileinput"

or

2.Switchlazyconversionoffincsvinput(thankstosbodenforthistip)

andyoushouldbeOK.

Thanksforthe"wonderfulwork"part

mc

Anonymoussays:

Thanksforyougoodwork!

Ifacethesameerror,butwhenIturnoffthecompatibilitymode,that'sright

FrankMilbonasays:

Thankyouverymuchforthistutorial.Itisstillveryhelpfulalthoughtheuserinterfacehasalreadychangedquiteabit.

Yousaidthatthestepsofatransformationrunconcurrently.Iwonderifthisisalsotrueforthestepsofajob.Willtheyalsorunconcurrentlyorwillthenextstepinajobonlyexecuteafterthepreviousonehascompleted?

...Sorry,theanswerisonthenextpage.Jobentriesareexecutedinsequence.

KennethFreidanksays:

(v4.2.1)COMMENTSTRANSFORMATIONWALKTHROUGH

CreatingtheTransformation

Closethewelcomescreenbeforestep1.Rightclick"Transformation1"togetto'Settings',orleftclickselect"Transformation1"andclick"Edit/Settings"fromthemenu.Afterclosingthesettingswindow,thetreelayoutonthelefthandofthescreenneedstoberefreshedbeforethenewnameappearsinplaceof"Transformation1".Youcanrefreshthetreebyclickingonthe"Design"tab,thenclickingbackonthe"View"tab.Aftersavingthetransfroamtion,thetabontherighthandsideofthescreenrefreshesfromtheoldname,"Transformation1",theyournewname.

ConstructiontheSkeleton....

Here'showtostartthetransform....Clickthe"Design"tabonthelefttogettothepallette.Afterdraggingthe"csv"totheright,collapsethe"Input"categorythatyouselectedontheleft,thenyoucanseethe"Scripting"category.Collapseeachcategroyafteritsuse,soyoucanseetheothercategories.Repeatedclicksonthetriangleicontotheleftofthecategorycollapsesandexpandsthecategory.Afterdraggingthe"csvfileinput"stepontopofthe"ModifiedJavaScriptValue"step,abluearrowappearedrepresentingthehop,andapopupmenuappeared.IfIdidnotleftclickthe"Mainoutputofstep"inthepopupmenu,thehopdidnotstay,butwasdeleted.So,makesureyouclickthe"Mainoutputofstep".Thebluearrowchangestoblackwhenthehopisaccepted.

ConfiguringtheCSVfile...Uncheckthe"lazyconversion"checkbox,asothershaveposted.Don'tknowwhatthisdoes,butitremovesanerrormessageyougetlateron.

ConfiguringtheModifiedJavascript...Clickthe"Compatibilitymode"checkbox,astheotherpostershavepointedout,sothatyouwillseethe".getString()"partshowupintheInputfieldslist.

ConfiguringtheXMLoutput...Thereisaseparatetextboxthatholdsthefileextension.Thisextensionisappendedtothefilename.Thereisahandy"showfilename"buttonthatyoucanusetocheckifyou'vegotthefullpathandcorrectfilenamethatwillactuallybeused.Step4saystoclick"GetFields".Clickthe"Fields"tabatthetopofthewindowbefore"Getfields".Todeletethenameandlast_name,Ifirstclickedthenumberofthefieldontheleftofthegrid.Thisselectedtheentirerow.

Verify,Preview,...."Verify"isunderthe"Action"menuitem.Thereisalsoa"verify"iconatthetopoftheworkspace,justunderthenameofthetransformwhichshowsasthetaboftheworkspace.Theverifyiconlookslikeasheetoofpaperwithagreencheckmark.Ifyouforgottouncheckthe"lazyconversion"checkboxinthe"CSVFileSettings",youwillgetanerrorwhenverifying.The"TransformDebug"dialogboxdoesnotshowthepreviewoptionsuntilyouclickoneofthestepsontheleftofthedialogbox.Theresultsoftherunaredisplayedinanewwindowthatopensbelowtheworkspacecalled"ExecutionResults".Thelogisatabinthenew

Jun29,2012

Jul11,2013

Dec20,2013

Dec20,2013

Dec30,2013

window.ThelettersI,O,R,W,U,andEareabreviationsfortherecordedstatsINPUT,OUTPUT,READ,WRITTEN,ERRORS.Idon'tknowwhatUmeans.

BrianWusays:

@Kenneththanksforthe4.2.1updateveryyyhelpfulandwishIsawitbeforeIhadgonethroughitmyself).

Whenverifying,Igeta"Filespecificationsarenotchecked."Itriedtofigureoutwhattheproblemwasforawhile,butthetransformationendeduprunningwithouterrors,soI'vestoppedlookingintoit.

DavidLopezsays:

@Kenneth

Thankssomuchforthecomment.

RightnowIamworkingwithKettleSpoon4.4.1GAand,concerningtheExecutionstep,theExecutionResultswindowdisplayedbelowholdsfourtabs:

ExecutionHistoryLoggingStepMetricsPerformanceGraph

TheonethatcontainsINPUT,OUTPUT,READ,WRITTEN,ERRORSisStepMetrics,whichdisplaysallthesedataregardingeachstepeverythingisdisplayedinagrid,whereeachrowisaStepandeachcolumnoneoftheseattributes,Bytheway,UisUPDATED:)

RandySinuratsays:

Everytimeisettheoutputxmlstep,ialwaysgetthiserror:

"Noenumconstclassorg.pentaho.di.trans.steps.xmloutput.XMLField$ContentType."

Afterthat,icannotopenthestepconfigurationwindow.

Iamusingkettleversion5.0.1.

Whatactuallyhappened???Iamnewbie,bytheway.

Thankyou.

NOTE:

Ialreadyknowtheproblem.It'sthekettleversion.

Kettle5.0.1won'tdotheoutputworkonme.Ithinkitisnotstableyet.CMIIW.

Thankyou.

MarvinHorstsays:

Randy

IjustfinishedthehelloworldtutorialandIalsogotthiserror.TheproblemwasthatIdidn'thaveaContentTypedefinedinthefieldstaboftheXMLoutputcommand.AfterdefiningaContenttypetheerrorwentaway

RandySinuratsays:

DearMarvin,

Itriedyouradvice,andit'sworked.

Idefinedthecontenttype.Ichoseelement.

Whatidon'tknowisthedifferencebetweenelementandattributecontenttype.

Thankyou.

"ThisdocumentationismaintainedbythePentahocommunity,andmembersareencouragedtocreatenewpagesintheappropriatespaces,oreditexistingpagesthatneedtobecorrectedorupdated."

"PleasedonotleavecommentsonWikipagesaskingforhelp.Theywillbedeleted.Usetheforumsinstead."

PentahoDataIntegrationLatestPentahoDataIntegration(akaKettle)Documentation

AguidetosettingupPDIinaMicrosoftclientserverstyleenvironmentBlackBoxTestingCarteUserDocumentationClusteringwithPentahoDataIntegrationExportingresourcesFeaturecheckboxesFrequentlyAskedQuestionsGettingStartedKitchenUserDocumentationLaunchingjobentriesinparallelListofAvailablePentahoDataIntegrationPlugInsMytransformationisrunningslow,whatdoIdo?!NamedParametersPanUserDocumentationPDIDeveloperinformationPentahoDataIntegration(Kettle)Tutorial

01.InstallingKettle02.SpoonIntroduction03.HelloWorldExample04.RefiningHelloWorld

PentahoDataIntegration3.0migrationguidePentahoDataIntegrationCaseStudies

BrowseSpace

PagesLabelsAttachmentsNewsAdvanced

AddContent

NotationGuide

YourAccountAnonymous

HistoryLogInSignUp

PentahoDataIntegrationJavaAPIExamplesPentahoDataIntegrationJobEntriesPentahoDataIntegrationScreenshotsPentahoDataIntegrationRecordedDemosPentahoDataIntegrationv3.2.JobEntriesSlaveserversandclusteringSpecialdatabaseissuesandexperiencesSpoonUserGuideStepperformancemonitoringWhat'snewinPDIversion3.1What'snewinPDIversion3.2SpecialOperatingSystemissuesandexperiencesWritingyourownPentahoDataIntegrationPlugInDocumentingPentahoDataIntegration(Kettle)ProjectsKettledependencymanagementKettleExchangeOSGIenabledKettleMonitoringSWTGraphicsResourceswithSleak

DataQualityIntegrationHomePartitioningdatawithPDIImportUserDocumentationConfiguringlogtablesforconcurrentaccessPentahoDataIntegration(akaKettle)Concepts,BestPracticesandSolutionsPigScriptExecutorMarketplaceTheThinKettleJDBCdriverDatabasetransactionsinjobsandtransformations

JobcheckpointsandrestartabilityCarteConfigurationAddingIvySupportandProjectRestructureColumnFormatMongoDBOutputICNuoDBDocumentationTemplateforStepsandJobEntriesMongoDBInputICtestServices_Yarn_DocumentationAlfrescoOutputPluginforKettlePentahoDataIntegrationStepsWhat'snewinPDI4.0

MongoDBInputStep(Draft_5_3)OSGIinKettle

AdaptavistThemeBuilder(4.2.0)PoweredbyAtlassianConfluence3.3.3,theEnterpriseWiki