interactive visualization of volumetric data with webgl in...
TRANSCRIPT
![Page 1: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/1.jpg)
InteractivevisualizationofvolumetricdatawithWebGLinreal-time
MarioJ.Lorenzo
![Page 2: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/2.jpg)
Problem
Renderingofvolumetricdatawiththefollowing…– Real-time– Realistic– Interactive
…supportedthroughaweb-browser
![Page 3: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/3.jpg)
ProblemDomainBackground
MedicalMedicalimagingtechniquessuchasCTscan,MRI,andPETproducesetsofparallelslicesthatformvolumetricdataset.StorageandDistributionofmedicalimagesaremanagedbyaPictureArchivingandCommunicationSystems(PACS)
• Client–to-Serverarchitecture• Requiresspecializedsoftware
Congoteetal,2011
WeatherWeatherdataisalsovolumetric• Collectedbyusingradarthatscansavolume
surroundingit• Collectsvaluesinacircularorconicalscansurface• Dopplerradaralsosamplesphysicalvariables
(reflectivity,radialvelocityetc)
![Page 4: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/4.jpg)
ProblemDomainMotivation
• ScientificfieldssuchasMedicalImagingandMeteorologyhavebeenpioneersin3Dvisualization
• VisualizationistheprimarytargetofanalysisbyRadiologistandMeteorologist– TheadventofCTandMRIscansprovidesslicesofimagesthatcanbeusedto
render3Dvolumes– 3Dvolumerenderingasabasisforvisualassessmentbyradiologistisstillnewand
notwidelyused.– Mayrequirere-trainingforhowradiologistsdiagnose
• Improvingfidelityanddimensionalityofthevisualizationcandirectlyimprovetheoverallaccuracyofthedecisionmaking.
![Page 5: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/5.jpg)
BackgroundandPriorResearch
• VolumeRenderingseekstoprojecta3Dmodelinto2Dimage– Theabilitytointeractinreal-timegivestheillusionofa3Dimage
• PriortoWebGL,3Dvisualizationwasperformedthroughnativeapplicationsornon-standardplug-insinstalledinthewebbrowser– Requiresaccesstolocalhardware(i..eGPU)
• Mostlyrenderingpolygonmeshes– StraightforwardandsupportedbyGPUrasterrendering
![Page 6: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/6.jpg)
Background:VolumeRending
LEVOY,M.1988.Displayofsurfacesfromvolumedata.IEEEComput.Graph.Appl.8,3,29–37.
Demonstratedthatvolume-renderingtechniquesproducedbetterimagesfromvolumetricdatathanusingconventionalgeometricprimitives.
MarcLevoy-consideredapioneerinvolumerendering;workedonvisualizingCTimaging
Levoy,1988
![Page 7: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/7.jpg)
Hadwigeretal-VolumeRaycastingHADWIGER,M.,LJUNG,P.,SALAMA,C.R.,ANDROPINSKI,T.2009.Advancedilluminationtechniquesforgpu-basedvolumeraycasting.InACMSIGGRAPH2009Courses,ACM,1–166.• DescribesVolumeray-castingismostbasic,butmostflexiblevolumerendering
algorithm• Ray-castingasanumericalmethodforevaluatingvolumerenderingintegral
• Integralisbasedonintegratinglightinteractioneffectsalongaviewingrayontheopticalmodel
• Absorption–radiantenergyabsorbedoverdistance• AlphaBlending–combiningtranslucentforegroundcolorwithbackgroundcolor
Volumeissampledatregularintervalsbetweenstartingandendingpositions Hadwigeretal,2009
Emptyskippingandestablishingboundinggeometry
![Page 8: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/8.jpg)
ProposedMethodCongote,J.,Segura,A.,Kabongo,L.,Moreno,A.,Posada,J.,&Ruiz,O.(2011,June).Interactivevisualizationofvolumetricdatawithwebgl inreal-time. InProceedingsofthe16thInternationalConferenceon3DWebTechnology(pp.137-146).ACM.
• ApproachreliesonLevoyandHadwigeretalworkonvolumerenderingalgorithms
• AdaptedVolumeRay-castingalgorithmsforWebGL
• Volumeinterpolationfrom2Dimageslices• Avoidsuseofserver-sideforremoterendering,enablingbetterscalability
forlarge-userenvironments(suchasinmedicalimaging)• Doesnotrequirepreprocessing
• Realismisaccomplishedthroughtheuseofblendingfunctions,transferfunctions,andPhongillumination
![Page 9: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/9.jpg)
ComposingTexturefromImageslices• Constructa“texture”bytransformingindividualimageslicesintomosiac(matrix)ofimages• UsesinterpolationtofillinthegapsinbetweentheslicesIdentificationofRaycoordinates• Usesacubeastheboundinggeometryfortheimagebeingrendered• Rendersthecubeintosceneusingviewpoint• Firstrendersthecolorusingdepthfunctionchange
– 1stpasspresentsclosestregionofthecubetothecamera– 2ndpasspresentsfarregion
Raygeneration• Rayiscalculatedfromfrontregiontobackregion
– Colorinterpolationisused• Cartesianx,y,zcoordinatesdefinedwhereonedimensionisanimagesliceselectorandthe
othertwoisacoordinatewithinthatimage• RayisdividedbySsteps(numberofsamples)
– Computex,y,zforeachsampleinthevolume– Valueofpositionisinterpolatedfromthetexture
TransferFunction• Usevalueoftextureat(x,y,z)toidentifycolor• Usecolorincompositionfunction• Rayfinisheswhencompositionfunctionreaches
theendinthecube• Assignscolorvaluetoeachpixelalongtheray
MethodDetails
![Page 10: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/10.jpg)
CTImageSlicesusedforVolumeRendering
![Page 11: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/11.jpg)
Congoteetal,2011
![Page 12: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/12.jpg)
PerformanceResults
![Page 13: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/13.jpg)
DemoofCTScan3DRendering
Congote, J., Segura,A.,Kabongo, L.,Moreno,A.,Posada, J.,&Ruiz,O. (2011, June).Interactivevisualizationofvolumetricdatawithwebglinreal-time.InProceedingsofthe16thInternationalConferenceon3DWebTechnology(pp.137-146).ACM.http://demos.vicomtech.org/volren/
![Page 14: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/14.jpg)
RelatedWork
Haehn,D.,Rannou,N.,Ahtam,B.,Grant,E.,&Pienaar,R.(2014).NeuroimaginginthebrowserusingtheXToolkit.FrontiersinNeuroinformatics,101.
Qiao,L.,Chen,X.,Zhang,Y.,Zhang,J.,Wu,Y.,Li,Y.,...&Qiu,M.(2017).AnHTML5-BasedPureWebsiteSolutionforRapidlyViewingandProcessingLarge-Scale3DMedicalVolumeReconstructiononMobileInternet.Internationaljournaloftelemedicineandapplications,2017.
• Combinesurfacerenderingcombinedwithremotevolumerendering
• WebGLbasedonclientsideandnativeprocessforvolumerendering
• Solutionfor3Dmedicalvolumereconstruction• Claimitis“purewebenvironment”,butreliesonremote
renderingcalled“master-slave”
• XTK–“TheXToolkit”• ClaimstobefirstWebGL/JavaScript-based
visualizationformedicalimaging• http://socr.umich.edu/HTML5/BrainViewer/
![Page 15: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/15.jpg)
Review
• Latencyforloadingtheoriginalimages(whichcanbeverylargeinmedicalimaging)– Considercompressionoptimizations,videocompressionschemesmayhelp
eliminatetemporalredundancy
• Surfacerenderingfortexturescanbeveryhelpfultoradiologists• ExploreuseofThreeJSframeworktoabstractfromWebGL(suchasXTKand
ATM)• ReducedResolution
– MedicalImagesuselargesamplebitdepth(vscommon8bit)
• IssueswithreliabilityandconsistencyofJavascriptsetTimeout()functionhavinga10msoverheadpercall,causingperformanceissues
• Exposingadditionalcontrolsformanipulatinglighting,maybehelpful
• Ageneralconcernontheeffectsofinterpolationandothersimulationsthatcandistortormisleadanexpert
![Page 16: Interactive visualization of volumetric data with WebGL in real-timeblog.mjlorenzo.com/papers/MarioJLorenzo_ReviewVolumetric... · 2018-10-13 · • Prior to WebGL, 3D visualization](https://reader034.vdocuments.site/reader034/viewer/2022043004/5f8649eac51aa21ad02aa587/html5/thumbnails/16.jpg)
ReferencesReviewedPaper:Congote,J.,Segura,A.,Kabongo,L.,Moreno,A.,Posada,J.,&Ruiz,O.(2011,June).Interactivevisualizationofvolumetricdatawithwebglinreal-time.InProceedingsofthe16thInternationalConferenceon3DWebTechnology(pp.137-146).ACM.Congote,J.,Segura,A.,Kabongo,L.,Moreno,A.,Posada,J.,&Ruiz,O.–DemoofReal-TimeInteractiveVisualizationofVolumetricDatawithWebGL.http://demos.vicomtech.org/volren/.AdditionalpapersHADWIGER,M.,LJUNG,P.,SALAMA,C.R.,ANDROPINSKI,T.2009.Advancedilluminationtechniquesforgpu-basedvolumeraycasting.InACMSIGGRAPH2009Courses,ACM,1–166.Haehn,D.,Rannou,N.,Ahtam,B.,Grant,E.,&Pienaar,R.(2014).NeuroimaginginthebrowserusingtheXToolkit.FrontiersinNeuroinformatics,101.LEVOY,M.1988.Displayofsurfacesfromvolumedata.IEEEComput.Graph.Appl.8,3,29–37.Min,Q.,Wang,Z.,&Liu,N.(2018).AnEvaluationofHTML5andWebGLforMedicalImagingApplications.Journalofhealthcareengineering,2018.