the virtual microscope

Download The Virtual Microscope

Post on 05-Feb-2016

26 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

The Virtual Microscope. Umit V. Catalyurek Department of Biomedical Informatics Division of Data Intensive and Grid Computing. The Virtual Microscope. Joel Saltz Renato Ferreira Michael Beynon Chialin Chang Alan Sussman Tahsin Kurc Robert Miller Angelo Demarzo Mark Silberman - PowerPoint PPT Presentation

TRANSCRIPT

  • The Virtual MicroscopeUmit V. CatalyurekDepartment of Biomedical InformaticsDivision of Data Intensive and Grid Computing

  • The Virtual MicroscopeJoel SaltzRenato Ferreira Michael BeynonChialin ChangAlan SussmanTahsin KurcRobert MillerAngelo DemarzoMark SilbermanAsmara AfeworkAnthony Wiegering

  • Virtual Microscope (VM)Interactive software emulation of high power light microscope for processing image datasetsvisualize and explore microscopy imagesscreen for cancercategorize images for associative retrievalelectronic capture of slide examination process used in resident trainingcollaborative diagnosisVirtual Microscope (Hopkins/UMD), Distributed Telemicroscopy System (Rutgers), [Gu] Virtual Telemicroscope, Virtual Microscopy (UPMC), Baccus Virtual Microscope

  • The Virtual MicroscopeData requirementFull cases consisting of multiple digitized glass slides with data acquired at 400XSingle spot 1000x1000 pixels, 3-byte RGB=3MBA slide of 2.5cmx3.5cm requires 50x70 grid = 10GB uncompressedEach slide can have multiple focal planesJohns Hopkins alone generates 500,000 slides per year

  • The Virtual MicroscopeClient-server architectureJava 1.2 ClientPortabilityData storage & Image compressionMore efficient storage, reduced transmission time2 server implementations:Customized instance of Active Data RepositoryImproved scalability, portability, user-defined processingComponent-based implementation using DataCutterHeterogeneous systems, portability, user-defined processingCaching in the VM ClientImproved response timeExperimental Results

  • VM Client

  • VM Client

  • Image Declustering

  • Image CompressionJPEG compression - storage and networkdata reduction by a factor of 10still may take long time to transmit imagesFor example, 640x480 image920 KB uncompressed~ 90 KB jpeg compressed~ 13 seconds to transfer using 56 Kb modem

  • Active Data Repository (ADR)A C++ class library and runtime system for building parallel databases of multi-dimensional datasetsenables integration of storage, retrieval and processing of multiple datasets on parallel machines and clusters. provides support for common operations such as data retrieval, memory management, scheduling of processing across a parallel machine.can be customized for various applications.Front-end: the interface between clients and back-end. Back-end: data storage, retrieval, and processing.Distributed memory parallel machine or cluster, with multiple disks attached to each nodeCustomizable services for application-specific processing

  • Virtual Microscope with ADRQuery InterfaceServiceQuery SubmissionServiceFront-endVirtual Microscope Front-end Dataset ServiceAttribute SpaceServiceData AggregationServiceIndexingServiceQuery ExecutionServiceQuery PlanningServiceBack-endClientClientClientClient...Query:* Slide number* Focal plane* Magnification* Region of interestImage blocks

  • DataCutterA suite of Middleware for subsetting and filtering multi-dimensional datasets stored in a distributed environment Indexing Service Multilevel hierarchical indexes based on spatial indexing methods e.g., R-treesFiltering ServiceDistributed C++ component frameworkSpecialized components for processing datafilters logical unit of computation, high level tasks, init,process,finalize interfacestreams how filters communicateunidirectional buffer pipesuses fixed size buffers (min, good)manually specify filter connectivity and filter-level characteristics

  • Virtual Microscope with DataCutterDC-5FDC-3FDC-2F

  • Caching in the ClientReduce data re-transmission Cache part of the retrieved data in the clientCache multiple resolutions/magnificationsCache only what the user viewsTwo-level cacheclient memory is the first level cachelocal disk on the client machine is the second level

  • Caching Multiresolution Images

  • VM Server Performance

  • ADR VM Server Performance

  • VM ADR Server under workload

    Chart2

    3.443111

    1.8523.197.61825.568

    1.1131.8444.26814.043

    0.7681.2332.6588.537

    ADR

    ADR-1bg

    ADR-4bg

    ADR-16bg

    Number of Processors

    Response Time (seconds)

    Average Response Time for 1024x1024 Output

    SMP-deathstar

    RDCZV

    Transparent copies for DC/#process for ADRnameRDCZV512x5121024x1024

    w/ hoardw/ hoardw/o hardw/o hardw/o hardw/o hardw/ hoardw/ hoardw/ hoardw/ hoardADR-8node0.3180.903

    512ADRDC-5FDC-2FDC-5FDC-3FDC-2FMR-NCMR-NCMR-wCSR-NCSR-wCDC-5F-4copies0.5531.797

    11.0721.2601.0321.0060.9821.0100.9781.0091.0681.0481.096DC-3F-8copies0.3581.057

    20.6040.9290.6560.6150.6030.6100.8650.9040.7400.6530.680DC-2F-8copies (8xR-8xDCZV)0.3360.987

    40.3951.0280.4610.5530.3950.3960.6960.7330.4740.4600.4741xR-2xDCZV120.6011.954

    80.3182.0540.4480.6160.3580.3360.3620.4750.4880.4490.4701xR-4xDCZV140.3911.215

    min0.3180.9290.4480.5530.3580.3360.3620.4750.4740.4490.4701xR-6xDCZV160.3391.015

    1xR-8xDCZV180.3260.966

    1024ADRDC-5FDC-2FDC-5FDC-3FDC-2FMR-NCMR-NCMR-wCSR-NCSR-wC2xR-3xDCZV230.4651.463

    13.9574.2043.4703.3493.2733.3973.2423.3593.5923.4923.7172xR-4xDCZV240.3961.222

    22.1453.0452.1121.9871.8651.9713.1253.2423.0942.1292.2232xR-6xDCZV260.3361.006

    41.3023.4431.4361.7971.1851.2272.8903.0042.4011.4411.4962xR-8xDCZV280.3200.959

    80.9037.7251.3841.9841.0570.9872.5002.7231.7711.3871.4414xR-5xDCZV450.3601.090

    min0.9033.0451.3841.7971.0570.9872.5002.7231.7711.3871.4414xR-6xDCZV460.3361.011

    4xR-7xDCZV470.3260.973

    512x5124xR-8xDCZV480.3210.960

    #clientADR8x(R-DCZV)8x(R-DCZV)-wC4x(2xR-2xDCZV)2x(4xR-4xDCZV)4x(2xR-4xD-2xCZV)4x(2xR-3xD-2xCZV)1xR-4xD-1xCZV1410.3791.147

    10.3271.0201.0790.6120.4040.3920.4511xR-6xD-1xCZV1610.3700.985

    20.4961.1211.1460.7040.5670.5590.5571xR-8xD-1xCZV1810.3801.149

    41.0271.1571.2240.8500.8120.8670.8702xR-4xD-1xCZV2410.3791.152

    82.1821.4361.4791.4931.5911.7001.7212xR-6xD-1xCZV2610.3711.141

    2xR-8xD-1xCZV2810.3791.155

    1024x10244xR-5xD-1xCZV4510.3741.143

    #clientADR8x(R-DCZV)8x(R-DCZV)-wC4x(2xR-2xDCZV)2x(4xR-4xDCZV)4x(2xR-4xD-2xCZV)4x(2xR-3xD-2xCZV)4xR-6xD-1xCZV4610.3741.143

    10.9133.4303.6321.9751.2301.1591.3904xR-8xD-1xCZV4810.3811.160

    21.3883.6573.9432.1621.8261.6731.6991xR-4xD-2xCZV1420.3761.143

    42.4774.0394.1022.8052.8563.0282.9161xR-6xD-2xCZV1620.3210.960

    84.9674.8465.0955.1445.5205.9045.8641xR-8xD-2xCZV1820.3341.016

    2xR-4xD-2xCZV2420.3851.158

    2xR-6xD-2xCZV2620.3300.972

    2xR-8xD-2xCZV2820.3441.016

    4xR-5xD-2xCZV4520.3491.034

    4xR-6xD-2xCZV4620.3330.982

    4xR-8xD-2xCZV4820.3461.017

    1xR-4xD-4xCZV1440.3861.172

    1xR-6xD-4xCZV1640.3381.008

    1xR-8xD-4xCZV1840.3351.007

    2xR-4xD-4xCZV2440.3891.173

    2xR-6xD-4xCZV2640.3371.009

    2xR-8xD-4xCZV2840.3361.008

    4xR-5xD-4xCZV4540.3531.055

    4xR-6xD-4xCZV4640.3381.021

    4xR-8xD-4xCZV4840.3411.015

    SMP-deathstar

    0000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000

    ADR-8node

    DC-5F-4copies

    DC-3F-8copies

    DC-2F-8copies (8xR-8xDCZV)

    1xR-2xDCZV

    1xR-4xDCZV

    1xR-6xDCZV

    1xR-8xDCZV

    2xR-3xDCZV

    2xR-4xDCZV

    2xR-6xDCZV

    2xR-8xDCZV

    4xR-5xDCZV

    4xR-6xDCZV

    4xR-7xDCZV

    4xR-8xDCZV

    1xR-4xD-1xCZV

    1xR-6xD-1xCZV

    1xR-8xD-1xCZV

    2xR-4xD-1xCZV

    2xR-6xD-1xCZV

    2xR-8xD-1xCZV

    4xR-5xD-1xCZV

    4xR-6xD-1xCZV

    4xR-8xD-1xCZV

    1xR-4xD-2xCZV

    1xR-6xD-2xCZV

    1xR-8xD-2xCZV

    2xR-4xD-2xCZV

    2xR-6xD-2xCZV

    2xR-8xD-2xCZV

    4xR-5xD-2xCZV

    4xR-6xD-2xCZV

    4xR-8xD-2xCZV

    1xR-4xD-4xCZV

    1xR-6xD-4xCZV

    1xR-8xD-4xCZV

    2xR-4xD-4xCZV

    2xR-6xD-4xCZV

    2xR-8xD-4xCZV

    4xR-5xD-4xCZV

    4xR-6xD-4xCZV

    4xR-8xD-4xCZV

    ADR vs DC

    DC vs ADR

    00000

    00000

    00000

    00000

    ADR

    8x(R-DCZV)

    4x(2xR-2xDCZV)

    2x(4xR-4xDCZV)

    4x(2xR-4xD-2xCZV)

    Number of Clients

    Response Time (seconds)

    Average Response Time for 512x512 Output

    q1024

    00000

    00000

    00000

    00000

    ADR

    8x(R-DCZV)

    4x(2xR-2xDCZV)

    2x(4xR-4xDCZV)

    4x(2xR-4xD-2xCZV)

    Number of Clients

    Response Time (seconds)

    Average Response Time for 1024x1024 Output

    q512

    Transparent copies for DC/#process for ADR

    w/ hoardw/ hoardw/o hardw/o hardw/o hardw/o hardw/ hoardw/ hoardw/ hoardw/ hoard

    512SMP512ADRDC-5FDC-2FDC-5FDC-3FDC-2FMR-NCMR-NCMR-wCSR-NCSR-wC#clientADRDC-2F-wC512x5121024x1024Cold Cache

    #procsADRADR-1bgADR-4bgADR-16bgDC-1VDC-5FDC-5F-1bgDC-5F-4bgDC-5F-16bgDC-2FDC-2F-1bgDC-2F-4bgDC-2F-16bg11.0721.2601.0321.0060.9821.0100.9781.0091.0681.0481.09610.3271.124ADR-8node0.3180.903512ADRDC-2F

    10.9290.9290.9290.9291.5021.5021.5021.5021.5020.9420.9420.9420.94220.6040.9290.6560.6150.6030.6100.8650.9040.7400.6530.68020.4961.261DC-5F-4copies0.5531.79713.6634.1122xR-4xDCZV1.5294xR-5xDCZV1.373

    20.5241.0062.3158.110.8550.8381.0541.4452.6460.5470.7381.1052.22540.3951.0280.4610.5530.3950.3960.6960.7330.4740.4600.47441.0271.333DC-3F-8copies0.3581.05722.0052.3782xR-5xDCZV1.6624xR-6xDCZV1.366

    40.3350.6621.5615.4870.5320.5020.6470.9552.2160.340.4680.7871.9580.3182.0540.4480.6160.3580.3360.3620.4750.4880.4490.47082.1822.305DC-2F-8copies (8xR-8xDCZV)0.3360.98741.3461.5892xR-6xDCZV1.3894xR-7xDCZV1.400

    80.2460.4871.2073.780.3620.3410.4430.7662.1780.2370.3360.6311.787min0.3180.9290.4480.5530.3580.3360.3620.4750.4740.4490.4701xR-2xDCZV0.6011.95481.2241.4842xR-8xDCZV1.4444xR-8xDCZV1.421

    1xR-4xDCZV0.3911.215163.073

    1024ADRDC-5FDC-2FDC-5FDC-3FDC-2FMR-NCMR-NCMR-wCSR-NCSR-wC#clientADRDC-2F-wC0.0001xR-6xDCZV0.3391.015

Recommended

View more >