introduction to laboratory of dis, ntuim

Introduction toLaboratory of DIS, NTUIM

Introduction toLaboratory of DIS, NTUIM

Andy Cheng, C. H. Kang, C. T. FangDIS Lab, Dept. of Information Management, NTU Mar. 14, 2003

2003/03/14 Introduction to Laboratory of DIS, NTUIM

2

OutlineOutline

• Introduction• Director & Members• Research Topics

• Wireless Ad Hoc Network & Topology Formation (2000 ~ 2002)• Personal Data Licensing (2000 ~ )• Peer-to-Peer Networks (2000 ~ )• Grid Computing (2002 ~ )• Agent (2002 ~ )• Recommendation (2002 ~ )• Reputation (2002 ~ )

• Publications• Awards & Honors• References


3

IntroductionIntroduction

• Research:• Distributed computing

• Visions:• Before 1Q 2006, 20 conference papers accepted by first-class

ACM conference/journals.• Before 1Q 2004, a sound training course include distributed sy

stems/algorithms and software engineering for undergraduate students.

• Slogans: • “格物致知” , • “跟我們聯誼吧 !”

• Style & Culture• 老實 , 勤奮 , 努力 , 向上 ...

真的嗎 ?!

你相信了喔 !!


4

Director & MembersDirector & Members

• Director: Dr. Yuh-Jzer Joung• http://joung.im.ntu.edu.tw/joung/

• Members:• PHD Students:

• CSC, Shi-Cho Cha• http://mba.ntu.edu.tw/~csc/• Eric, Yu-En Lue• http://www.yuenlue.com/

• 2nd Year Graduate Students:• Andy Cheng• C. H. Kang• C. T. Fang

http://joung.im.ntu.edu.tw/joung/

http://mba.ntu.edu.tw/~csc/



http://www.yuenlue.com/


5

Members (con’d)Members (con’d)

• 1st Year Graduate Students: • 王教昌 , 宋華偉 , 李柏奇 , 林宜均 , 林章汶 , • 林盟凱 , 陳彥瀚 , 黃翊展 , 黃鈞塘 , 嚴正 .

• Current Undergraduate Students: • 李金諺 , 邱建樺 , 陳宇翔 , 陳宏典 , 陳玠均 , • 楊伶琪 , 詹子儀 , 簡伯翰 .


6

Wireless Ad Hoc Network & Topology Formation (2000 ~ 2002)


G. D. Haung

C. J. Hsu

S. H. Liu

Alan Chen

Gary Chung


7



• 可任意建置與擴展之隨到即連在地性服務 --系統建構與實作 by S. H. Liu

• 藍芽模擬環境的建構 by C. J. Hsu• 網路建構與路由演算法於 MANET 中效能比較

by Alan Chen• 建立藍芽拓樸之分散式演算法 by G. D. Haun

g


8

可任意建置與擴展之隨到即連在地性服務 --系統建構與實作

可任意建置與擴展之隨到即連在地性服務 --系統建構與實作

• 以無線區域網路 (Wireless LAN) 和無線行動隨建即連網路 (Mobile Ad Hoc Network MANET) 為基底 (pic)

• 目前所開發的系統程式是用於個人數位助理 (PDA) 之上


9

藍芽模擬環境的建構藍芽模擬環境的建構

• BuleHoc 不足之功能：裝置行為單一固定、行動模型支援從缺、模擬結果尚未整合、藍芽微網 (Piconet) 數目受限

• 建構在 IBM BlueHoc 之上，提供一個更有彈性、更為完善的藍芽模擬平台

• 實際執行相關演算法，證明可以改善原有的缺點　　 Ex: 模擬結果整合 (pic)


10

網路建構與路由演算法於MANET中效能比較網路建構與路由演算法於MANET中效能比較

• 研究 MANET 上 Hierarchical Routing 演算法或協定的效能

• 在模擬環境下進行實作，根據不同的節點移動模式針對整體網路的構建與相關路由的效能做一綜合性的比較

• 結論：沒有演算法是最佳的，所有的演算法都具備各自的優點及缺點，針對不同的環境，建議使用不同的路由演算法


11

建立藍芽拓樸之分散式演算法建立藍芽拓樸之分散式演算法

• 藍芽裝置在溝通前，必須先建立點對點的連線，形成一個藍芽拓譜　 (pic)

• 發展一個演算法可以快速地建構拓撲、確保拓撲的連通性、拓撲利於快速地路由

• 使用模擬器驗證演算法能夠正確的運作，並且擁有上述的優點


12

Gossip-based Resource Discovery Algorithms Gossip-based Resource Discovery Algorithms

• 發現資源問題：在分散式環境中，如何讓每個節點都能知道其他節點的存在

• 在認識其他節點的過程中，可以保有下面的性質：距離越近的節點，越容易被認識

• 用 ns2 模擬器驗證演算法的正確性


13

Personal Data Lisencing (2000 ~ )Personal Data Lisencing (2000 ~ )

S. C. Cha


14

Personal Data Licensing (2000 ~ )Personal Data Licensing (2000 ~ )

• Personal Data Backbone (PDB) is a Universal Profile System (UPS) over P2P network.

• A UPS is developed to allow a user to access different information services with only a single action of authentication and authorization. E.g., Microsoft Passport

• PDB can tolerate “a certain degree of failures” and to offer accountability of personal data access.


15

Peer-to-Peer Networks (2000 ~ )Peer-to-Peer Networks (2000 ~ )

J. W. Lin

Eriko Lue

Juicy WangC. T. Fang

Andy LinC. H. Gang


16

Peer-to-Peer Networks (2000 ~ )Peer-to-Peer Networks (2000 ~ )

• A P2P network by definition is a fully distributed, non-hierarchical network and each participating node is symmetric.

• Topics we are working on: • Distributed naming and routing.• Surrogate routing.• Meta search on P2P networks.• P2P file sharing system.• CDN over P2P networks.• Power computation networks.


17

Distributed Naming & RoutingDistributed Naming & Routing

• The problems is how to name a node/object and to resolve a name of a node/object on a large scale and totally decentralized environment.

• There are three usual ways to achieve this:• Centralized indexing, e.g., Napster.• Query flooding, e.g., Gnutella.• Hash and heuristic-based routing, e.g., Chord, CAN, Tap

estry, Pastry, P-Grid.• Others, e.g., PlanetP.


18

Napster’s ApproachNapster’s Approach

Napster’s Server Cluster

Where is “Dr. E

vil.avi”?

Dr. [email protected]

140.112.107.79


19

Gnutella’s ApproachGnutella’s Approach

Who has “Dr. Evil.avi”?

I have it!

I have it!

I am Dr. Evil!


20

Distributed Hash TableDistributed Hash Table

• A hash function maps a node/object onto a NodeID/ObjectID space.

• Nodes are organized as a hypercube, or a skip list.

• Objects are inserted into a distributed hash table (DHT).

• To resolve a name of a node/object, request is routed along a hypercube/skip list path.

A 4D hypercube


21


22

6610

5F88

7557

4567

3F88 83101234

8887

D357 1357

5678 AA57

舞澤圓

Hash ( [cyndi]舞澤圓 .avi ) = 5581357

IP: 140.112.107.68

[cyndi]舞澤圓 .avi @ 140.112.107.68

IP: 203.23.91.178

舞澤圓


23

Distributed Naming & Routing (con’d)Distributed Naming & Routing (con’d)

• Mystry – a Plaxton mesh based scheme designed by Eric, Yu-En Lue.

• Terrorists Win – a skip list based topology with dynamic load balancing mechanism – by OB, C. H. Kang.

• MagicCube – a hypercube based routing with surrogate selection algorithm – by J. C. (Juicy) Wang, J. W. Lin.


24

MagicCubeMagicCube


25

Surrogate RoutingSurrogate Routing

• In practical implementation, the number of nodes is often greatly less than the NodeID space.

• A deterministic surrogate routing scheme guarantees all nodes choose the same surrogate for a failure/inexistent node.


26

Keyword Search on DHT-based P2P NetworkKeyword Search on DHT-based P2P Network

• Google’s service is an example of keyword search, while it is a centralized approach.

• Current DHT-based P2P network only support search with exact object name.

• Distributed inverted index supports keyword search on DHT-based P2P networks while problem of unbalanced load arises.

• We’re now working on the improvement of distributed inverted index and a directed search using Bloom-filter ObjectIDs.


27

Keyword Search on DHT-based P2P Network (con’d)Keyword Search on DHT-based P2P Network (con’d)

Java

P2P

cyndi

Azumi

JavaP2P

Azumicyndi

RyokoAzumi

AzumicyndiP2P

Split based on Keywords Split based on Documents


28

Probabilistic Resource LocationProbabilistic Resource Location

• Andy Cheng finds that randomized replication on a P2P network guarantees certain level of hit rate.

• His research is currently on how to develop a gossip-based approach achieving randomized replication.

• In his thesis, a simulation will help to identify the performance.


29

P2P File Sharing SystemP2P File Sharing System

• MyP2P file sharing system is built on Mystry base layer.

MyP2P Common Service PlatformMyP2P Common Service Platform

Application ServicesApplication Services

BaseBaseMessage Routing

PlatformSecurity

Logical Identity

NameResolution

TopologyConfiguration

SystemServices

SystemServices

Replication&

Immigration

SecurityGuard

Agent/Service DiscoveryResourceAllocation

Service

Personal DataManagement Agent

Personal DataManagement Agent

Personal Identity & Profile

AccessControl

Personal PrivacyManagement

Agent Dock 1 Agent Dock 2

PowerComputingNetworkAgent

PowerComputingNetworkAgent

Agent Dock 2

P2PMessaging& FileSystem

P2PMessaging& FileSystem

Agent Dock 4

AgentCompositor

AgentCompositor

Agent Dock 5

InstantMessagingAgent

InstantMessagingAgent


30

CDN over P2P NetworksCDN over P2P Networks

• Content Delivery Network (CDN) delivers digital content to consumers with better scalability, better availability, and more efficient capacity utilization.

CNN web server

Users in U.S.Users in Asia

Users in Europe CRASHBOOMBANGAkamai CDN Serv

iceAkamai CDN Service


31

CDN over P2P Networks (con’d)CDN over P2P Networks (con’d)

• Content Sources join the network as MyP2P nodes and the whole network will be servicing as a content delivery network (CDN).

• When a file requested is transferred back along the path of request routing, each node along the path will dynamically decide to cache the file or not.

• Randomized hierarchical paging intelligently determines where to place replicas.


32

Power Computation NetworksPower Computation Networks

• Many difficult problems e.g., protein folding require enormous computation powers.

• We combine agent technology and randomized job assignment to provide computation power of a P2P network.

• Problems are implemented by inheriting our PCN APIs.• Mobile Agents which carry problems will travel around

MyP2P network to find hosts with redundant computing power.

• Jobs carried by MAs will split and be dispatched dynamically.

• Randomly picking nodes balances load all over the network.


33

Randomized Job AssignmentRandomized Job Assignment


34

Our MEDIC! SystemOur MEDIC! System


35

2003 New Topics2003 New Topics


36

Grid Computing (2002 ~ )Grid Computing (2002 ~ )

• Grid computing• Large-scale resource sharing• Innovative applications• High-performance orientation

• Grid problem• Flexible, secure, coordinated resource

sharing among virtual organization

• Globus (www.globus.org)• Resource management• Data management and access• Application development environments• Information service• security

http://www.globus.org/


37

Grid Computing (2002 ~ )Grid Computing (2002 ~ )

• Grid & P2P architectures• Missions on grids involve large proporti

on of members while tasks on P2P are small and related only to few members

• Grids organize resources on different machines into huge computational powers

• P2P communities facilitate participants to operate respectively


38

Agent (2002 ~ )Agent (2002 ~ )

Agent

Encapsulated Computer System

flexible, autonomous actionEnvironment

Design Objectives


39

Agent (2002 ~ )Agent (2002 ~ )

Agent

Flexible

Well-defined boundaries and interfaces

Autonomous

problem-solving entities


40

Collaborative filtering (2002 ~ )Collaborative filtering (2002 ~ )

• Collaborative filters help people make choices based on the opinions of other people

• GroupLens : a system for collaborative filtering of netnews (cs.umn.edu)

• EachMovie : a system that recommends users with (research.compaq.com/SRC/eachmovie/)


41


D i s t r i b u t e d m o d e l s ‧ Gnutella‧ Cons & pros + scalable , fault tolerance - expensive global view‧ Research topics in distributed

environment

C e n t r a l i z e d m o d e l s • www.amazon.com• Cons & pros + easy to implement , global view - scale weakly , single point of failure - centralized resource consumption

amazon_dis.jpg


42



43

Reputation & trust(2002 ~ )Reputation & trust(2002 ~ )

• The reputation of an entity is an expectation of its behavior based on other entities

• The trust of an entity A about another entity B is that A believes that B will act as A expects within a specific context at a given time

• Applications• eBay ( we have a funny example here )• Web service

• Research• A simulation platform for different reputation models• Reputation models in distributed environments• Problems (such as collusions)

ebay_reputation.jpg


44

PublicationsPublications

• WWW 2003 (Poster), S. C. Cha and Y. J. Joung, On Derived Data Service in the Cyberspace.

• AAS 2003, C. J. Hsu and Y. J. Joung, An NS-based Bluetooth Topology Construction Simulation Environment.

• PET 2003, S. C. Cha and Y. J. Joung, From P3P to OPDL.

• LawTech 2002, S. C. Cha and Y. J. Joung, Online Personal Data Lisencing.


45

Awards & HonorsAwards & Honors

• NCHC 高速計算獎

• 劉宗原 : 國科會九十一年度碩士論文獎

• Tic100 科技創新比賽冬令營活動分組第二名


46

ReferencesReferences

• …

~~Q & A~~Thank You!~~Q & A~~Thank You!

We will graduate, yeah, baby, yeah…


48

Ad Hoc networkAd Hoc network

Back

(a) Client2 requests

LAS Client1 Client2

Broadcast with (1) client2’s identity (MAC address)(2) destination LAS’s IP address(3) request message

Ad hoc mode

(b) Client1 relays the request

LAS Client1 Client2

TCP connectionwith the request message

Reply desired data

Infrastructure mode

LAS Client1 Client2

Reply desired data back using client2’s identity

Ad hoc mode


49

模擬結果整合模擬結果整合

Back


50

藍芽拓撲藍芽拓撲

Back

(a) a piconet, (b) a scatternet

introduction to laboratory of dis, ntuim

Documents

cha introduction

introduction tolaboratory

p2p networks

peer networks

network topology formation

ntuimpersonal data licensing

year graduate students

phd students