Intro2016

Διαχειριςτισ Δικτφου: super-hero ι super-engineer?

Απο τθν απόγνωςθ ςτθν επίγνωςθ

Οι απαιτιςεισ ςτα ςφγχρονα δίκτυα

Tι είναι (ςε γενικζσ γραμμζσ) θ διαχείριςθ δικτφου

• Ο απομακρυςμζνοσ ζλεγχοσ και (ανα)διαμόρφωςθ δικτυακών ςυςκευών.

• Η παρακολοφκθςθ του δικτφου μζςω των ςυςκευών που το ςυνκζτουν.

• Η ςφνκεςθ βάςεων δεδομζνων με το “ιςτορικό” τθσ δραςτθριότθτασ του δικτφου.

• Η δυνατότθτα τοποκζτθςθσ παγίδων (traps) και ςυναγερμών (alarms) ςτισ δικτυακζσ ςυςκευζσ.

Η «εικόνα» ενόσ ςφγχρονου εργαλείου διαχείριςθσ δικτφου

Τι standards υπάρχουν • ISO: CMIP και CMIS. • IAB (Internet Architecture Board): SNMP,

SNMPv2, RMON, CMOT. • IEEE: CMOL. • Το SNMP επικράτθςε κυρίωσ λόγω τθσ

απλότθτασ ςτθν υλοποίθςθ και τθν διαχείριςθ. – 1988: SNMPv1. – 1993: SNMPv2 (updated το 1996). – 1998: SNMPv3 (draft από IESG: RFC 2570-2575).

Network Management standards

OSI CMIP

• Common Management Information Protocol

• designed 1980’s: the unifying net management standard

• too slowly standardized

SNMP: Simple Network Management Protocol

• Internet roots (SGMP)

• started simple

• deployed, adopted rapidly

• growth: size, complexity

• currently: SNMP V3

• de facto network management standard

Τι προςφζρει θ διαχείριςθ δικτφων

• Proactive: – remote configuration. – network profiling. – modeling changes to the network.

• Reactive: – ειδοποίθςθ για προβλιματα ςτο δίκτυο. – διάγνωςθ προβλθμάτων. – αυτόματθ διαμόρφωςθ του δικτφου ςε περίπτωςθ

ςφάλματοσ.

• Interactive: – interactive troubleshooting

Τι χρειάηεται ζνα ςφςτθμα διαχείριςθσ

• Network Management Console: – Ο ςτακμόσ εργαςίασ όπου παρακολουκεί ο διαχειριςτισ

τθν κατάςταςθ του δικτφου.

• Network Management Protocol: – Το πρωτόκολλο με το οποίο κα επικοινωνεί με τισ

δικτυακζσ ςυςκευζσ.

• Νetwork Management Agent: – Το software που εγκακίςταται ςτθν δικτυακι ςυςκευι για

χριςθ του πρωτοκόλλου διαχείριςθσ.

• Δικτυακζσ ςυςκευζσ που να τρζχουν agents. – router, switches, hubs, servers, applications.

The notion of network management….

• “autonomous” systems (aka “network”): 100s or 1000s of interacting hardware/software components

• other complex systems requiring monitoring, control:

– jet airplane

– nuclear power plant

– others?

"Network management includes the deployment, integration and coordination of the hardware, software, and human elements to monitor, test, poll, configure, analyze, evaluate, and control the network and element resources to meet the real-time, operational performance, and Quality of Service requirements at a reasonable cost."

Infrastructure for network management

agent data

agent data

agent data

agent data

managed device

managed device

managed device

managed device

managing entity

data

network management

protocol

definitions:

managed devices contain managed objects whose data is gathered into a Management Information Base (MIB)

managing entity

Οι προκλιςεισ ςιμερα • Ραγδαία εξζλιξθ των ευρυηωνικών δικτφων, του Ιντερνετ και των

υπθρεςιών κινθτών επικοινωνιών (UMTS, LTE, FI) • Αναδιαμορφώςιμα δίκτυα – γνωςιακά δίκτυα (reconfigurable –

cognitive networks) • Υποςτιριξθ προθγμζνων υπθρεςίων:

– ςε πολλοφσ τομείσ δραςτθριοτιτων του χριςτθ – αυξθμζνθ πολυπλoκότθτα – απαιτιςεισ διαχείριςθσ ςυνδεςιμότθτασ και πρόςβαςθσ ςε ετερογενι

δίκτυα – διαχείριςθ πολυμεςικοφ περιεχομζνου

• Autonomic communications – αυτόνομεσ επικοινωνίεσ • Διαχείριςθ πλθροφορίασ πλαιςίου περιβάλλοντοσ • Επζκταςθ των εμπλεκόμενων επιχειρθματικών φορζων και των

ρόλων τουσ

Heterogeneous Networks

Mobile/wireless

Mesh Relay ad-hoc

Sensors / actuators, Cooperating object networks

Personal space

Seamless service – content networks, new traffic requirements

Seamless service offering

Βαςικι αλυςίδα αξίασ επιχειρθματικοφ μοντζλου για παροχι υπθρεςιών

•Content provider

•Service/Application developer/provider

•Network Operator

•End-user

CP

SP

OP

EU

SLA-contract

SLA-contract

SLA-contract

Service Level Agreement

•Ενα service level agreement είναι ζνα κείμενο που κακορίηει τισ ςχζςεισ μεταξφ δφο μερών: του προμθκευτι και του καταναλωτι (the provider and the recipient).

•Προδιαγράφει το πλαίςιο ςυνεργαςίασ, αποτροπισ προβλθμάτων, εγγφθςθσ των προδιαγεγραμμζνων υπθρεςιών και τθσ ςχετικισ ποιότθτασ, το πλαίςιο επίλυςθσ διαφορών που μπορεί να προκφπτουν κατά τθ χριςθ των υπθρεςιών κ.α.

Service Level Agreement

•SLA should embrace a wide range of issues. Amongst these are usually the following:

•Services to be delivered

•Performance, Tracking and Reporting

•Problem Management

•Legal Compliance and Resolution of Disputes

•Customer Duties and Responsibilities

•Security

•IPR and Confidential Information

•Fees and expenses

•Termination

Future Internet vision

Source DoCoMo

Super 3G – LTE: the vision

Source DoCoMo

The notion of Future Internet management

• The design of Future Internet network elements with the aim of mastering the increasing complexity of communication networks

• The network should be capable of real‐time, secure and cost-effective delivery of data. It is of utmost importance to increase the user’s perceived quality of life anywhere and anytime. – human‐to‐human – human‐to‐machine – machine‐to‐machine

Network Management Basics

• Network management requirements

• OSI Management Functional Areas

– Network monitoring: performance, fault, accounting

– Network control: configuration, security

• Standardization in network management

• Practical issue: introduction to SNMP

Network Management Requirements

Example of approach

• Controlling strategic assets

• Controlling complexity

• Improving service

• Balancing various needs: performance, availability, security, cost

• Reducing downtime

• Controlling costs

What are we talking about?

• Network Management Tasks

– fault management

– configuration management

– performance management

– security management

– inventory management

– accounting management

Network Management OSI functional areas

• Fault management – Detect the fault

– Determine exactly where the fault is

– Isolate the rest of the network from the failure so that it can continue to function

– Reconfigure or modify the network in such a way as to minimize the impact

– Repair or replace the failed components

– Tests: connectivity, data integrity, response-time, ….

Fault Management

• detection

• exception alarm generation

• investigation and analysis

• statistics for steady state behaviour characterisation

Network Management OSI functional classification

• Performance management: – What is the level of capacity (χωρθτικότθτα) utilization?

– Is there excessive traffic?

– Has throughput been reduced to unacceptable levels?

– Are there bottlenecks?

– Is response time increasing?

– Indicators: availability, response time, accuracy service

throughput, utilization efficiency

Performance Management

• Availability and Reliability metrics

• Quality metrics

• real-time measurement

• historical analysis

Network Management OSI functional classification

• Configuration and Name Management: – Installation of new hardware/software

– Tracking changes in control configuration

– Who, what and why? - network topology

– Revert/undo changes

– Change management

– Configuration audit

– Does it do what was intended

Configuration Management

• installation of new hardware/software

• tracking changes in control configuration

– who, what and why!

• revert/undo changes

• change management

• configuration audit

– does it do what was intended?

Network Management OSI functional classification

• Security management

– Security services: generating, distributing, storing of encryption keys for services

– Exception alarm generation, detection of problems

– Uniform access control to resources

– Backups, data security

– Security logging

Security Management

• exception alarm generation

• detection

• uniform access controls to resources

• backup

Network Management - FCAPS OSI functional classification

• Accounting management

– Identifying consumers and suppliers of network resources - users and groups

– Mapping network resources consumption to customer identity

– Billing

Accounting Management

• identifying consumers and suppliers

– of network resources

• mapping network resources to customer identity

• charge back

– volumetric data

– time data

– date time of day

IP Route Management

• routing integrity (ακεραιότθτα δρομολόγθςθσ)

• consistency with customer requirements

• consistency with external peers

• conformance with imposed policy constraints

Problem Tracking

• reporting procedures (διαδικαςίεσ αναφορών)

• fault management (διαχείριςθ ςφαλμάτων)

• escalation and referral (κλιμάκωςθ και προςφυγι)

• historical data for component reliability analysis

Inventory Control (ζλεγχοσ απογραφισ δικτυακών ςτοιχείων)

• hardware

– components

– identity

– location

• software

– version control

Knowledge Based Management

• "expert" systems

• Modelling

– simulation

– routing

– configuration changes

Current networking set-up • One of the first things to learn about a machine attached to an IP

network is its IP address. Let’s suppose that a machine named tristan is connected to the main desktop network (192.168.99.0/24).

• The machine tristan is alive on IP 192.168.99.35 and has been properly configured by the system administrator. By examining the route and ifconfig output we can learn a good deal about the network to which tristan is connected

• The IP address active on tristan is 192.168.99.35. This means that any IP packets created by tristan will have a source address of 192.168.99.35. Similarly any packet received by tristan will have the destination address of 192.168.99.35. When creating an outbound packet tristan will set the destination address to the server's IP. This gives the remote host and the networking devices in between these hosts enough information to carry packets between the two devices.

Current networking set-up • Because tristan will advertise that it accepts packets with a

destination address of 192.168.99.35, any frames (packets) appearing on the Ethernet bound for 192.168.99.35 will reach tristan. The process of communicating the ownership of an IP address is called Address Resolution Protocol (ARP) (πρωτόκολλο επίλυςθσ διευκφνςεων). This IP address is a unique identifier for the machine on the network to which it is connected.

• Common traffic to and from machines today is unicast IP traffic. Unicast traffic is essentially a conversation between two hosts. Though there may be routers between them, the two hosts are carrying on a private conversation. Examples of common unicast traffic are protocols such as HTTP (web), SMTP (sending mail), POP3 (fetching mail),

Δομή πακέτων • Τφποσ υλικοφ (hardware type) Ζνασ αρικμόσ

προςδιορίηεται ςε κάκε πρωτόκολλο του ςτρώματοσ ςυνδζςμου, π.χ. 1 για το Ethernet, και γράφεται ςτο πεδίο αυτό.

• Τφποσ πρωτόκολλου (protocol type) Ζνασ αρικμόσ προςδιορίηεται ςε κάκε πρωτόκολλο, π.χ. 0x0800 για το IPv4, που αντιγράφεται ςτο πεδίο αυτό.

• Μζγεκοσ τφπου υλικοφ (hardware length) Μζγεκοσ ςε bytes τθσ διεφκυνςθσ υλικοφ, π.χ. 6 για διευκφνςεισ Ethernet.

• Μζγεκοσ τφπου πρωτόκολλου Μζγεκοσ ςε bytes τθσ διεφκυνςθσ λογικοφ τφπου, π.χ. 4 για διευκφνςεισ IPv4.

• Ενζργεια (operation) Κακορίηει τθν ενζργεια που εκτελεί ο αποςτολζασ: 1 για ερώτθμα και 2 για απάντθςθ.

Δομή πακέτων • Διεφκυνςθ υλικοφ αποςτολζα (sender hardware address) Διεφκυνςθ

υλικοφ του αποςτολζα. Το μζγεκοσ του πεδίου αυτοφ δεν είναι ςτακερό εξαρτάται από το υλικό που χρθςιμοποιείται.

• Διεφκυνςθ πρωτοκόλλου αποςτολζα (sender protocol address) Διεφκυνςθ πρωτοκόλλου του αποςτολζα. Το μζγεκοσ του πεδίου αυτοφ δεν είναι ςτακερό εξαρτάται από το πρωτόκολλο που χρθςιμοποιείται.

• Διεφκυνςθ υλικοφ παραλιπτθ (target hardware address) Διεφκυνςθ υλικοφ του τελικού παραλιπτθ. Το μζγεκοσ του πεδίου αυτοφ δεν είναι ςτακερό εξαρτάται από το υλικό που χρθςιμοποιείται. Εαν θ ενζργεια είναι ερώτθμα, το πεδίο αυτό είναι άγνωςτο και εξ οριςμοφ τιμι είναι 0. Διεφκυνςθ πρωτοκόλλου παραλιπτθ (target protocol address) Διεφκυνςθ πρωτοκόλλου του τελικού παραλιπτθ. Το μζγεκοσ του πεδίου αυτοφ δεν είναι ςτακερό ˙ εξαρτάται από το πρωτόκολλο που χρθςιμοποιείται.

Classless Inter-Domain Routing (CIDR)

• Classless Inter-Domain Routing (CIDR) is a method for allocating IP addresses and routing Internet Protocol packets. The Internet Engineering Task Force introduced CIDR in 1993 to replace the previous addressing architecture of classful network design in the Internet. Their goal was to slow the growth of routing tables on routers across the Internet, and to help slow the rapid exhaustion of IPv4 addresses. CIDR moves way from the traditional IP classes (Class A, Class B, Class C

• IP addresses are described as consisting of two groups of bits in the address: the most significant part is the network address which identifies a whole network or subnet and the least significant portion is the host identifier, which specifies a particular interface of a host on that network. This division is used as the basis of traffic routing between IP networks and for address allocation policies. Classful network design for IPv4 sized the network address as one or more 8-bit groups, resulting in the blocks of Class A, B, or C addresses. Classless Inter-Domain Routing allocates address space to Internet service providers and end users on any address bit boundary, instead of on 8-bit segments. In IPv6, however, the interface identifier has a fixed size of 64 bits by convention, and smaller subnets are never allocated to end users.

No single system will solve all your problems or meet all your requirements

Any Network Management package can only

complement effective and efficient operational procedures

Need to identify what is important to you and

your organization

Intro to cognitive (self-) management

Autonomic concepts

Situation awareness and decision making

Knowledge fusion

Cognitive network management

Cognitive Network Knowledge Tools

Big Data : Why Distributed Dimensionality

Reduction?

• Can handle very large datasets – processing billions of records cannot take place on a single device.

• If data are dispersed in a number of devices, it is resource consuming to transmit all information to one single node.

MapReduce Programming Paradigm

• Distributed data processing model

• Two phases: map & reduce

• Both phases have key – value pairs as input and output

(GR, 188) (DE, 187) (ES, 173) (DE, 191) (GR, 178) (GR, 165)

(DE, 193) (GR, 180) (ES, 156) (GR, 182) (GR, 167) (DE, 180)

(DE, 191) (ES, 173) (GR, 188)

(DE, 193) (ES, 156) (GR, 182)

(DE, [191, 193]) (ES, [173, 156]) (GR, [188, 182])

(DE, 193) (ES, 173) (GR, 188)

map

reduce

map

56

Apache Hadoop

•

Apache Hadoop

• Runs on commodity hardware

• Designed to handle very large files (Gigabytes, Terabytes)

• Block size 64 Mb (default)

• Optimized for fast access to the whole dataset, not the first row

• Not a good choice for many small files

• Does not support simultaneous writers in a file, nor modifications in a random spot of a file

Apache Hadoop – MapReduce

• As many map tasks as the number of blocks of a file (input splits)

• After map phase, the mappers output is sorted and grouped by key

• The number of reducers can be defined by the user – programmer

split 0 map

split 2

map

split 1

map

reduce part 0 merg

e

sort