A#1R Vers Perf A#1R

Introduction III

Radu NicolescuDepartment of Computer Science

University of Auckland

19 July 2018

1 / 25

A#1R Vers Perf A#1R

1 A#1R Short FAQ

2 FP LINQ versatility

3 Performance issues

4 Assignment #1R F#

2 / 25

A#1R Vers Perf A#1R

A#1R Short FAQ

• Read regex format? Please see the revised/extended handout

#2: @”\s+” or ”\\s+”

• Output file? Write to stdout, i.e. using C#:Console.WriteLine() , F#: printfn , node.js: console . log .

The markers will redirect such outputs to actual files:

1 jbon007 . exe b . t x t 7 > jbon007 . l o g 2> jbon007 . e r r23 jbon007 . exe b . t x t 7 1> jbon007 . l o g 2> jbon007 . e r r45 jbon007 . exe b . t x t 7 1> jbon007 . l o g 2>&1

PS. In Unix parlance, everything is a file, including stdout...

4 / 25

A#1R Vers Perf A#1R

A#1R Short FAQ

• Select first?

• Pipeline of fluent method chaining: LINQ-like high-ordermethods can be arranged in any order, as long as the outputon one is still compatible with the input of the next.

• Alternate query syntax: The order is relevant, but we haven’tdiscussed this yet .

5 / 25

A#1R Vers Perf A#1R

FP LINQ versatility

Essentially the high-level same code can be used for:

• Processing in memory sequences, e.g. arrays, lists, ...

• In LINQ, all sequences are covered by the super-interfaceIEnumerable

• Processing data base tables, views, ...

• SQL Server, Azure SQL, MySQL, NoSQL, MongoDB,CosmosDB, ...

• Processing web services

• SOAP based (XML)• REST based (XML/ATOM or JSON)

7 / 25

A#1R Vers Perf A#1R

Web Services REST

• REST = Representational State Transfer... What?

• REST = software architectural style that allows you to buildhighly scalable distributed systems

• REST API is simply an Application Programming Interfacethat adheres to the above principles

• How many APIs for Web programming? Almost 20,000registered!

• Sorry! We are only able to look at a few, such as: Web API,ODATA, ...

• OData is just one of the many ways to build a REST API –but quite high-level, close to SQL

• We use several languages and platforms, so hopefully conceptswill surface

8 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – Demo code

Look, ma, no SQL, no HTTP, no URL, ...

1 var r e s = Ord er s2 . OrderBy ( o => o . Order ID )3 . Take ( 5 )4 . S e l e c t ( o => new {5 o . OrderID ,6 o . Customer . CompanyName ,7 EmployeeName =8 o . Employee . F i rstName + ” ” + o . Employee . LastName ,9 } ) ;

1011 r e s . Dump( ” r e s ” ) ;

9 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – Demo code

Same result from both a local SQl database and from a remoteODATA service:

10 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – URL

Magic: FP, LINQ, Lambdas⇒ Auto-generated URL – GET request

1 ” h t t p : / / s e r v i c e s . odata . org / Northwind / Northwind . s v c /” +2 ” Ord er s ( ) ?” +3 ”$ o r d e r b y=OrderID &” +4 ”$top=5 &” +5 ”$expand=Customer , Employee &” +6 ”$ s e l e c t=OrderID , Customer /CompanyName , Employee / FirstName , Employee /LastName” ;

11 / 25

A#1R Vers Perf A#1R

FP LINQ versatility – SQL code

Magic: FP, LINQ, Lambdas ⇒ Auto-generated SQL code

1 −− Region Parameter s2 DECLARE @p0 NVarChar (1000) = ’ ’3 −− EndRegion4 SELECT [ t1 ] . [ Order ID ] , [ t2 ] . [ CompanyName ] , ( [ t3 ] . [ F i rstName ] + @p0 ) + [ t3 ] . [ LastName ] AS [ EmployeeName ]5 FROM (6 SELECT TOP ( 5 ) [ t0 ] . [ Order ID ] , [ t0 ] . [ CustomerID ] , [ t0 ] . [ EmployeeID ]7 FROM [ O rd er s ] AS [ t0 ]8 ORDER BY [ t0 ] . [ Order ID ]9 ) AS [ t1 ]

10 LEFT OUTER JOIN [ Customers ] AS [ t2 ] ON [ t2 ] . [ CustomerID ] = [ t1 ] . [ CustomerID ]11 LEFT OUTER JOIN [ Employees ] AS [ t3 ] ON [ t3 ] . [ EmployeeID ] = [ t1 ] . [ EmployeeID ]12 ORDER BY [ t1 ] . [ Order ID ]

12 / 25

A#1R Vers Perf A#1R

Data Parallelism – First, sequential versions

• Classical for loop: sum the “worked” results over a given array

1 long For ( long [ ] a ) {2 var s = 0L ;3 foreach ( var n i n a ) {4 s = s + Work ( n ) ;5 }6 return s ;7 }

• Equivalent (still) sequential C# LINQ:

1 long Seq ( long [ ] a ) {2 return3 a . S e l e c t ( n => Work ( n ) ) . Sum ( ) ;4 }

• Which one is faster?14 / 25

A#1R Vers Perf A#1R

Data Parallelism – Look, ma, no threads!

• Equivalent parallel C# LINQ:

1 long Par ( long [ ] a ) {2 return3 a . A s P a r a l l e l ( ) . S e l e c t ( n => Work ( n ) ) . Sum ( ) ;4 }

• Just insert . AsParallel () and the code will automatically scaleto use all available cores!

• No worries about semaphores, monitors, synchronised regions,locks, deadlocks, ...

• All this critical stuff is well encapsulated under the high-orderfunctions...

• How fast are all these?

15 / 25

A#1R Vers Perf A#1R

Data Parallelism – Look, ma, no threads!

• Linqpad results on a lab machine, with 4 cores – results areabout twice faster with optimised compiled code

1 . . . For d u r a t i o n : 15905 ms f o r ( ; ; )23 . . . Seq d u r a t i o n : 15802 ms LINQ45 . . . Par d u r a t i o n : 4104 ms LINQ parallel

• Your conclusions?

16 / 25

A#1R Vers Perf A#1R

JS – reputed very slow

• Classical for loop

1 funct ion For ( a ) {2 var s = 0 ;3 var l e n = a . l e n g t h ;4 f o r ( var i = 0 ; i < l e n ; i ++) {5 s = s + Work ( a [ i ] ) ;6 }7 return s ;8 }

• Equivalent sequential JS LINQ:

1 funct ion Seq ( a ) {2 var s =3 a . S e l e c t ( n => Work ( n ) ) . Sum ( ) ;4 return s ;5 }

17 / 25

A#1R Vers Perf A#1R

node.js performance

• Node.js results on the same lab machine, with 4 cores:

1 . . . For d u r a t i o n : 16601.501 ms f o r ( ; ; )23 . . . Seq d u r a t i o n : 16573.154 ms LINQ JS

• Well, JS is not for number crunching, but not too bad...

• Your conclusions?

18 / 25

A#1R Vers Perf A#1R

How to measure performance, the FP way

• Encapsulate all measurement details in one single genericfunction – once for all!

1 s t a t i c T Durat ion<T> ( s t r i ng cap , Func<T> f ) {2 GC . C o l l e c t ( ) ;3 C o n s o l e . W r i t e L i n e ( // NB: string interpolation4 $”\ r \n . . . { cap} s t a r t ” ) ;5 var t i m e r = new Stopwatch ( ) ;6 t i m e r . S t a r t ( ) ;7 var r e s = f ( ) ;8 t i m e r . Stop ( ) ;9 C o n s o l e . W r i t e L i n e (

10 $” . . . { cap} d u r a t i o n : ” +11 ”{ t i m e r . E l a p s e d M i l l i s e c o n d s } ms” ) ;12 return r e s ;13 }

19 / 25

A#1R Vers Perf A#1R

How to invoke Duration

• Problem: Duration’s parameter f is a generic function (ok!)...but one that takes no arguments (empty parameter list)

• How to call Duration to measure a function that needsarguments, such as Seq (a), etc?

• Solution: call Duration to measure a no-arg lambda that inturn calls the required function!

1 var f = D u r a t i o n ( ” For ” , ( ) => For ( a ) ) ;2 var s = D u r a t i o n ( ” Seq ” , ( ) => Seq ( a ) ) ;3 var p = D u r a t i o n ( ” Par ” , ( ) => Par ( a ) ) ;

• For example, given () => For (a), Duration will call(() => For (a)) (), and finally For (a)!

• Thus, courtesy of lambdas and closures, our Durationfunction is really general purpose and applicable in all cases!

20 / 25

A#1R Vers Perf A#1R

Assignment #1R F#

• Structure defined by indentation: Strict offside rules! Spacesnot tabs!

• Very powerful type inference: F# is statically/strongly typed,but most types are inferred!

• Usual exceptions: types of the C# API may still need to bespecified

• True pipelines |> (cf. Unix shell) – no method chainingsurrogates

• F# lambdas: fun a −> v cf. C#: a => v

• No function return: the last expression of the logical branch isthe return value

• Parentheses required only where really needed

22 / 25

A#1R Vers Perf A#1R

Assignment #1R F# – Command-line

• From Environment

1 l e t a r g s = Envi ronment . GetCommandLineArgs ( )

• If you want a main entry point – you chose its name

1 [<E n t r y P o i n t >]2 l e t main a r g s =3 . . .

• [<EntryPoint>] is an F# attribute

23 / 25

A#1R Vers Perf A#1R

Assignment #1R F# – Read

1 open System2 open System . IO3 open System . Text . R e g u l a r E x p r e s s i o n s45 t ry6 l e t fname = ” r e a d−t e s t . t x t ”7 l e t t e x t = F i l e . ReadAl lText fname8 l e t r g x = Regex @”\ s+”9 l e t words =

10 r g x . R e p l a c e ( t e x t , ” ” ) . Trim ( ) . S p l i t ( ’ ’ )1112 words |> Array . i t e r ( fun w −> p r i n t f n ”%s ” w)1314 with ex −>15 p r i n t f n ”∗∗∗ E r r o r : %s ” ex . Message16 Envi ronment . Ex i tCode <− 1

24 / 25

A#1R Vers Perf A#1R

Assignment #1R F# – Frequencies (LP)

1 l e t F r e q u e n c i e s ( words : l i s t <s t r ing >) =2 words3 |> Seq . groupBy ( fun w −> w. ToUpper ( ) )4 |> Seq . map ( fun ( k , s ) −> ( Seq . l e n g t h s , k ) )5 |> Seq . s o r t B y D e s c e n d i n g ( fun ( c , k ) −> c )6 // . . .78 l e t Main ( ) =9 l e t words = [ ”dd” ; ”DD” ; ” ccc ” ; . . . ]

10 l e t f r s = F r e q u e n c i e s words11 f r s . Dump ” f r s ”1213 Main ( )

• F#: powerful pattern matching, everywhere, even for thefunction parameters (see map, sortBy lambdas) –cf. more limited C#: deconstructions, JS: destructuring.

25 / 25