functional programming in c# map, reduce, join part ii · 2 let f = x => array . from(x) ... in...

Aggr Group Query QQuiz Elem Misc Set Quant

Functional programming in C#Map, Reduce, Join — Part II

Radu NicolescuDepartment of Computer Science

University of Auckland

9 October 2018

1 / 40


1 Fold = Aggregate ≈ Reduce

2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

2 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

3 / 40


Fold = Aggregate ≈ Reduce

Definition (Fold)

Fold is a higher-order function that iterates a given function overan input sequence of elements and returns a cumulated result.

According to the platform, fold is also known as aggregate,accumulate, reduce, etc.

Typically, a fold function uses an internal accumulator, whichstarts with a given seed value and is successively modifiedaccording to the current sequence item.

4 / 40


Aggregate

In C# LINQ, this signature is given as:

1 s t a t i c U // output result2 Aggregate<T, U> (3 t h i s IEnumerab le<T> sequence , // input sequence4 U seed , // initial accumulator value5 Func<U, T, U> f u n c ) // cumulates result

5 / 40


Aggregate sample scenario

Often (but not always), the cumulated result is of a simple type,such as the integer sum of an integer sequence. Consider thefollowing simple snippet:

1 var a = new in t [ ] { 10 , 20 , . . . , 90 } ;2 var sum = 0 ;3 foreach ( i n t v i n a ) sum = sum + v ;

Can we solve the problem at a higher-level?

1 var a = new in t [ ] { 10 , 20 , . . . , 90 } ;2 var sum = a . Aggregate ( 0 , ( s , v ) => s+v ) ;

6 / 40


Aggregate

The full implementation of an universal Aggregate:

1 s t a t i c U Aggregate<T, U> (2 t h i s IEnumerab le<T> sequence ,3 U seed ,4 Func<U, T, U> f u n c ) {56 U acc = s e e d ; // initial accumulator value78 foreach (T item i n s e q u e n c e ) { //input sequence9 acc = f u n c ( acc , i tem ) ; //update accumulator

10 }1112 return acc ;13 }

7 / 40


Aggregate

• This Aggregate implementation is not an iterator: beforereturning, it effectively runs over the whole input sequence –it is neither deferred nor lazy

• Can Aggregate simulate Select? Yes, if we take the returntype U to be itself a sequence!

• However, the Aggregate-simulated Select will return a realsequence, not a virtual one (so, you can’t apply it tosemi-infinite sequences).

• The next slide suggests how to define such a simulation

8 / 40


Aggregate and Select

Select simulated by Aggregate:

1 var a = new in t [ ] { 10 , 20 , 30 } ;2 var m = a . S e l e c t ( v => v+v ) ;34 var f = a . Aggregate (5 new L i s t <int >() ,6 ( c , v ) => { c . Add ( v+v ) ; return c ; }7 ) ;

The two results, m and f, are identical

1 acc : {} ; {2 0} ; {2 0 , 4 0} ; {20 ,40 ,60} ≡ f ≡ m

9 / 40


Aggregate – builtin special cases

• Sum()

• Min()

• Max()

• Average()

10 / 40


LINQ Aggregate – F# fold, JS reduce (1+2)

• All start with an initial given accumulator value (here v)!

• F# fold s |> Seq.fold (fun a x −> ...) v

1 l e t myfo ld ( f : ’ a−>’b−>’a ) ( a : ’ a ) ( s : seq<’b>) =2 l e t myfo ld f a s =3 l e t mutable a ’ = a4 f o r x i n s do5 a ’ <− f a ’ x6 a ’

• JS reduce (1+2) s .reduce ((a,x) => ..., v)

1 funct ion myfo ld ( s , f , a ) {2 f o r ( l e t i =0; i<s . l e n g t h ; i ++) {3 a = f ( a , s [ i ] )4 }5 return a ;6 }

11 / 40


F# reduce, JS reduce (1+1)

• All start the first sequence value!

• F# reduce s |> Seq.reduce (fun a x −> ...)

1 l e t myreduce ( f : ’ a−>’a−>’a ) ( s : seq<’a>) =2 l e t myreduce f s =3 l e t mutable a ’ = Seq . head s4 f o r x i n Seq . s k i p 1 s do5 a ’ <− f a ’ x6 a ’

• JS reduce (1+1) s .reduce ((a,x) => ...)

1 funct ion myreduce ( s , f ) {2 l e t a = s [ 0 ]3 f o r ( l e t i =1; i<s . l e n g t h ; i ++) {4 a = f ( a , s [ i ] )5 }6 return a ;7 } 12 / 40


SelectMany/flatMap (re)defined on Aggregate/reduce

• (*) C# SelectMany (here called FlatMap) redefined onAggregate and Concat

1 s . FlatMap ( f ) ⇔ s . Aggregate (2 Enumerable . Empty<U>() , ( a , c)=>a . Concat ( f ( c ) ) )

1 var s = new [ ] { ” t he ” , ” q u i c k ” , ” brown ” , ” f o x ” , } ;2 var z = s . FlatMap ( x=>x ) ; // ” t he ”=>’t ’ , ’ h ’ , ’ e ’ ,3 // z = ’ t ’ , ’ h ’ , ’ e ’ , ’ q ’ , . . .

• (*) JS flatMap defined on reduce and concat

1 s . flatMap ( f ) ⇔ s . r e d u c e ( ( a , c)=>a . c o n c a t ( f ( c ) ) , [ ] )

1 l e t s = [ ” t he ” , ” q u i c k ” , ” brown ” , ” f o x ” , ]2 l e t f = x => Array . from ( x ) // ” t he ”=>’t ’ , ’ h ’ , ’ e ’ ,3 l e t z = s . f l a tMap ( f )4 l e t z = s . r e d u c e ( ( a , c ) => a . c o n c a t ( f ( c ) ) , [ ] )

13 / 40


Flat (re)defined

• (**) C# Flat defined on SelectMany and id

1 s . Flat ( ) ⇔ s . Se lectMany ( z => z )

• (**) F# flat defined on collect (aka flatMap) and id

1 s |> flat ⇔ s |> Seq . c o l l e c t ( i d ) // fun x −> x

• (**) JS flat defined on reduce and concat

1 s . flat ( ) ⇔ s . r e d u c e ( ( a , c ) => a . c o n c a t ( c ) )

14 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

15 / 40


GroupBy

1 // f o x e s : t he q u i c k brown f o x jumps o v e r th e l azy dog2 var g = f o x e s . GroupBy ( s => s . Length ) ;

16 / 40


GroupBy

Groupings are subsequences that implement the interfaceIGrouping<K,T>, i.e.

• a Key property of type K plus

• an IEnumerable<T>, which groups together all elements withthe same key

1 pub l i c i n t e r f a ce I G r o u p i n g<K, T>: IEnumerab le<T> {2 K Key { get ; } ;3 }

17 / 40


Comparative Summary

.NET LINQ (C#, F#) F# Seq F# Array JS ArrayAggregate (3) fold (3) fold (3) reduce (3)

- reduce (2) reduce (2) reduce (2)

GroupBy(2) groupBy (2) groupBy (2) -

• Aggregate (3), fold (3), and reduce (3) require an initialaccumulator value

• reduce (2) initializes the accumulator from the first sequenceitem

18 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

19 / 40


Query = Declarative = SQL-like = Comprehension Syntax

C# language extension – full query syntax charthttp://www.albahari.com/nutshell/linqsyntax.aspx

Contrast the query syntax with its underlying fluent method syntax

1 // f o x e s : t he q u i c k brown f o x jumps o v e r th e l azy dog23 var q = from s i n f o x e s4 where s . Length == 55 orderby s6 s e l e c t s . ToUpper ( ) ;78 var m = f o x e s9 . Where ( s => s . Length == 5)

10 . OrderBy ( s => s )11 . S e l e c t ( s => s . ToUpper ( ) ) ;

Results are identical: BROWN JUMPS QUICK

20 / 40

http://www.albahari.com/nutshell/linqsyntax.aspx


Query Syntax

• The fluent method syntax seems preferred by functionalprogramming people

• The query syntax seems preferred by people with databasesbackground

• The query syntax seems better suited in some complexscenarios

• But the query syntax is less powerful then the method syntax– many methods are NOT available, e.g. Take(n)

• Therefore, the two syntaxes are often mixed

1 from s i n s e q u e n c e . Take ( 1 0 )23 // SELECT TOP 10 FROM . . .

21 / 40


Query Syntax

• Important, the true semantics of the query syntax are oftenignored

• Cookbook recipes?

• In fact, the query syntax does NOT have any own semantics!

• The query syntax is first translated to method calls, regardlessof the object types (“Duck typing”)...

• ... only after this the complier attempts to resolve these calls,to find out what needs to be done

22 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

23 / 40


Query Syntax Quiz

What is the result?

1 i n t te n = 1 0 ;2 i n t twenty = 2 0 ;3 i n t t h i r t y = 3 0 ;45 i n t r e s u l t =6 from c i n t en7 where twenty8 s e l e c t t h i r t y ;

24 / 40


Query Syntax Quiz

Consider the following two extension methods, in a static class X:

1 pub l i c s t a t i c i n t2 Where ( t h i s i n t c , Func<int , int> f ) {3 return c ∗ f ( 0 ) ;4 }5 pub l i c s t a t i c i n t6 S e l e c t ( t h i s i n t c , Func<int , int> f ) {7 return c + f ( 0 ) ;8 }

Does this context help?

1 // ten =10 twenty=20 t h i r t y =302 i n t r e s u l t =3 from c i n t en4 where twenty5 s e l e c t t h i r t y ;

25 / 40


Query Syntax Quiz


Consider the translation to method calls... does this help?

1 // ten =10 twenty=20 t h i r t y =302 i n t r e s u l t =3 ten . Where ( t => twenty ) . S e l e c t ( t => t h i r t y ) ;

26 / 40


Query Syntax Quiz


Apply the pipeline, step-by-step

1 i n t temp = 1 0 . Where ( t => 2 0 ) ;2 // c = 1 0 ; f = t => 2 0 ; temp = 20034 i n t r e s u l t = temp . S e l e c t ( t => 3 0 ) ;5 // c = 2 0 0 ; f = t => 3 0 ; r e s u l t = 230

27 / 40


Lookup

1 // f o x e s : t he q u i c k brown f o x jumps o v e r th e l azy dog2 var h = f o x e s . ToLookup ( s => s . Length ) ;

28 / 40


Lookup

The results of ToLookup(...)

• are similar to the results of GroupBy(...)

• but is also an indexer

• it adds direct fast key-based access to the groupings (cf.indexes in databases)

1 var h4 = h [ 4 ] ;

29 / 40


Converting to concrete sequences – summary

• ToLookup(key)

• ToDictionary(key)

• ToList()

• ToArray()

30 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

31 / 40


Elements

• First(), FirstOrDefault()

• Last(), LastOrDefault()

• ElementAt(), ElementAtOrDefault()

• Single(), SingleOrDefault()

Caveat: exceptions may be raised in some cases, e.g. for First(),when the sequence is empty...

32 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

33 / 40


Equality and Concatenation

• Consider the following sequences

1 var seq1a =2 new s t r i ng [ ] { ” t he ” , ” q u i c k ” , ” brown ” , ” f o x ” } ;3 var seq1b =4 new L i s t <s t r ing>{” t he ” , ” q u i c k ” , ” brown ” , ” f o x ” } ;5 var seq2 =6 new [ ] {” jumps ” , ” o v e r ” , ” th e ” , ” l a z y ” , ” dog ” } ;

• Equality testing – contents based

1 var a = seq1a . SequenceEqua l ( seq1b ) ; // true

• Concatenation – contents based

1 var seq3 = seq1b . Concat ( seq2 ) ; // f o x e s

34 / 40


Zip

1 var seq1 = new L i s t <s t r ing>2 { ” t he ” , ” q u i c k ” , ” brown ” , ” f o x ” } ;34 var seq2 = new [ ]5 { ” jumps ” , ” o v e r ” , ” th e ” , ” l a z y ” , ” dog ” } ;67 var z = seq1 . Zip ( seq2 , ( s1 , s2 ) => new { s1 , s2 } ) ;

35 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

36 / 40


Distinct

Sample usage

1 // f o x e s : t he q u i c k brown f o x jumps o v e r the l azy dog23 var d i s t i n c t = f o x e s . D i s t i n c t ( ) ; // 2nd “the” removed

Implementation

1 pub l i c s t a t i c IEnumerab le<T>2 D i s t i n c t <T>( t h i s IEnumerab le<T> s o u r c e ) {3 var d i c t = new D i c t i o n a r y <T, object >() ;4 foreach (T e le me nt i n s o u r c e ) {5 i f ( ! d i c t . Conta insKey ( e l e me nt ) ) {6 d i c t . Add ( e lement , nu l l ) ;7 y i e l d return e l em en t ;8 }9 }

10 }

37 / 40


Except, Intersect, Union – set results (no dup)

1 s t r i ng [ ] s equence1 =2 { ” t he ” , ” q u i c k ” , ” brown ” , ” f o x ” } ;3 s t r i ng [ ] s equence2 =4 { ” jumps ” , ” o v e r ” , ” th e ” , ” l a z y ” , ” dog ” } ;5 var i n t e r s e c t i o n = sequence1 . I n t e r s e c t ( sequence2 ) ;6 var d i f f e r e n c e = sequence1 . Except ( sequence2 ) ;7 var un ion = sequence1 . Union ( sequence2 ) ;

38 / 40


Outline


2 Grouping

3 Query syntax

4 Query Syntax Quiz

5 Elements

6 Miscellanea

7 Set operations

8 Quantifiers

39 / 40


Quantifiers: All, Any, Contains

Sample usage

1 // f o x e s : t he q u i c k brown f o x jumps o v e r th e l azy dog23 bool a l l = f o x e s . A l l ( s => s . Length= =3); // f a l s e45 bool any = f o x e s . Any ( s => s . Length= =3); // true67 bool c o n t a i n s = f o x e s . C o n t a i n s ( ” f o x ” ) ; // true

40 / 40

functional programming in c# map, reduce, join part ii · 2 let f = x => array . from(x) ... in...

Documents