Functional Programming

Colin ShawDec 13, 2016

What Are We Doing Here?

● What does the functional in functional programming mean

● Step through imperative vs. functional code

● Solving problems with functional programming

● Powerful features of functional languages

● Idioms used in functional languages

● Benefits of functional programming

How Is This Being Presented?

OCaml

● Expressive, concise syntax

● Fast native code compilation

● Commercial support from the sciences and finance

● F# is a derivative language under the CLR

What Does Functional Really Mean?

● Functions are applicative

● Functions are first-class

● Functions are higher-order

● There is no state (or side effects of state)

These are all related ideas!

What Does Functional Really Mean?

● Applicative: Allowing arbitrary application of functions

● First-class function: Function that can be used like any other variable

● Higher-order function: The returned value of a function is itself a function

Prelude To Benefits of Functional Programming

● Determinism: the evaluation graph is invariant, reproducible

● Provability: evaluation invariance facilitates provable execution

● Statelessness: the entire computation is in one large step

Hot research topic (coq, CertiKOS)

There are other advantages, such as code quality, concise abstractions, etc.

Introduction To Types

● In the function sin(x), x must be a float

● x has a type; it happens to be float

● sin(x) is represented by a signature bearing types

○ float → float

Function Currying I

It gets more interesting with functions having multiple arguments

f = x + y

The signature is actually

int → int → intThis is because functions are operations on one variable. Think about it - multiple argument functions are syntactic sugar for this in any language.

Function Currying IIThink about it more this way:

add x y = (add x) y

That is, adding two arguments involves a meta-function that adds a constant, which happens to be the first argument, to the second argument.

Now we have a signature:

(int → int) → int which is just int → int → int

add x is incomplete. It is a function returning a function, which is complete when provided with y.

This is really talking about the work of Alonzo Church and his lambda calculus, as well as the work of Haskell Curry.

Function Currying IIICurrying really isn’t so strange:

mov eax, 1 mov ebx, 2 add eax, ebx

mov ebx, 3 add eax, ebx

Machine code is curried. Remember all the recommendations about not have long argument lists? It is rooted in making better quality code since the underlying machine doesn’t support that abstraction. As you will see, currying in functional languages is not just a closer representation of the machine, it is also an extremely powerful way of abstracting concepts and writing cleaner code.

1 + 2 + 3

Function Currying Is Your FriendLet’s consider a case of adding 2 to a number y:

let add x y = x + y

What if we let a new function add2 be defined like:

let add2 y = add 2 y

y is redundant in the definition, so just:

let add2 = add 2

It is easier to see with prefix functions than infix

int → int → int = int → int → int

int → int = int → int

Our code has the ability to be very concise and expressive precisely because we have first-class and higher-order functions that allow for currying and partial function application!

If these are equivalent the signatures must be as well.

This is actually what the above will show because it will automatically get rid of y.

Code Comparison I

Imperative

int x, y;

x = 5;

y = x + 2;

Functional

let x = 5inlet y = x + 2

Variables require memory.

Variables have state,Physical parts of memory that can be mutated by other processes.

Let syntax just means where x is use 5 instead. It is not a variable, it is lexical scoping.

In other words y = 5 + 2 = 7.Again, it is not a variable, this is stateless syntax.

Code Comparison II

Imperative

int x, y;

x = 5;

int add(int x, int y) { return x + y;}

y = add(x, 2);

Functional

let x = 5inlet add x y = x + yin add x 2

Typical to see compile-time static typing.

Functions have arbitrary arity.

We aren’t even going to talk about weak typing and features of languages that tend to cause significant unexpected problems.

Here x is syntactically scoped to the let definition.

let-in defines logical blocks of syntactically relevant expressions.

Since this is syntax lexicon and not a sequence of imperative operations, we can infer all types based on their use and the known type for 5. This is Hindley-Milner in action!

More Advanced TypesLet’s construct disjunctive types of cats:

type cat = Persian | Tabby | Burmese

Or record types:

type kitty = {age : int; kind : cat}

Or even a polymorphic types with conjunctive and disjunctive aspects:

type ‘a lst = Empty | Cons of ‘a * ‘a lst

Any time we use one of theCat constructors we know itis of the cat type. These are exclusive (disjunctive) types.

Kitties have an age and are of a cat type. This is a record type. These are inclusive (conjunctive) types.

‘a is a polymorphic type. This is a self-referential linked list - either a tuple of a value of type ‘a and a list, or Empty.

Introduction To Matching

Type matching is critical, as we soon will see:

match c with| Persian -> ...| Tabby -> ...| Burmese -> ...

Looks a lot like disjunctive union type declaration because this example is a disjunctive union deconstructor.

Functional language programming is about creating a single function to evaluate by constructing and deconstructing things so that the entire operation is evaluated atomically.

Instead of building sequences of operations using state, we build one expression solving the entire problem.

How Versatile Are Lists?How general are lists?

● List of items (array)

● Annotated list of items (dictionary)

● Self-referential list of one type and list

● Self-referential list of two types and list

● List of steps in a computation (recursion)

We just saw a type declaration for a parametric polymorphic linked list!

Interesting, huh?

Trees and lists are very similar!

A Deeper Look At ListsLists are a tuple of a value and a self-referential type, or are Empty:

type ‘a lst = Empty | Cons of ‘a * ‘a lst

You can write a list like this:

let l = Cons(2, Cons(4, Cons(5, Empty)))

Common syntactic shorthand for Cons is ::, and for Empty is [ ]:

let l = 2::4::5::[ ]

Or even more simply:

let l = [2;4;5]Lists are so central to functional languages that there is ample built-in syntax for list operations.

Parts of a List

Suppose we have this list:

1::2::3::4::[ ]

It has a head and a tail:

1 :: 2::3::4::[ ] Head Tail

Deconstructing ListsLists are our basic means of making non-trivial programs. We need to be able to both construct them and deconstruct them. This is just matching:

match l with| h::t -> do something with the head and tail| [ ] -> there is no more list

Matching in OCaml is extremely robust. You can match almost anything. This is critical to effective, concise construction and deconstruction for building clean code.

Now What?Suppose you have a list of integers and you want to find their sum:

let rec sum x =match x with

| h::t -> h + sum t| [ ] -> 0

rec scopes the name sum to the function

If list is non-empty, add the head element to the sum of the remainder of the list (the tail).

If list is empty just return 0.

The code is just a recursive generator of more code!

What Does Execution Look Like?Suppose we execute sum [1;2;3]:

sum [1;2;3]1 + (sum [2;3])1 + (2 + sum [3])1 + (2 + (3 + sum [ ]))1 + (2 + (3 + 0))1 + (2 + 3)1 + 56

Deconstruction steps

Construction steps

But Wait, Isn’t That Doubling Up On Work?

There is an optimization that the sum example didn’t make: tail call optimization.

Here is how you can rewrite it to exhibit tail call recursion:

let rec sum x y =match x with

| h::t -> sum t (h + y)| [ ] -> y

Introduce passthrough variable

Rewrite recursion so that the final function in the recursion is the recursion itself

Return the passthrough variable

I know what you are thinking: “For a problem of length n I was expecting a solution of length n + k, not 2n + k.”

Now What Does Execution Look Like?We can now evaluate sum [1;2;3] 0 (initializing the passthrough):

sum [1;2;3] 0sum [2;3] 1sum [3] 3sum [ ] 66

Now we don’t have the reconstructive steps with the addition operation that was pushed on the call stack!

Did You Notice...that the order of the list is reversed in the tail recursive form?

For things like sum it isn’t a problem, but let’s implement rev_map:

let rec rev_map f x y =match x with

| h::t -> rev_map f t ((f h)::y)| [ ] -> y

Function to map

Here we are building a list as an output.

Hopefully now you are starting to see why first-class and higher-order functions help make code so elegant!

List ReversalIf we execute rev_map add2 [1;2;3] [ ] we will get [5;4;3]

It is common to make a tail recursive reverse function to accommodate this:

let rec rev x y =match x with

| h::t -> rev t (h::y)| [ ] -> y

Reversed, as expected

Pro Tip: You can generally tell whether a function is tail recursive or not by whether or not the result is reversed.

List Reversal IINow, if we need an efficient list in the right order, we can execute:

rev (rev_map add2 [1;2;3] [ ]) [ ]

Or, if you like, define map more simply:

let map f l = rev (rev_map f l [ ]) [ ] Now you can just map add2 [1;2;3] and it will work for all lengths efficiently.

Pro Tip: As a rule of thumb, if the list is >10,000 elements you should use tail call optimized list functions.

Let’s Just Go Ahead And Implement FoldFold is known as reduce in other languages. You can implement it like this:

let rec fold f a x =match x with

| h::t -> fold f (f a h) t| [ ] -> a

And could be invoked like:

fold (fun a e -> a + e) 0 [1;2;3]I wonder what this results in...

Anonymous function

Defining Sum With Fold

Now that we know about fold and currying, we could rewrite sum:

let sum = fold (+) 0

This syntax just means use + as the function rather than the application of it.

Now you are seeing some of the power of the expressiveness of a functional language!

Homework!

● Explain why fold has more utility than sum

● Explain why in practice there is fold_left and fold_right

● Meditate on the idea of generators

Congrats!

● rev● map● fold● iter

Now you are on your way to knowing how to implement the normal cannon of list functions in a functional language:

With these basic list operations, defining new types and a little imagination you can solve an extremely wide range of problems!

A Bit About ProvabilitySo far we have defined an algebra with our functional programming system:

● Basic units (types)● Elementary operations

○ Constructive (cons)○ Destructive (match)

● Well-defined recursion

The algebra generated by these allows us to write proofs about the execution of our code!

No Testing → Unit testing → Stochastic Testing, etc. → Provable Execution

Sometimes You Have To Call People Out

When someone says: “My favorite imperative language now is embracing the functional style by adding map …”

You can now say: “You are likely missing the point. Anyone can add map to your language; it is just a method on a collection. It may, however, help you avoid certain side effects. Functional programming, on the other hand, is an entirely different idea about how to imagine computation that helps make exceptionally elegant, clean code and reason about execution provability.”

Unfortunately, ideas historically related to functional languages — list functions, anonymous functions, lambda calculus and notation — have become hyped terms used in feature marketing.

Structuring CodeOCaml uses modules to organize code:

module Add = struct

let add x y = x + yend

Modules are containers (structures) for types and functions. In a more mathematical sense, modules are categories.

Unfortunately, modules didn’t work with the CLR, so F# does not have modules. However, it does have the ability to link with all of the other CLR software!

OCaml automatically generates modules from file name, so you always have modules, even if not explicitly defined.

ConstraintsModules can have a signature that defines function signatures and sets visibility:

module type AddSig = sig

val add : int -> int -> intend

In this case, anything external to the module would not be able to see anything other than add (supposing we had defined anything else).

Module signatures can be thought of somewhat like interfaces.

It Gets Way More Interesting

We can have parameterized signatures:

module type AddIn = sig

type tval add : t -> t -> t

end

module type AddOut = sig

type tval add3 : t -> t -> t -> t

end

Definite generic type t

Define functions with signatures of generic type

An input signature

An output signature

Functors IJust define the module as a mapping:

module MakeAdd (M : AddIn) : (AddOut with type t := M.t) = struct

open Mlet add3 x y z = add x (add y z)

end

Parameterized signature to build from

Signature to end up with using type parameter t

We are defining add3 constrained to a known signature, based on arbitrary types and the dependent implementation of add. add won’t be visible since it isn’t in the signature.

So we are exposed to the implementation of add that our signature claims we have.

Functors IINow just define a definite input module:

module AddInt = struct

type t = intlet add x y = x + y

end

And build new mapped modules:

module WeLoveAddingInts = MakeAdd(AddInt)

Modules are first-class too!

Define the concrete implementation

Wait, Why Was This Awesome?You could have made any input type complying with the signature:

module AddBool = struct

type t = boollet add x y = x && y

end

And stamped out similar results:

module WeLoveAddingBools = MakeAdd(AddBool)

Now you have a real module, built at compile time, having type generality but modular constraint, without the overhead that many modularization schemes (OO) have.

Putting the O in OCaml

● OCaml supports object oriented design

● People sometimes use it for the interface syntax

● Initializers can be handy (but aren’t that special)

● People often avoid it because functors are much more powerful

Sorry F#

A Note About this and self

In object oriented programming we have the pseudo-operators generally referred to as this or self that provide reference to the current object. When you view these as an identity operation (endofunctor) on a category, you see immediately how modules and functors are more powerful. Languages like OCaml support first-class and modules that have parameterizable signatures, whereas most object oriented languages only provide a pseudo-operator for the identity.

What If I Want That Imperative Feel?

Good question! It brings up an idiom commonly used in functional programming — the monad.

● Monads are another category theory construct that we can build using advanced languages.

● Monads are typically bind and return operations, though there are equivalent operations that are different.

HW: But why would I want an imperative feel?

Monads are used in a lot of functional languages, but more so in languages like Haskell.

Building a Simple Monad

Consider these functions:

let bind m f = f mlet return m = m

Perhaps we want our bind function to print a string monad:

let bind m f = let _ = Printf.printf “%s” m in flet print = return

Prototype of a monad: we pass bind a monad and a function, and it applies the function to the monad.

return lifts m to a monad. The identity is trivial, but other monads are more elaborate.

Here we print the monad m and then continue to f.

We will call return print instead — you will see why in a moment.

Building a Simple Monad II

bind is often used in practice as an infix operator:

let (>>) = bind

And here is how we can use it:

“Functional ” >>“programming ” >>“is ” >>“cool!” >>print

Because we have so much control of evaluation in a functional language, we can create our own syntactic conveniences that emulate imperative programming style!

This would output “Functional programming is cool!”

Sometimes these are called continuations because they facilitate continued execution of terminal code. Often used in I/O and other non-functional aspects of a program.

Many More Functional Topics

● Combinators

● Lazy types

● Metaprogramming

● Concurrency models

● Functional Reactive Programming

What Are Functional Languages Good For?

● Compilers

● Theorem provers

● Any program desiring provable execution

● Any program desiring elegant abstraction

● List, tree and graph problems

Financial and telecom industries

Scientific research

Problems where the normal functional operations are fitting for the problem.

The type system and matching are ideal for abstract syntax.

What Are Functional Languages Bad At?

● High performance math

● Algorithms operating on lists that do not behave sequentially

Heaps are inherently bad for cache performance and sequential throughput

Decimation in time FFTs are a good example — they split lists in half, so are cumbersome when the base operations are sequential on lists.

What Kind Of Performance To Expect

● Generally on par with tuned C if the problem is appropriate

● Up to 100x slower if it the problem is not appropriate (fast math)

In these cases you should write the tuned parts of your code in a different language.

What Degree of Code Reduction To Expect

10% - 20% as much code as C

This is a very important consideration since it impacts development time, affects maintenance and affects the number of bugs and testing!

In Recap — What Have We Learned

● What is functional programming

● Why functional programming is important

● How to implement basic list functions

● Code modularity and functors

● Idioms like monads

● Benefits and limitations of functional programming

The End

Questions? Comments? Concerns?