functional programming
TRANSCRIPT
What Are We Doing Here?
● What does the functional in functional programming mean
● Step through imperative vs. functional code
● Solving problems with functional programming
● Powerful features of functional languages
● Idioms used in functional languages
● Benefits of functional programming
How Is This Being Presented?
OCaml
● Expressive, concise syntax
● Fast native code compilation
● Commercial support from the sciences and finance
● F# is a derivative language under the CLR
What Does Functional Really Mean?
● Functions are applicative
● Functions are first-class
● Functions are higher-order
● There is no state (or side effects of state)
These are all related ideas!
What Does Functional Really Mean?
● Applicative: Allowing arbitrary application of functions
● First-class function: Function that can be used like any other variable
● Higher-order function: The returned value of a function is itself a function
Prelude To Benefits of Functional Programming
● Determinism: the evaluation graph is invariant, reproducible
● Provability: evaluation invariance facilitates provable execution
● Statelessness: the entire computation is in one large step
Hot research topic (coq, CertiKOS)
There are other advantages, such as code quality, concise abstractions, etc.
Introduction To Types
● In the function sin(x), x must be a float
● x has a type; it happens to be float
● sin(x) is represented by a signature bearing types
○ float → float
Function Currying I
It gets more interesting with functions having multiple arguments
f = x + y
The signature is actually
int → int → intThis is because functions are operations on one variable. Think about it - multiple argument functions are syntactic sugar for this in any language.
Function Currying IIThink about it more this way:
add x y = (add x) y
That is, adding two arguments involves a meta-function that adds a constant, which happens to be the first argument, to the second argument.
Now we have a signature:
(int → int) → int which is just int → int → int
add x is incomplete. It is a function returning a function, which is complete when provided with y.
This is really talking about the work of Alonzo Church and his lambda calculus, as well as the work of Haskell Curry.
Function Currying IIICurrying really isn’t so strange:
mov eax, 1 mov ebx, 2 add eax, ebx
mov ebx, 3 add eax, ebx
Machine code is curried. Remember all the recommendations about not have long argument lists? It is rooted in making better quality code since the underlying machine doesn’t support that abstraction. As you will see, currying in functional languages is not just a closer representation of the machine, it is also an extremely powerful way of abstracting concepts and writing cleaner code.
1 + 2 + 3
Function Currying Is Your FriendLet’s consider a case of adding 2 to a number y:
let add x y = x + y
What if we let a new function add2 be defined like:
let add2 y = add 2 y
y is redundant in the definition, so just:
let add2 = add 2
It is easier to see with prefix functions than infix
int → int → int = int → int → int
int → int = int → int
Our code has the ability to be very concise and expressive precisely because we have first-class and higher-order functions that allow for currying and partial function application!
If these are equivalent the signatures must be as well.
This is actually what the above will show because it will automatically get rid of y.
Code Comparison I
Imperative
int x, y;
x = 5;
y = x + 2;
Functional
let x = 5inlet y = x + 2
Variables require memory.
Variables have state,Physical parts of memory that can be mutated by other processes.
Let syntax just means where x is use 5 instead. It is not a variable, it is lexical scoping.
In other words y = 5 + 2 = 7.Again, it is not a variable, this is stateless syntax.
Code Comparison II
Imperative
int x, y;
x = 5;
int add(int x, int y) { return x + y;}
y = add(x, 2);
Functional
let x = 5inlet add x y = x + yin add x 2
Typical to see compile-time static typing.
Functions have arbitrary arity.
We aren’t even going to talk about weak typing and features of languages that tend to cause significant unexpected problems.
Here x is syntactically scoped to the let definition.
let-in defines logical blocks of syntactically relevant expressions.
Since this is syntax lexicon and not a sequence of imperative operations, we can infer all types based on their use and the known type for 5. This is Hindley-Milner in action!
More Advanced TypesLet’s construct disjunctive types of cats:
type cat = Persian | Tabby | Burmese
Or record types:
type kitty = {age : int; kind : cat}
Or even a polymorphic types with conjunctive and disjunctive aspects:
type ‘a lst = Empty | Cons of ‘a * ‘a lst
Any time we use one of theCat constructors we know itis of the cat type. These are exclusive (disjunctive) types.
Kitties have an age and are of a cat type. This is a record type. These are inclusive (conjunctive) types.
‘a is a polymorphic type. This is a self-referential linked list - either a tuple of a value of type ‘a and a list, or Empty.
Introduction To Matching
Type matching is critical, as we soon will see:
match c with| Persian -> ...| Tabby -> ...| Burmese -> ...
Looks a lot like disjunctive union type declaration because this example is a disjunctive union deconstructor.
Functional language programming is about creating a single function to evaluate by constructing and deconstructing things so that the entire operation is evaluated atomically.
Instead of building sequences of operations using state, we build one expression solving the entire problem.
How Versatile Are Lists?How general are lists?
● List of items (array)
● Annotated list of items (dictionary)
● Self-referential list of one type and list
● Self-referential list of two types and list
● List of steps in a computation (recursion)
We just saw a type declaration for a parametric polymorphic linked list!
Interesting, huh?
Trees and lists are very similar!
A Deeper Look At ListsLists are a tuple of a value and a self-referential type, or are Empty:
type ‘a lst = Empty | Cons of ‘a * ‘a lst
You can write a list like this:
let l = Cons(2, Cons(4, Cons(5, Empty)))
Common syntactic shorthand for Cons is ::, and for Empty is [ ]:
let l = 2::4::5::[ ]
Or even more simply:
let l = [2;4;5]Lists are so central to functional languages that there is ample built-in syntax for list operations.
Parts of a List
Suppose we have this list:
1::2::3::4::[ ]
It has a head and a tail:
1 :: 2::3::4::[ ] Head Tail
Deconstructing ListsLists are our basic means of making non-trivial programs. We need to be able to both construct them and deconstruct them. This is just matching:
match l with| h::t -> do something with the head and tail| [ ] -> there is no more list
Matching in OCaml is extremely robust. You can match almost anything. This is critical to effective, concise construction and deconstruction for building clean code.
Now What?Suppose you have a list of integers and you want to find their sum:
let rec sum x =match x with
| h::t -> h + sum t| [ ] -> 0
rec scopes the name sum to the function
If list is non-empty, add the head element to the sum of the remainder of the list (the tail).
If list is empty just return 0.
The code is just a recursive generator of more code!
What Does Execution Look Like?Suppose we execute sum [1;2;3]:
sum [1;2;3]1 + (sum [2;3])1 + (2 + sum [3])1 + (2 + (3 + sum [ ]))1 + (2 + (3 + 0))1 + (2 + 3)1 + 56
Deconstruction steps
Construction steps
But Wait, Isn’t That Doubling Up On Work?
There is an optimization that the sum example didn’t make: tail call optimization.
Here is how you can rewrite it to exhibit tail call recursion:
let rec sum x y =match x with
| h::t -> sum t (h + y)| [ ] -> y
Introduce passthrough variable
Rewrite recursion so that the final function in the recursion is the recursion itself
Return the passthrough variable
I know what you are thinking: “For a problem of length n I was expecting a solution of length n + k, not 2n + k.”
Now What Does Execution Look Like?We can now evaluate sum [1;2;3] 0 (initializing the passthrough):
sum [1;2;3] 0sum [2;3] 1sum [3] 3sum [ ] 66
Now we don’t have the reconstructive steps with the addition operation that was pushed on the call stack!
Did You Notice...that the order of the list is reversed in the tail recursive form?
For things like sum it isn’t a problem, but let’s implement rev_map:
let rec rev_map f x y =match x with
| h::t -> rev_map f t ((f h)::y)| [ ] -> y
Function to map
Here we are building a list as an output.
Hopefully now you are starting to see why first-class and higher-order functions help make code so elegant!
List ReversalIf we execute rev_map add2 [1;2;3] [ ] we will get [5;4;3]
It is common to make a tail recursive reverse function to accommodate this:
let rec rev x y =match x with
| h::t -> rev t (h::y)| [ ] -> y
Reversed, as expected
Pro Tip: You can generally tell whether a function is tail recursive or not by whether or not the result is reversed.
List Reversal IINow, if we need an efficient list in the right order, we can execute:
rev (rev_map add2 [1;2;3] [ ]) [ ]
Or, if you like, define map more simply:
let map f l = rev (rev_map f l [ ]) [ ] Now you can just map add2 [1;2;3] and it will work for all lengths efficiently.
Pro Tip: As a rule of thumb, if the list is >10,000 elements you should use tail call optimized list functions.
Let’s Just Go Ahead And Implement FoldFold is known as reduce in other languages. You can implement it like this:
let rec fold f a x =match x with
| h::t -> fold f (f a h) t| [ ] -> a
And could be invoked like:
fold (fun a e -> a + e) 0 [1;2;3]I wonder what this results in...
Anonymous function
Defining Sum With Fold
Now that we know about fold and currying, we could rewrite sum:
let sum = fold (+) 0
This syntax just means use + as the function rather than the application of it.
Now you are seeing some of the power of the expressiveness of a functional language!
Homework!
● Explain why fold has more utility than sum
● Explain why in practice there is fold_left and fold_right
● Meditate on the idea of generators
Congrats!
● rev● map● fold● iter
Now you are on your way to knowing how to implement the normal cannon of list functions in a functional language:
With these basic list operations, defining new types and a little imagination you can solve an extremely wide range of problems!
A Bit About ProvabilitySo far we have defined an algebra with our functional programming system:
● Basic units (types)● Elementary operations
○ Constructive (cons)○ Destructive (match)
● Well-defined recursion
The algebra generated by these allows us to write proofs about the execution of our code!
No Testing → Unit testing → Stochastic Testing, etc. → Provable Execution
Sometimes You Have To Call People Out
When someone says: “My favorite imperative language now is embracing the functional style by adding map …”
You can now say: “You are likely missing the point. Anyone can add map to your language; it is just a method on a collection. It may, however, help you avoid certain side effects. Functional programming, on the other hand, is an entirely different idea about how to imagine computation that helps make exceptionally elegant, clean code and reason about execution provability.”
Unfortunately, ideas historically related to functional languages — list functions, anonymous functions, lambda calculus and notation — have become hyped terms used in feature marketing.
Structuring CodeOCaml uses modules to organize code:
module Add = struct
let add x y = x + yend
Modules are containers (structures) for types and functions. In a more mathematical sense, modules are categories.
Unfortunately, modules didn’t work with the CLR, so F# does not have modules. However, it does have the ability to link with all of the other CLR software!
OCaml automatically generates modules from file name, so you always have modules, even if not explicitly defined.
ConstraintsModules can have a signature that defines function signatures and sets visibility:
module type AddSig = sig
val add : int -> int -> intend
In this case, anything external to the module would not be able to see anything other than add (supposing we had defined anything else).
Module signatures can be thought of somewhat like interfaces.
It Gets Way More Interesting
We can have parameterized signatures:
module type AddIn = sig
type tval add : t -> t -> t
end
module type AddOut = sig
type tval add3 : t -> t -> t -> t
end
Definite generic type t
Define functions with signatures of generic type
An input signature
An output signature
Functors IJust define the module as a mapping:
module MakeAdd (M : AddIn) : (AddOut with type t := M.t) = struct
open Mlet add3 x y z = add x (add y z)
end
Parameterized signature to build from
Signature to end up with using type parameter t
We are defining add3 constrained to a known signature, based on arbitrary types and the dependent implementation of add. add won’t be visible since it isn’t in the signature.
So we are exposed to the implementation of add that our signature claims we have.
Functors IINow just define a definite input module:
module AddInt = struct
type t = intlet add x y = x + y
end
And build new mapped modules:
module WeLoveAddingInts = MakeAdd(AddInt)
Modules are first-class too!
Define the concrete implementation
Wait, Why Was This Awesome?You could have made any input type complying with the signature:
module AddBool = struct
type t = boollet add x y = x && y
end
And stamped out similar results:
module WeLoveAddingBools = MakeAdd(AddBool)
Now you have a real module, built at compile time, having type generality but modular constraint, without the overhead that many modularization schemes (OO) have.
Putting the O in OCaml
● OCaml supports object oriented design
● People sometimes use it for the interface syntax
● Initializers can be handy (but aren’t that special)
● People often avoid it because functors are much more powerful
Sorry F#
A Note About this and self
In object oriented programming we have the pseudo-operators generally referred to as this or self that provide reference to the current object. When you view these as an identity operation (endofunctor) on a category, you see immediately how modules and functors are more powerful. Languages like OCaml support first-class and modules that have parameterizable signatures, whereas most object oriented languages only provide a pseudo-operator for the identity.
What If I Want That Imperative Feel?
Good question! It brings up an idiom commonly used in functional programming — the monad.
● Monads are another category theory construct that we can build using advanced languages.
● Monads are typically bind and return operations, though there are equivalent operations that are different.
HW: But why would I want an imperative feel?
Monads are used in a lot of functional languages, but more so in languages like Haskell.
Building a Simple Monad
Consider these functions:
let bind m f = f mlet return m = m
Perhaps we want our bind function to print a string monad:
let bind m f = let _ = Printf.printf “%s” m in flet print = return
Prototype of a monad: we pass bind a monad and a function, and it applies the function to the monad.
return lifts m to a monad. The identity is trivial, but other monads are more elaborate.
Here we print the monad m and then continue to f.
We will call return print instead — you will see why in a moment.
Building a Simple Monad II
bind is often used in practice as an infix operator:
let (>>) = bind
And here is how we can use it:
“Functional ” >>“programming ” >>“is ” >>“cool!” >>print
Because we have so much control of evaluation in a functional language, we can create our own syntactic conveniences that emulate imperative programming style!
This would output “Functional programming is cool!”
Sometimes these are called continuations because they facilitate continued execution of terminal code. Often used in I/O and other non-functional aspects of a program.
Many More Functional Topics
● Combinators
● Lazy types
● Metaprogramming
● Concurrency models
● Functional Reactive Programming
What Are Functional Languages Good For?
● Compilers
● Theorem provers
● Any program desiring provable execution
● Any program desiring elegant abstraction
● List, tree and graph problems
Financial and telecom industries
Scientific research
Problems where the normal functional operations are fitting for the problem.
The type system and matching are ideal for abstract syntax.
What Are Functional Languages Bad At?
● High performance math
● Algorithms operating on lists that do not behave sequentially
Heaps are inherently bad for cache performance and sequential throughput
Decimation in time FFTs are a good example — they split lists in half, so are cumbersome when the base operations are sequential on lists.
What Kind Of Performance To Expect
● Generally on par with tuned C if the problem is appropriate
● Up to 100x slower if it the problem is not appropriate (fast math)
In these cases you should write the tuned parts of your code in a different language.
What Degree of Code Reduction To Expect
10% - 20% as much code as C
This is a very important consideration since it impacts development time, affects maintenance and affects the number of bugs and testing!
In Recap — What Have We Learned
● What is functional programming
● Why functional programming is important
● How to implement basic list functions
● Code modularity and functors
● Idioms like monads
● Benefits and limitations of functional programming