relations of ideas are matters of fact: a unified theory of theoretical unification

Relations of Ideas Relations of Ideas are are

Matters of Fact: Matters of Fact: A Unified Theory of A Unified Theory of

Theoretical UnificationTheoretical Unification

Kevin T. KellyDepartment of Philosophy

Carnegie Mellon [email protected]

Les relations d'idées Les relations d'idées sont sont

des questions de fait:des questions de fait:Une théorie unifiée d'unification Une théorie unifiée d'unification

théoriquethéorique

Kevin T. KellyDepartment of Philosophy

Carnegie Mellon [email protected]

I. Relations of Ideas I. Relations of Ideas and and

Matters of FactMatters of Fact

Table of OppositesTable of Opposites

Relations of IdeasRelations of Ideas Matters of factMatters of fact

AnalyticAnalytic SyntheticSynthetic

CertainCertain UncertainUncertain

A PrioriA Priori A PosterioriA Posteriori

Philosophy of MathPhilosophy of Math Philosophy of Philosophy of ScienceScience

Proofs and Proofs and algorithmsalgorithms ConfirmationConfirmation

Truth-findingTruth-finding ““Rationality”Rationality”

ComputabilityComputability ProbabilityProbability

ProgressProgress


AnalyticAnalytic SyntheticSynthetic







ProgressProgress







Old Wine in New Bottles?Old Wine in New Bottles?







Philosophy of Science 101Philosophy of Science 101

FormalEmpirical

Verifiable

UnverifiableThe sun The sun will will nevernever stop stop rising.rising.

The The given given number number is even.is even.

Wrong Twice!Wrong Twice!

FormalEmpirical

Verifiable



The The currentcurrentemerald emerald isisgreengreen..The given The given

computaticomputation will on will never never halt.halt.


Aristotelian ExcuseAristotelian Excuse

Rational Animal

Rational ?

Reason Observation

UnboundedexperienceCompleted analysis

??Always black?


Leibnizian DoubtsLeibnizian Doubts

Rational ?

Reason Observation

Unboundedexperience

Never white?Always black?

Unboundedanalysis

????

Synthetic A PrioriSynthetic A Priori A PosterioriA Posteriori

Bounded Kantian SynthesisBounded Kantian Synthesis

Pure intuition Observation

Unboundedexperience

3+2 = 5 ?

3 4 5

Never white?Always black?

boundedsynthesis

23

??


Unbounded Kantian SynthesisUnbounded Kantian Synthesis

Pure intuition Observation

Unboundedexperience

Never cross? Never white?Always black?

Unbounded Synthesis?

????


Kantian AntinomiesKantian Antinomies

Beyond all pure intuition Beyond all possible observation

Physicalcontinuum

Infinite?Dense? Never white?

Infinite?Dense?

Geometricalcontinuum

????

Formal ProblemsFormal Problems Empirical Empirical ProblemsProblems

GGöödel and Turingdel and Turing

Never halts?

Computer Observation

Unboundedreality

Unboundedcomputation

Input

n

Never white?Always black??? ??

Wrong First CutWrong First Cut

FormalEmpirical

Verifiable



The The currentcurrentemerald emerald isisgreen.green.The given The given


Right First CutRight First Cut

Formal Empirical

Verifiable



The The currentcurrentemerald emerald isisgreen.green.The given The given


Unified EpistemologyUnified Epistemology

For both For both formalformal and and empiricalempirical questions:questions:

Justification =Justification = truth-finding truth-finding performanceperformance

Find the truth in theFind the truth in the best best possible sense,possible sense, as as efficiently as possibleefficiently as possible. .

II.II. VerificationVerification

Verification as Verification as

Halting Halting with the Truthwith the Truth

Conclusion

Yes!

Maybe youwill change yourmind.

Verification as Verification as Halting Halting with the Truthwith the Truth

Conclusion



Conclusion

Bang!

Verification as Verification as Halting Halting with the Truthwith the Truth

Conclusion



Conclusion

VerifiabilityVerifiability

Conclusion true

Yes!

Conclusion false

Classical Problem of InductionClassical Problem of Induction

Always green?

I won’t present blue until he says yes!

Empirical demon

Empirical scientist

Always green?


Always green?

You lose if you never halt with “yes”!


Always green?


Yes!

Always green?


Always green?


Always green?

You loseanyway.


““Problem” of UncomputabilityProblem” of Uncomputability

Never halts?

Formal demon Formal scientist

Maybe I am in an infinite loop.

Or maybe I will halt later…

Similar to the Problem of Induction?Similar to the Problem of Induction?

?

Never halts?

Apparently Not SimilarApparently Not Similar

I see your program!

Never halts? 666


Now I can cleverly calculate a priori what you will do!

Never halts? 666

Never shoots?


Never halts? 21456666

I’m beaten!

666

chugchugchug

I’m beaten!

Never shoots?

But Maybe Similar After All…But Maybe Similar After All…

666Never halts?

666

Aha! His program isalso showing!

Aha! His program isalso showing!

2145623455

Never shoots?


666Never halts?

I won’t halt until he halts with “yes” in mya priori simulation ofhim looking at me!

666

I won’t halt until he haltswith “yes” in mya priori simulation ofhim looking at me!

2145623455

666


2145623455

Never shoots?


666Never halts?

I won’t halt until he halts with “yes” in mya priori simulation ofhim looking at me!

666


2145623455

666


2145623455

Self-reference by Kleene fixed point theorem

Never shoots?


666Never halts? 2145623455

I didn’t halt…

2145623455 666

666

I didn’t halt…

I didn’t halt…

Never shoots?


666Never halts? 2145623455

I still didn’t halt…

2145623455 666

666



Never shoots? 666Never halts? 2145623455

2145623455

I give up! You’ll never halt!

I give up! You’ll never halt!

666

666


Never shoots? 666Never halts?

Ban

g!

23455

23455

TheEnd

TheEnd



23455

23455



666

23455

Sucker!

23455


666

23455

666

Ban

g!

TheEnd


Never shoots?Never halts?

234556

66



234556

66

R.I.P.TM23455

Fooleda priori



Rice-Shapiro TheoremRice-Shapiro Theorem

Q

Whatever a computable cognitive scientist could verify about an arbitrary computer’s input-output behavior by formally analyzing its program…

cog-sci

666

Rice-Shapiro TheoremRice-Shapiro Theorem…could have been verified by a behaviorist computer who empirically studies only the computer’s input-output behavior.

Qcog-sci

666

AnalogyAnalogy

Universal Formal Universal Formal ProblemProblem

Universal Empirical Universal Empirical ProblemProblem

UnverifiableUnverifiable UnverifiableUnverifiable

Computable Computable demon can fool demon can fool

computablecomputable agent agent

Empirical demon Empirical demon can can

fool fool idealideal agent agent

Answer runs Answer runs beyond beyond formalformal

experienceexperience

Answer runs beyond Answer runs beyond empiricalempirical

experienceexperience

Apparent DisanalogyApparent Disanalogy


Universal Universal Empirical ProblemEmpirical Problem

Input Input givengiven Input Input never fully never fully givengiven

Response: Digested vs. GivenResponse: Digested vs. Given


Universal Universal Empirical ProblemEmpirical Problem

Input Input givengiven Input Input never fully never fully givengiven

Input never fully Input never fully digesteddigested

Input never fully Input never fully digesteddigested

. . .Infinitely slow-melting candy Infinitely long noodle

Another Apparent DisanalogyAnother Apparent Disanalogy

Formal ScienceFormal Science Empirical ScienceEmpirical Science

IncompletenessIncompleteness ErrorError

Focus on ConsistencyFocus on Consistency

Formal ScienceFormal Science Empirical ScienceEmpirical Science

IncompletenessIncompleteness ErrorError

Add axiomsAdd axioms Conjecture a theoryConjecture a theory

Who knows if they Who knows if they are consistent with are consistent with

low-level low-level combinatorics?combinatorics?

Who knows if it is Who knows if it is consistent with all consistent with all future empirical future empirical

data?data?

Assume they are Assume they are and keep checkingand keep checking

Assume it is and Assume it is and keep checkingkeep checking

ThesisThesisThe problem of induction and the problem of uncomputability are similar.

So the problems should be understood similarly.

•Philosophy of Science•“Problem” of induction demands a “solution”.•“Confirmation” and “Bayesian rationality”•No clear connection with truth.

•Philosophy of Math•Uncomputability is a fact, not a “problem”.•Logic and algorithms•Insistence on connection with truth.

A House DividedA House Divided

III.III. Beyond Beyond VerifiabilityVerifiability

Drop the Halting RequirementDrop the Halting Requirement

•Ought implies can.•If you can’t halt with the truth, find the truth in the best feasible sense.

Q

(W. James, H. Putnam, E.M. Gold, R. Jeroslow, P. Hájek)

Gun control!Second amendment!

Focus on Focus on QuestionsQuestions

B

Answers partition the set of possible inputs

A C D

Infinite inputFinite input

Convergence Without HaltingConvergence Without Halting

In the limit

With halting 1

0

Gradual

1

0

At most finitely many mind-changes

1

0

Ever-closer to full confidence

No possible opinion change after halting

Halts

SuccessSuccess

Domain over which convergence to the truth is required

BA C D

Otherwise, non-convergence to falsehood is required.

Success Over Binary PartitionsSuccess Over Binary Partitions

BA BA

BA

Verify Ainfalliblyin the limitgradually

Refute A

infalliblyin the limitgradually

Decide A


Catch-all

Success Over Countable PartitionsSuccess Over Countable Partitions

= 0 = 1 = 2 . . .

Compute partial

0 1 2 . . .

Theory selection

Dom(

Formalcase

Empiricalcase



RetractionsRetractions

1

0

•A retraction occurs when an answer is “taken back”.•Retractions are what halting is supposed to prevent.•When you can’t prevent all retractions, minimize them.•Halting condition, when feasible, is special case.

2 retractions

Retraction EfficiencyRetraction Efficiency

Retraction-efficiency = success in the limit under the least feasible retraction bound in each answer.

BA

5 6

Retraction EfficiencyRetraction Efficiency

BA BA

BA

Halting-verifiable =1-verifiable

1 0 0 1

0 0

Halting-refutable =1-refutable

Halting-decidable =0-decidable

Kantian Antinomies Kantian Antinomies RevisitedRevisited

Matter

Finitelydivisible

Infinitelydivisible

Upper Complexity BoundUpper Complexity Bound

I say infinitely divisible when you let me cut.

Finitelydivisible

Infinitelydivisible

Lim-ref Lim-ver

. . .

. . .

Cut allowed by demon

Answer produced by scientist

Lower Complexity BoundLower Complexity Bound

Finitelydivisible

Infinitelydivisible

Lim-ref-Lim-ver

Lim-ver-Lim-ref

I let you cut when you say finitely divisible.

Purely Formal AnaloguePurely Formal Analogue

666 2145623455

Finitedomain

Infinitedomain

Lim-ref-Lim-ver

Lim-ver-Lim-ref

Purely Formal AnaloguePurely Formal Analogue

666 2145623455

I halt on another input each time he says finitely divisible in my a priori simulation of him looking at my program.

Finitedomain

Infinitedomain

Lim-ref-Lim-ver

Lim-ver-Lim-ref

Mixed Example: PenroseMixed Example: Penrose

UncomputableComputable

Undergraduate subjectCognitive scientist

.

???


Cognitive scientist

You can verify human computabilityin the limit…

NCNCCCCCCC…

Undergraduate subject



Cognitive scientist

…but if humans are computers, then you are a computer, in which case you can’t verify in the limit that humans are computers. CNCNCCNCCN…

Undergraduate subject


Formal ComplexityFormal Complexity

. . .

ArithmeticalHierarchy

2145623455

n


. . .


2145623455

n

grad vergrad ref

lim ver lim ref

halt refhalt ver

lim decgrad dec

halt dec


. . .


n

3-ref3-ver

2-ver 2-ref

halt ref

halt-ver1-ver

1-dec

grad dec

0-dec

grad vergrad ref

Putnam’sn-trialpredicates

lim reflim ver

2-dec

halt-dec

halt-ref 1-ref

2145623455

Empirical ComplexityEmpirical Complexity

. . .

Borelhierarchy

3-ref3-ver

2-ver 2-ref

halt ref

halt-ver1-ver

1-dec

grad dec

0-dec

grad vergrad ref

Differencehierarchy

lim reflim ver

2-dec

halt-dec

halt-ref 1-ref

Joint ComplexityJoint Complexity

. . .

EffectiveBorel Hierarchy

3-ref3-ver

2-ver 2-ref

halt ref

halt-ver1-ver

1-dec

grad dec

0-dec

grad vergrad ref

Effectivedifferencehierarchy

lim reflim ver

2-dec

halt-dec

halt-ref 1-ref

2145623455

Wrong First Cut in Wrong First Cut in PhilosophyPhilosophy

??Empirical Joint Formal

2145623455. . . . . . . . .

Empirical Joint Formal2145623455

. . . . . . . . .

Right First Cut in PhilosophyRight First Cut in Philosophy

2145623455

How Complexity Inverts SkepticismHow Complexity Inverts Skepticism•Justification is best possible truth-finding performance.•To establish best possible performance, one must show that better senses of success are infeasible.•Therefore, skeptical arguments justify induction!

. . .

Best possibleperformance

Impossibleperformance

IV. Bounded IV. Bounded RationalityRationality

Bayesian

2145623455

To be empirically ignorant is human. To be logically ignorant is irrational. ?

Philosophy of Science 101Philosophy of Science 101•“Science works by deducing theoretical predictions and comparing them against the empirical data”.

TheoremProver

Empirical data n No

t m

t

“Scientific Method”

Q

What if Predictions are Non-What if Predictions are Non-Computable?Computable?

Empirical data n No

Qt


Poof!

Computable Refutation with Computable Refutation with Halting Can Still be Possible!Halting Can Still be Possible!

Empirical data n No

Qt


?

?

Non-computable PredictionsNon-computable Predictions

Everything but h

h

(with Oliver Schulte)

There exists predictive hypothesis {h} such that:

•h is infinitely non-computable;•But a computer can refute hypothesis {h} in the halting sense!

Inductive Complexity : Deductive Inductive Complexity : Deductive Complexity Complexity

:: :: Implicit Definability : Explicit DefinabilityImplicit Definability : Explicit Definability

Learning complexity of {h}= joint complexity of {h}.

{h}

Deductive complexity of {h}= formal complexity of h.

. . .

. . .

hTypical case

Singleton Non-Basis Singleton Non-Basis TheoremTheorem

{h} is 1 h is non-arithmetical

There exists h such that:

h is the mu-defined Skolem function for the Delta-2 implicit definition of arithmetical truth (Hilbert and Bernays).

{h}

. . .

. . .

h

Cor. 1: A Foolish Cor. 1: A Foolish ConsistencyConsistency

•Consistent computers can’t verify or refute {h} even gradually!•So computable truth-seekers should be inconsistent.

Q

BayesianComputabledemon

InconsistencyComputablescientist

Cor. 2: No FirewallCor. 2: No Firewall•No computer architecture that insulates theorem proving from future empirical data can gradually verify or refute {h}.

TheoremProver

Empirical data n No

Q

t m

t


Empirical data

Cor. 2: No FirewallCor. 2: No Firewall

TheoremProver

n

m

No

Q

“Improved Scientific Method”

•But a computer that has a time lag before noticing inconsistency can refute {h} with halting.

V. Unification UnifiedV. Unification Unified

Which Explanation is Right?Which Explanation is Right?

???

Ockham Says:Ockham Says:

Choose theSimplest!

But Maybe…But Maybe…

Gotcha!

PuzzlePuzzle

A reliable indicator must be A reliable indicator must be sensitivesensitive to what it indicates. to what it indicates.

simple

A reliable indicator must be A reliable indicator must be sensitivesensitive to what it indicates. to what it indicates.

complex

PuzzlePuzzle

PuzzlePuzzle

But Ockham’s razor But Ockham’s razor alwaysalways points points at simplicity.at simplicity.

simple

PuzzlePuzzle

But Ockham’s razor But Ockham’s razor alwaysalways points points at simplicity.at simplicity.

complex

PuzzlePuzzle

How can a broken compass help How can a broken compass help you find something unless you you find something unless you already know where it is?already know where it is?

complex

Bayesian ExplanationBayesian Explanation

Ignorance (over simple vs. complex)Ignorance (over simple vs. complex) + Ignorance (over parameter settings)+ Ignorance (over parameter settings) = Knowledge (that simple data can’t be = Knowledge (that simple data can’t be

produced by the complex theory).produced by the complex theory).

Simple Complex

= The Old Paradox of = The Old Paradox of IndifferenceIndifference

Bayesian

Ignorance (over blue, non-blue)Ignorance (over blue, non-blue) + Ignorance (over ways of being non-+ Ignorance (over ways of being non-

blue)blue) = Knowledge (that the truth is blue as = Knowledge (that the truth is blue as

opposed to any other color)opposed to any other color)Not-BlueBlue

= The Old Paradox of = The Old Paradox of IndifferenceIndifference

Bayesian

Ignorance (over blue, non-blue)Ignorance (over blue, non-blue) + Ignorance (over ways of being non-+ Ignorance (over ways of being non-

blue)blue) = Knowledge (that the truth is blue as = Knowledge (that the truth is blue as

opposed to any other color)opposed to any other color)Not-BlueBlue

And how does he explain formal simplicity?

Better Idea: Retraction Better Idea: Retraction EfficiencyEfficiency

Truth

Truth

Better Idea: Retraction Better Idea: Retraction EfficiencyEfficiency

Curve FittingCurve Fitting

Data = arbitrarily precise intervals Data = arbitrarily precise intervals around around YY at specified values of at specified values of X.X.

Question: assuming that the truth is Question: assuming that the truth is polynomial, what is the degree?polynomial, what is the degree?


Demon shows flat line until Demon shows flat line until convergent method concludes flat.convergent method concludes flat.

Zero degree curve


Then switches to tilted line until Then switches to tilted line until convergent method concludes linear.convergent method concludes linear.

First degree curve


Then switches to parabola until Then switches to parabola until convergent method concludes convergent method concludes quadratic …quadratic …

Second degree curve

SimplicitySimplicity

ConstantLinear

Quadratic

Cubic

The current simplicity of answer H = the number of distinct answers the demon can force an arbitrary convergent method to produce before producing H.

Game-theoretic, topological invariant.

01

2

In Step with the DemonIn Step with the Demon

ConstantLinear

Quadratic

Cubic

There yet?Maybe.


ConstantLinear

Quadratic

Cubic

There yet?

Maybe.


ConstantLinear

Quadratic

Cubic

There yet?Maybe.

Ahead of the DemonAhead of the Demon

ConstantLinear

Quadratic

Cubic

There yet?Maybe.


ConstantLinear

Quadratic

Cubic

I know you’re coming!


ConstantLinear

Quadratic

Cubic

Maybe.


ConstantLinear

Quadratic

Cubic

!!!

Hmm, it’s quite nice here…


ConstantLinear

Quadratic

Cubic

You’re back!Learned your lesson?

Ockham Violator’s PathOckham Violator’s Path

ConstantLinear

Quadratic

Cubic

Ockham PathOckham Path

ConstantLinear

Quadratic

Cubic

So Ockham is uniquelyretraction efficient!

Same Argument Works ForSame Argument Works For•Fewer causes•Unified causes•More Conserved quantities•Hidden particles•Fewer free parameters

FormalFormal Curve Fitting Curve Fitting

MM is given the index of a computable is given the index of a computable polynomial function on the reals.polynomial function on the reals.

MM must converge to that function’s must converge to that function’s polynomial degree.polynomial degree.

Finite stage of effectivedecimal expansion

M

““Computational Computational Experience”Experience”

MM alsoalso has to refute each polynomial has to refute each polynomial degree with a special halting state degree with a special halting state for that degree. for that degree.

Finite stage of effectivedecimal expansion

M

Definitely not quadratic!

Halting state 2

Formal Curve-Fitting DemonFormal Curve-Fitting Demon

Polynomial degree?

For j = k to n-1:act like a polynomial of degree j until:

M refutes degree k-1 and M concludes that I am a polynomial of degree j;

set j = j+1;act like a polynomial of degree n.

21456M

D(M, k, n)

Formal Ockham’s RazorFormal Ockham’s Razor

•M should not conclude that D(M, k, n) has degree > k when M’s experience has refuted at most degree k.

•If M is Ockham, M retracts at most n - j times on D(M, k, n) on experience refuting degree < k.

•Else, M retracts at least n – j + 1 times. Q

Formal Causal InferenceFormal Causal Inference

•Given an index of a computable joint probability distribution whose conditional independence relations are represented by a causal network, compute the equivalence class of causal networks representing the distribution.

•N-choose-2 retractions required.•Empirical complexity = # of edges in graph•Ockham’s razor = assume no more causes than necessary!

X

YZ WCausal

network?

W

Formal Causal InferenceFormal Causal Inference

•Given an index of a computable joint probability distribution whose conditional independence relations are represented by a causal network, compute the equivalence class of causal networks representing the distribution.

•N-choose-2 retractions required.•Empirical complexity = # of edges in graph•Ockham’s razor = assume no more causes than necessary!

X

YZ W

W

Causal network?

Unified Account of Unified Account of UnificationUnification

•When halting is dropped as a condition for success, a penchant for elegance is truth-conducive (retraction-efficient) in both formal and empirical inquiry.

Q

DiscussionDiscussion

Q

1. Uncomputability is similar to the problem of induction.

2. In both cases, halting with the truth is infeasible.3. Without the halting requirement, skeptical arguments

justify inductive reasoning by establishing efficiency.4. Truth-finding efficiency explains Ockham’s razor

both in empirical and formal reasoning.5. Insistence on separation of formal and empirical

reasoning restricts the power of computable science.

THE ENDTHE END

relations of ideas are matters of fact: a unified theory of theoretical unification

Documents

given number

given computation

prioria posteriori gdel

currentemerald isgreen

verification verification

relations dides sont

fact leibnizian doubtsrational

best possible sense