GENERAL OVERVIEW OF COMPUTER

The first counting device was the abacus, originally from Asia. It worked on a place-

value notion meaning that the place of a bead or rock on the apparatus determined how

much it was worth.

• 1600s : John Napier discovers logarithms.

Robert Bissaker invents the slide rule which will remain in popular use until 19??.

• 1642 : Blaise Pascal, a French mathematician and philosopher, invents the first

mechanical digital calculator using gears, called the Pascaline. Although this machine

could perform addition and subtraction on whole numbers, it was too expensive and only

Pascal himself could repare it.

• 1804 : Joseph Marie Jacquard used punch cards to automate a weaving loom.

• 1812 : Charles P. Babbage, the "father of the computer", discovered that many long

calculations involved many similar, repeated operations. Therefore, he designed a

machine, the difference engine which would be steam-powered, fully automatic and

commanded by a fixed instruction program. In 1833, Babbage quit working on this

machine to concentrate on the analytical engine.

• 1840s: Augusta Ada. "The first programmer" suggested that a binary system should be

used for storage rather than a decimal system.

• 1850s : George Boole developed Boolean logic which would later be used in the

design of computer circuitry.

• 1890: Dr. Herman Hollerith introduced the first electromechanical, punched-card

data-processing machine which was used to compile information for the 1890 U.S.

census. Hollerith's tabulator became so successful that he started his own business to

market it. His company would eventually become International Business Machines

(IBM).

• 1906 : The vacuum tube is invented by American physicist Lee De Forest.

• 1939 : Dr. John V. Atanasoff and his assistant Clifford Berry build the first

electronic digital computer. Their machine, the Atanasoff-Berry-Computer (ABC)

provided the foundation for the advances in electronic digital computers.

• 1941 : Konrad Zuse (recently deceased in January of 1996), from Germany,

introduced the first programmable computer designed to solve complex engineering

equations. This machine, called the Z3, was also the first to work on the binary system

instead of the decimal system.

• 1943 : British mathematician Alan Turing developed a hypothetical device, the

Turing machine which would be designed to perform logical operation and could read

and write. It would presage programmable computers. He also used vacuum technology

to build British Colossus, a machine used to counteract the German code scrambling

device, Enigma.

• 1944 : Howard Aiken, in collaboration with engineers from IBM, constructed a large

automatic digital sequence-controlled computer called the Harvard Mark I. This

computer could handle all four arithmetic opreations, and had special built-in programs

for logarithms and trigonometric functions.

• 1945 : Dr. John von Neumann presented a paper outlining the stored-program

concept.

• 1947 : The giant ENIAC (Electrical Numerical Integrator and Calculator) machine was

developed by John W. Mauchly and J. Presper Eckert, Jr. at the University of

Pennsylvania. It used 18, 000 vacuums, punch-card input, weighed thirty tons and

occupied a thirty-by-fifty-foot space. It wasn't programmable but was productive from

1946 to 1955 and was used to compute artillery firing tables. That same year, the

transistor was invented by William Shockley, John Bardeen and Walter Brattain of Bell

Labs. It would rid computers of vacuum tubes and radios.

• 1949 : Maurice V. Wilkes built the EDSAC (Electronic Delay Storage Automatic

Computer), the first stored-program computer. EDVAC (Electronic Discrete Variable

Automatic Computer), the second stored-program computer was built by Mauchly,

Eckert, and von Neumann. An Wang developped magnetic-core memory which Jay

Forrester would reorganize to be more efficient.

• 1950 : Turing built the ACE, considered by some to be the first programmable digital

computer.

The First Generation (1951-1959)

• 1951: Mauchly and Eckert built the UNIVAC I, the first computer designed and sold

commercially, specifically for business data-processing applications.

• 1950s : Dr. Grace Murray Hopper developed the UNIVAC I compiler.

• 1957 : The programming language FORTRAN (FORmula TRANslator) was designed

by John Backus, an IBM engineer.

• 1959 : Jack St. Clair Kilby and Robert Noyce of Texas Instruments manufactured the

first integrated circuit, or chip, which is a collection of tiny little transistors.

The Second Generation (1959-1965)

• 1960s : Gene Amdahl designed the IBM System/360 series of mainframe (G)

computers, the first general-purpose digital computers to use intergrated circuits.

• 1961: Dr. Hopper was instrumental in developing the COBOL (Common Business

Oriented Language) programming language.

• 1963 : Ken Olsen, founder of DEC, produced the PDP-I, the first minicomputer (G).

• 1965 : BASIC (Beginners All-purpose Symbolic Instruction Code) programming

language developped by Dr. Thomas Kurtz and Dr. John Kemeny.

The Third Generation (1965-1971)

• 1969 : The Internet is started.

• 1970 : Dr. Ted Hoff developed the famous Intel 4004 microprocessor (G) chip.

• 1971 : Intel released the first microprocessor, a specialized integrated circuit which was

ale to process four bits of data at a time. It also included its own arithmetic logic unit.

PASCAL, a structured programming language, was developed by Niklaus Wirth.

The Fourth Generation (1971-Present)

• 1975 : Ed Roberts, the "father of the microcomputer" designed the first

microcomputer, the Altair 8800, which was produced by Micro Instrumentation and

Telemetry Systems (MITS). The same year, two young hackers, William Gates and Paul

Allen approached MITS and promised to deliver a BASIC compiler. So they did and

from the sale, Microsoft was born.

• 1976 : Cray developed the Cray-I supercomputer (G). Apple Computer, Inc was

founded by Steven Jobs and Stephen Wozniak.

• 1977 : Jobs and Wozniak designed and built the first Apple II microcomputer.

• 1970 : 1980: IBM offers Bill Gates the opportunity to develop the operating system for

its new IBM personal computer. Microsoft has achieved tremendous growth and success

today due to the development of MS-DOS. Apple III was also released.

• 1981 : The IBM PC was introduced with a 16-bit microprocessor.

• 1982 : Time magazine chooses the computer instead of a person for its "Machine of the

Year."

• 1984 : Apple introduced the Macintosh computer, which incorporated a unique

graphical interface, making it easy to use. The same year, IBM released the 286-AT.

• 1986 : Compaq released the DeskPro 386 computer, the first to use the 80036

microprocessor.

• 1987 : IBM announced the OS/2 operating-system technology.

• 1988 : A nondestructive worm was introduced into the Internet network bringing

thousands of computers to a halt.

• 1989 : The Intel 486 became the world's first 1,000,000 transistor microprocessor.

• 1993s: The Energy Star program, endorsed by the Environmental Protection Agency

(EPA), encouraged manufacturers to build computer equipment that met power

consumpton guidelines. When guidelines are met, equipment displays the Energy Star

logo. The same year, Several companies introduced computer systems using the Pentium

microprocessor from Intel that contains 3.1 million transistors and is able to perform 112

million instructions per second (MIPS)

MODULE TWO

NATURAL LANGUAGE

2.0 PREAMBLE

In the philosophy of language, a natural language (or ordinary language) is any

language which arises in an unpremeditated fashion as the result of the innate facility for

language possessed by the human intellect. A natural language is typically used for

communication, and may be spoken, signed, or written. Natural language is distinguished

from constructed languages and formal languages such as computer-programming

languages or the "languages" used in the study of formal logic, especially mathematical

logic.

Defining natural language

Though the exact definition varies between scholars, natural language can broadly be

defined in contrast on the one hand to artificial or constructed languages, such as

computer programming languages like Python and international auxiliary languages

like Esperanto, and on the other hand to other communication systems in nature, such as

the waggle dance of bees. Although there are a variety of natural languages, any

cognitively normal human infant is able to learn any natural language. By comparing the

different natural languages, scholars hope to learn something about the nature of human

intelligence and the innate biases and constraints that shape natural language, which are

sometimes called universal grammar.

The term "natural language" refers only a language that has developed naturally and

hence to actual speech, rather than prescribed speech.

Hence, unstandardized speech such as African American Vernacular English) is natural,

whereas standardized speech such as Standard American English, which is in part

prescribed, is somewhat artificial.

2.1 ORIGINS OF NATURAL LANGUAGE

There is disagreement among anthropologists on when language was first used by

humans (or their ancestors). Estimates range from about two million (2,000,000) years

ago, during the time of Homo habilis, to as recently as forty thousand (40,000) years ago,

during the time of Cro-Magnon man. However recent evidence suggests modern human

language was invented or evolved in Africa prior to the dispersal of humans from Africa

around 50,000 years ago. Since all people including the most isolated indigenous groups

such as the Andamanese or the Tasmanian aboriginals possess language, then it was

presumably present in the ancestral populations in Africa before the human population

split into various groups to inhabit the rest of the world.

Controlled languages

Controlled natural languages are subsets of natural languages whose grammars and

dictionaries have been restricted in order to reduce or eliminate both ambiguity and

complexity (for instance, by cutting down on rarely used superlative or adverbial forms

or irregular verbs). The purpose behind the development and implementation of a

controlled natural language typically is to aid non-native speakers of a natural language

in understanding it, or to ease computer processing of a natural language. An example of

a widely used controlled natural language is Simplified English, which was originally

developed for aerospace industry maintenance manuals.

Modalities

Natural language manifests itself in modalities other than speech.

Sign languages

A sign language is a language which conveys meaning through visual rather than acoustic

patterns—simultaneously combining hand shapes, orientation and movement of the

hands, arms or body, and facial expressions to express a speaker's thoughts. Sign

languages are natural languages which have developed in Deaf communities, which can

include interpreters and friends and families of deaf people as well as people who are

deaf or hard of hearing themselves.

In contrast, a manually coded language (or signed oral language) is a constructed sign

system combining elements of a sign language and an oral language. For example,

Signed Exact English (SEE) did not develop naturally in any population, but was "created

by a committee of individuals".

Written languages

In a sense, written language should be distinguished from natural language. Until recently

in the developed world, it was common for many people to be fluent in spoken and yet

remain illiterate; this is still the case in poor countries today. Furthermore, natural

language acquisition during childhood is largely spontaneous, while literacy must usually

be intentionally acquired

2.2 NATURAL LANGUAGE PROCESSING

Natural language processing (NLP) is a field of computer science, artificial

intelligence, and linguistics concerned with the interactions between computers and

human (natural) languages. As such, NLP is related to the area of human–computer

interaction. Many challenges in NLP involve natural language understanding -- that is,

enabling computers to derive meaning from human or natural language input.

An automated online assistant providing customer service on a web page, an example of

an application where natural language processing is a major component.

2.2.1 History

The history of NLP generally starts in the 1950s, although work can be found from earlier

periods. In 1950, Alan Turing published his famous article "Computing Machinery and

Intelligence" which proposed what is now called the Turing test as a criterion of

intelligence. This criterion depends on the ability of a computer program to impersonate a

human in a real-time written conversation with a human judge, sufficiently well that the

judge is unable to distinguish reliably — on the basis of the conversational content

alone — between the program and a real human.

The Georgetown experiment in 1954 involved fully automatic translation of more than

sixty Russian sentences into English. The authors claimed that within three or five years,

machine translation would be a solved problem. However, real progress was much

slower, and after the ALPAC report in 1966, which found that ten years long research

had failed to fulfill the expectations, funding for machine translation was dramatically

reduced. Little further research in machine translation was conducted until the late 1980s,

when the first statistical machine translation systems were developed.

Some notably successful NLP systems developed in the 1960s were SHRDLU, a natural

language system working in restricted "blocks worlds" with restricted vocabularies, and

ELIZA, a simulation of a Rogerian psychotherapist, written by Joseph Weizenbaum

between 1964 to 1966. Using almost no information about human thought or emotion,

ELIZA sometimes provided a startlingly human-like interaction. When the "patient"

exceeded the very small knowledge base, ELIZA might provide a generic response, for

example, responding to "My head hurts" with "Why do you say your head hurts?".

During the 70's many programmers began to write 'conceptual ontologies', which

structured real-world information into computer-understandable data. Examples are

MARGIE (Schank, 1975), SAM (Cullingford, 1978), PAM (Wilensky, 1978), TaleSpin

(Meehan, 1976), QUALM (Lehnert, 1977), Politics (Carbonell, 1979), and Plot Units

(Lehnert 1981). During this time, many chatterbots were written including PARRY,

Racter, and Jabberwacky.

Up to the 1980s, most NLP systems were based on complex sets of hand-written rules.

Starting in the late 1980s, however, there was a revolution in NLP with the introduction

of machine learning algorithms for language processing. This was due both to the steady

increase in computational power resulting from Moore's Law and the gradual lessening of

the dominance of Chomskyan theories of linguistics (e.g. transformational grammar),

whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies

the machine-learning approach to language processing. Some of the earliest-used

machine learning algorithms, such as decision trees, produced systems of hard if-then

rules similar to existing hand-written rules. Increasingly, however, research has focused

on statistical models, which make soft, probabilistic decisions based on attaching real-

valued weights to the features making up the input data. The cache language models upon

which many speech recognition systems now rely are examples of such statistical models.

Such models are generally more robust when given unfamiliar input, especially input that

contains errors (as is very common for real-world data), and produce more reliable results

when integrated into a larger system comprising multiple subtasks.

Many of the notable early successes occurred in the field of machine translation, due

especially to work at IBM Research, where successively more complicated statistical

models were developed. These systems were able to take advantage of existing

multilingual textual corpora that had been produced by the Parliament of Canada and the

European Union as a result of laws calling for the translation of all governmental

proceedings into all official languages of the corresponding systems of government.

However, most other systems depended on corpora specifically developed for the tasks

implemented by these systems, which was (and often continues to be) a major limitation

in the success of these systems. As a result, a great deal of research has gone into

methods of more effectively learning from limited amounts of data.

Recent research has increasingly focused on unsupervised and semi-supervised learning

algorithms. Such algorithms are able to learn from data that has not been hand-annotated

with the desired answers, or using a combination of annotated and non-annotated data.

Generally, this task is much more difficult than supervised learning, and typically

produces less accurate results for a given amount of input data. However, there is an

enormous amount of non-annotated data available (including, among other things, the

entire content of the World Wide Web), which can often make up for the inferior results.

2.2.2 NLP using machine learning

Modern NLP algorithms are based on machine learning, especially statistical machine

learning. The paradigm of machine learning is different from that of most prior attempts

at language processing. Prior implementations of language-processing tasks typically

involved the direct hand coding of large sets of rules. The machine-learning paradigm

calls instead for using general learning algorithms — often, although not always,

grounded in statistical inference — to automatically learn such rules through the analysis

of large corpora of typical real-world examples. A corpus (plural, "corpora") is a set of

documents (or sometimes, individual sentences) that have been hand-annotated with the

correct values to be learned.

Many different classes of machine learning algorithms have been applied to NLP tasks.

These algorithms take as input a large set of "features" that are generated from the input

data. Some of the earliest-used algorithms, such as decision trees, produced systems of

hard if-then rules similar to the systems of hand-written rules that were then common.

Increasingly, however, research has focused on statistical models, which make soft,

probabilistic decisions based on attaching real-valued weights to each input feature. Such

models have the advantage that they can express the relative certainty of many different

possible answers rather than only one, producing more reliable results when such a model

is included as a component of a larger system.

Systems based on machine-learning algorithms have many advantages over hand-

produced rules:

The learning procedures used during machine learning automatically focus on the

most common cases, whereas when writing rules by hand it is often not obvious at

all where the effort should be directed.

Automatic learning procedures can make use of statistical inference algorithms to

produce models that are robust to unfamiliar input (e.g. containing words or

structures that have not been seen before) and to erroneous input (e.g. with

misspelled words or words accidentally omitted). Generally, handling such input

gracefully with hand-written rules — or more generally, creating systems of hand-

written rules that make soft decisions — is extremely difficult, error-prone and

time-consuming.

Systems based on automatically learning the rules can be made more accurate

simply by supplying more input data. However, systems based on hand-written

rules can only be made more accurate by increasing the complexity of the rules,

which is a much more difficult task. In particular, there is a limit to the complexity

of systems based on hand-crafted rules, beyond which the systems become more

and more unmanageable. However, creating more data to input to machine-

learning systems simply requires a corresponding increase in the number of man-

hours worked, generally without significant increases in the complexity of the

annotation process.

The subfield of NLP devoted to learning approaches is known as Natural Language

Learning (NLL) and its conference CoNLL and peak body SIGNLL are sponsored by

ACL, recognizing also their links with Computational Linguistics and Language

Acquisition. When the aims of computational language learning research is to understand

more about human language acquisition, or psycholinguistics, NLL overlaps into the

related field of Computational Psycholinguistics.

2.2.3 Major tasks in NLP

The following is a list of some of the most commonly researched tasks in NLP. Note that

some of these tasks have direct real-world applications, while others more commonly

serve as subtasks that are used to aid in solving larger tasks. What distinguishes these

tasks from other potential and actual NLP tasks is not only the volume of research

devoted to them but the fact that for each one there is typically a well-defined problem

setting, a standard metric for evaluating the task, standard corpora on which the task can

be evaluated, and competitions devoted to the specific task.

Automatic summarization: Produce a readable summary of a chunk of text. Often

used to provide summaries of text of a known type, such as articles in the financial

section of a newspaper.

Coreference resolution: Given a sentence or larger chunk of text, determine

which words ("mentions") refer to the same objects ("entities"). Anaphora

resolution is a specific example of this task, and is specifically concerned with

matching up pronouns with the nouns or names that they refer to. For example, in

a sentence such as "He entered John's house through the front door", "the front

door" is a referring expression and the bridging relationship to be identified is the

fact that the door being referred to is the front door of John's house (rather than of

some other structure that might also be referred to).

Discourse analysis: This rubric includes a number of related tasks. One task is

identifying the discourse structure of connected text, i.e. the nature of the

discourse relationships between sentences (e.g. elaboration, explanation, contrast).

Another possible task is recognizing and classifying the speech acts in a chunk of

text (e.g. yes-no question, content question, statement, assertion, etc.).

Machine translation: Automatically translate text from one human language to

another. This is one of the most difficult problems, and is a member of a class of

problems colloquially termed "AI-complete", i.e. requiring all of the different

types of knowledge that humans possess (grammar, semantics, facts about the real

world, etc.) in order to solve properly.

Morphological segmentation: Separate words into individual morphemes and

identify the class of the morphemes. The difficulty of this task depends greatly on

the complexity of the morphology (i.e. the structure of words) of the language

being considered. English has fairly simple morphology, especially inflectional

morphology, and thus it is often possible to ignore this task entirely and simply

model all possible forms of a word (e.g. "open, opens, opened, opening") as

separate words. In languages such as Turkish, however, such an approach is not

possible, as each dictionary entry has thousands of possible word forms.

Named entity recognition (NER): Given a stream of text, determine which items

in the text map to proper names, such as people or places, and what the type of

each such name is (e.g. person, location, organization). Note that, although

capitalization can aid in recognizing named entities in languages such as English,

this information cannot aid in determining the type of named entity, and in any

case is often inaccurate or insufficient. For example, the first word of a sentence is

also capitalized, and named entities often span several words, only some of which

are capitalized. Furthermore, many other languages in non-Western scripts (e.g.

Chinese or Arabic) do not have any capitalization at all, and even languages with

capitalization may not consistently use it to distinguish names. For example,

German capitalizes all nouns, regardless of whether they refer to names, and

French and Spanish do not capitalize names that serve as adjectives.

Natural language generation: Convert information from computer databases into

readable human language.

Natural language understanding: Convert chunks of text into more formal

representations such as first-order logic structures that are easier for computer

programs to manipulate. Natural language understanding involves the

identification of the intended semantic from the multiple possible semantics which

can be derived from a natural language expression which usually takes the form of

organized notations of natural languages concepts. Introduction and creation of

language metamodel and ontology are efficient however empirical solutions. An

explicit formalization of natural languages semantics without confusions with

implicit assumptions such as closed world assumption (CWA) vs. open world

assumption, or subjective Yes/No vs. objective True/False is expected for the

construction of a basis of semantics formalization.

Optical character recognition (OCR): Given an image representing printed text,

determine the corresponding text.

Part-of-speech tagging: Given a sentence, determine the part of speech for each

word. Many words, especially common ones, can serve as multiple parts of

speech. For example, "book" can be a noun ("the book on the table") or verb ("to

book a flight"); "set" can be a noun, verb or adjective; and "out" can be any of at

least five different parts of speech. Note that some languages have more such

ambiguity than others. Languages with little inflectional morphology, such as

English are particularly prone to such ambiguity. Chinese is prone to such

ambiguity because it is a tonal language during verbalization. Such inflection is

not readily conveyed via the entities employed within the orthography to convey

intended meaning.

Parsing: Determine the parse tree (grammatical analysis) of a given sentence. The

grammar for natural languages is ambiguous and typical sentences have multiple

possible analyses. In fact, perhaps surprisingly, for a typical sentence there may be

thousands of potential parses (most of which will seem completely nonsensical to

a human).

Question answering: Given a human-language question, determine its answer.

Typical questions have a specific right answer (such as "What is the capital of

Canada?"), but sometimes open-ended questions are also considered (such as

"What is the meaning of life?").

Relationship extraction: Given a chunk of text, identify the relationships among

named entities (e.g. who is the wife of whom).

Sentence breaking (also known as sentence boundary disambiguation): Given a

chunk of text, find the sentence boundaries. Sentence boundaries are often marked

by periods or other punctuation marks, but these same characters can serve other

purposes (e.g. marking abbreviations).

Sentiment analysis: Extract subjective information usually from a set of

documents, often using online reviews to determine "polarity" about specific

objects. It is especially useful for identifying trends of public opinion in the social

media, for the purpose of marketing.

Speech recognition: Given a sound clip of a person or people speaking, determine

the textual representation of the speech. This is the opposite of text to speech and

is one of the extremely difficult problems colloquially termed "AI-complete" (see

above). In natural speech there are hardly any pauses between successive words,

and thus speech segmentation is a necessary subtask of speech recognition (see

below). Note also that in most spoken languages, the sounds representing

successive letters blend into each other in a process termed coarticulation, so the

conversion of the analog signal to discrete characters can be a very difficult

process.

Speech segmentation: Given a sound clip of a person or people speaking, separate

it into words. A subtask of speech recognition and typically grouped with it.

Topic segmentation and recognition: Given a chunk of text, separate it into

segments each of which is devoted to a topic, and identify the topic of the

segment.

Word segmentation: Separate a chunk of continuous text into separate words. For

a language like English, this is fairly trivial, since words are usually separated by

spaces. However, some written languages like Chinese, Japanese and Thai do not

mark word boundaries in such a fashion, and in those languages text segmentation

is a significant task requiring knowledge of the vocabulary and morphology of

words in the language.

Word sense disambiguation: Many words have more than one meaning; we have

to select the meaning which makes the most sense in context. For this problem, we

are typically given a list of words and associated word senses, e.g. from a

dictionary or from an online resource such as WordNet.

In some cases, sets of related tasks are grouped into subfields of NLP that are often

considered separately from NLP as a whole. Examples include:

Information retrieval (IR): This is concerned with storing, searching and

retrieving information. It is a separate field within computer science (closer to

databases), but IR relies on some NLP methods (for example, stemming). Some

current research and applications seek to bridge the gap between IR and NLP.

Information extraction (IE): This is concerned in general with the extraction of

semantic information from text. This covers tasks such as named entity

recognition, Coreference resolution, relationship extraction, etc.

Speech processing: This covers speech recognition, text-to-speech and related

tasks.

Other tasks include:

Stemming

Text simplification

Text-to-speech

Text-proofing

Natural language search

Query expansion

Automated essay scoring

Truecasing

2.3 STATISTICAL NLP

Statistical natural-language processing uses stochastic, probabilistic and statistical

methods to resolve some of the difficulties discussed above, especially those which arise

because longer sentences are highly ambiguous when processed with realistic grammars,

yielding thousands or millions of possible analyses. Methods for disambiguation often

involve the use of corpora and Markov models. Statistical NLP comprises all quantitative

approaches to automated language processing, including probabilistic modeling,

information theory, and linear algebra. The technology for statistical NLP comes mainly

from machine learning and data mining, both of which are fields of artificial intelligence

that involve learning from data.

2.4 EVALUATION OF NATURAL LANGUAGE PROCESSING

The goal of NLP evaluation is to measure one or more qualities of an algorithm or

a system, in order to determine whether (or to what extent) the system answers the goals

of its designers, or meets the needs of its users. Research in NLP evaluation has received

considerable attention, because the definition of proper evaluation criteria is one way to

specify precisely an NLP problem, going thus beyond the vagueness of tasks defined only

as language understanding or language generation. A precise set of evaluation criteria,

which includes mainly evaluation data and evaluation metrics, enables several teams to

compare their solutions to a given NLP problem.

2.5 DIFFERENT TYPES OF EVALUATION

Depending on the evaluation procedures, a number of distinctions are traditionally

made in NLP evaluation.

Intrinsic vs. extrinsic evaluation

Intrinsic evaluation considers an isolated NLP system and characterizes its performance

mainly with respect to a gold standard result, pre-defined by the evaluators. Extrinsic

evaluation, also called evaluation in use considers the NLP system in a more complex

setting, either as an embedded system or serving a precise function for a human user.

The extrinsic performance of the system is then characterized in terms of its utility with

respect to the overall task of the complex system or the human user. For example,

consider a syntactic parser that is based on the output of some new part of speech (POS)

tagger. An intrinsic evaluation would run the POS tagger on some labelled data, and

compare the system output of the POS tagger to the gold standard (correct) output. An

extrinsic evaluation would run the parser with some other POS tagger, and then with the

new POS tagger, and compare the parsing accuracy.

Black-box vs. glass-box evaluation

Black-box evaluation requires one to run an NLP system on a given data set and to

measure a number of parameters related to the quality of the process (speed, reliability,

resource consumption) and, most importantly, to the quality of the result (e.g. the

accuracy of data annotation or the fidelity of a translation). Glass-box evaluation looks at

the design of the system, the algorithms that are implemented, the linguistic resources it

uses (e.g. vocabulary size), etc. Given the complexity of NLP problems, it is often

difficult to predict performance only on the basis of glass-box evaluation, but this type of

evaluation is more informative with respect to error analysis or future developments of a

system.

Automatic vs. Manual Evaluation

In many cases, automatic procedures can be defined to evaluate an NLP system by

comparing its output with the gold standard (or desired) one. Although the cost of

producing the gold standard can be quite high, automatic evaluation can be repeated as

often as needed without much additional costs (on the same input data). However, for

many NLP problems, the definition of a gold standard is a complex task, and can prove

impossible when inter-annotator agreement is insufficient. Manual evaluation is

performed by human judges, which are instructed to estimate the quality of a system, or

most often of a sample of its output, based on a number of criteria. Although, thanks to

their linguistic competence, human judges can be considered as the reference for a

number of language processing tasks, there is also considerable variation across their

ratings. This is why automatic evaluation is sometimes referred to as objective evaluation,

while the human kind appears to be more "subjective."

Standardization in NLP

An ISO sub-committee is working in order to ease interoperability between lexical

resources and NLP programs. The sub-committee is part of ISO/TC37 and is called

ISO/TC37/SC4. Some ISO standards are already published but most of them are under

construction, mainly on lexicon representation (see LMF), annotation and data category

registry.

MODULE THREE

(PROGRAMMING LANGUAGE – LISP)

3.0 PREAMBLE

Lisp (historically, LISP) is a family of computer programming languages with a

long history and a distinctive, fully parenthesized Polish prefix notation. Originally

specified in 1958, Lisp is the second-oldest high-level programming language in

widespread use today; only Fortran is older (by one year).

Like Fortran, Lisp has changed a great deal since its early days, and a number of dialects

have existed over its history. Today, the most widely known general-purpose Lisp

dialects are Common Lisp and Scheme.

Lisp was originally created as a practical mathematical notation for computer programs,

influenced by the notation of Alonzo Church's lambda calculus. It quickly became the

favored programming language for artificial intelligence (AI) research.

As one of the earliest programming languages, Lisp pioneered many ideas in computer

science, including tree data structures, automatic storage management, dynamic typing,

and the self-hosting compiler.

The name LISP derives from "LISt Processing". Linked lists are one of Lisp language's

major data structures, and Lisp source code is itself made up of lists. As a result, Lisp

programs can manipulate source code as a data structure, giving rise to the macro systems

that allow programmers to create new syntax or even new domain-specific languages

embedded in Lisp.

The interchangeability of code and data also gives Lisp its instantly recognizable syntax.

All program code is written as s-expressions, or parenthesized lists. A function call or

syntactic form is written as a list with the function or operator's name first, and the

arguments following; for instance, a function f that takes three arguments might be called

using (f arg1 arg2 arg3).

3.1 HISTORY

Lisp was invented by John McCarthy in 1958 while he was at the Massachusetts

Institute of Technology (MIT). McCarthy published its design in a paper in

Communications of the ACM in 1960, entitled "Recursive Functions of Symbolic

Expressions and Their Computation by Machine, Part I" ("Part II" was never published).

He showed that with a few simple operators and a notation for functions, one can build a

Turing-complete language for algorithms.

Information Processing Language was the first AI language, from 1955 or 1956, and

already included many of the concepts, such as list-processing and recursion, which came

to be used in Lisp.

McCarthy's original notation used bracketed "M-expressions" that would be translated

into S-expressions. As an example, the M-expression car[cons[A,B]] is equivalent to the

S-expression (car (cons A B)). Once Lisp was implemented, programmers rapidly chose

to use S-expressions, and M-expressions were abandoned. M-expressions surfaced again

with short-lived attempts of MLISP by Horace Enea and CGOL by Vaughan Pratt.

Lisp was first implemented by Steve Russell on an IBM 704 computer. Russell had read

McCarthy's paper, and realized (to McCarthy's surprise) that the Lisp eval function could

be implemented in machine code. The result was a working Lisp interpreter which could

be used to run Lisp programs, or more properly, 'evaluate Lisp expressions.'

Two assembly language macros for the IBM 704 became the primitive operations for

decomposing lists: car (Contents of the Address part of Register number) and cdr

(Contents of the Decrement part of Register number).

From the context, it is clear that the term "Register" is used here to mean "Memory

Register", nowadays called "Memory Location". Lisp dialects still use car and cdr (pron.:

/ˈkɑr/ and /ˈkʊdər/) for the operations that return the first item in a list and the rest of the

list respectively.

The first complete Lisp compiler, written in Lisp, was implemented in 1962 by Tim Hart

and Mike Levin at MIT. This compiler introduced the Lisp model of incremental

compilation, in which compiled and interpreted functions can intermix freely. The

language used in Hart and Levin's memo is much closer to modern Lisp style than

McCarthy's earlier code.

Lisp was a difficult system to implement with the compiler techniques and stock

hardware of the 1970s. Garbage collection routines, developed by then-MIT graduate

student Daniel Edwards, made it practical to run Lisp on general-purpose computing

systems, but efficiency was still a problem. This led to the creation of Lisp machines:

dedicated hardware for running Lisp environments and programs. Advances in both

computer hardware and compiler technology soon made Lisp machines obsolete.

During the 1980s and 1990s, a great effort was made to unify the work on new Lisp

dialects (mostly successors to Maclisp like ZetaLisp and NIL (New Implementation of

Lisp)) into a single language. The new language, Common Lisp, was somewhat

compatible with the dialects it replaced (the book Common Lisp the Language notes the

compatibility of various constructs).

3.1.1 Connection to artificial intelligence

Since its inception, Lisp was closely connected with the artificial intelligence research

community, especially on PDP-10 systems. Lisp was used as the implementation of the

programming language Micro Planner which was used in the famous AI system

SHRDLU. In the 1970s, as AI research spawned commercial offshoots, the performance

of existing Lisp systems became a growing issue.

3.1.2 Genealogy and variants

Over its fifty-year history, Lisp has spawned many variations on the core theme of an

S-expression language. Moreover, each given dialect may have several implementations

—for instance, there are more than a dozen implementations of Common Lisp.

Differences between dialects may be quite visible—for instance, Common Lisp uses the

keyword defun to name a function, but Scheme uses define. Within a dialect that is

standardized, however, conforming implementations support the same core language, but

with different extensions and libraries.

3.1.3 Historically significant dialects

LISP 1 – First implementation.

LISP 1.5 – First widely distributed version, developed by McCarthy and others

at MIT. So named because it contained several improvements on the original

"LISP 1" interpreter, but was not a major restructuring as the planned LISP 2

would be.

Stanford LISP 1.6 – This was a successor to LISP 1.5 developed at the

Stanford AI Lab, and widely distributed to PDP-10 systems running the TOPS-

10 operating system. It was rendered obsolete by Maclisp and InterLisp.

MACLISP – developed for MIT's Project MAC (no relation to Apple's

Macintosh, nor to McCarthy), direct descendant of LISP 1.5. It ran on the

PDP-10 and Multics systems. (MACLISP would later come to be called

Maclisp, and is often referred to as MacLisp.)

InterLisp – developed at BBN Technologies for PDP-10 systems running the

Tenex operating system, later adopted as a "West coast" Lisp for the Xerox

Lisp machines as InterLisp-D. A small version called "InterLISP 65" was

published for Atari's 6502-based computer line. For quite some time Maclisp

and InterLisp were strong competitors.

Franz Lisp – originally a Berkeley project; later developed by Franz Inc. The

name is a humorous deformation of the name "Franz Liszt", and does not refer

to Allegro Common Lisp, the dialect of Common Lisp sold by Franz Inc., in

more recent years.

XLISP, which AutoLISP was based on.

Standard Lisp and Portable Standard Lisp were widely used and ported,

especially with the Computer Algebra System REDUCE.

ZetaLisp, also known as Lisp Machine Lisp – used on the Lisp machines,

direct descendant of Maclisp. ZetaLisp had big influence on Common Lisp.

LeLisp is a French Lisp dialect. One of the first Interface Builders was written

in LeLisp.

Common Lisp (1984), as described by Common Lisp the Language – a

consolidation of several divergent attempts (ZetaLisp, Spice Lisp, NIL, and S-1

Lisp) to create successor dialects to Maclisp, with substantive influences from

the Scheme dialect as well. This version of Common Lisp was available for

wide-ranging platforms and was accepted by many as a de facto standard until

the publication of ANSI Common Lisp (ANSI X3.226-1994).

Dylan was in its first version a mix of Scheme with the Common Lisp Object

System.

EuLisp – attempt to develop a new efficient and cleaned-up Lisp.

ISLISP – attempt to develop a new efficient and cleaned-up Lisp. Standardized

as ISO/IEC 13816:1997 and later revised as ISO/IEC 13816:2007 Information

technology – Programming languages, their environments and system software

interfaces – Programming language ISLISP.

IEEE Scheme – IEEE standard, 1178–1990 (R1995)

ANSI Common Lisp – an American National Standards Institute (ANSI)

standard for Common Lisp, created by subcommittee X3J13, chartered[18] to

begin with Common Lisp: The Language as a base document and to work

through a public consensus process to find solutions to shared issues of

portability of programs and compatibility of Common Lisp implementations.

Although formally an ANSI standard, the implementation, sale, use, and

influence of ANSI Common Lisp has been and continues to be seen

worldwide.

ACL2 or "A Computational Logic for Applicative Common Lisp", an

applicative (side-effect free) variant of Common LISP. ACL2 is both a

programming language in which you can model computer systems and a tool to

help proving properties of those models.

3.1.4 Major dialects

The two major dialects of Lisp used for general-purpose programming today are

Common Lisp and Scheme. These languages represent significantly different design

choices.

Common Lisp is a successor to MacLisp. The primary influences were Lisp Machine

Lisp, MacLisp, NIL, S-1 Lisp, Spice Lisp, and Scheme.[25] It has many of the features of

Lisp Machine Lisp (a large Lisp dialect used to program Lisp Machines), but was

designed to be efficiently implementable on any personal computer or workstation.

Common Lisp has a large language standard including many built-in data types,

functions, macros and other language elements, as well as an object system (Common

Lisp Object System or shorter CLOS). Common Lisp also borrowed certain features from

Scheme such as lexical scoping and lexical closures.

Scheme (designed earlier) is a more minimalist design, with a much smaller set of

standard features but with certain implementation features (such as tail-call optimization

and full continuations) not necessarily found in Common Lisp.

Scheme is a statically scoped and properly tail-recursive dialect of the Lisp programming

language invented by Guy Lewis Steele Jr. and Gerald Jay Sussman. It was designed to

have exceptionally clear and simple semantics and few different ways to form

expressions. A wide variety of programming paradigms, including imperative, functional,

and message passing styles, find convenient expression in Scheme. Scheme continues to

evolve with a series of standards (Revisedn Report on the Algorithmic Language Scheme)

and a series of Scheme Requests for Implementation.

Clojure is a recent dialect of Lisp that principally targets the Java Virtual Machine, as

well as the CLR, the Python VM, the Ruby VM YARV, and compiling to JavaScript. It is

designed to be a pragmatic general-purpose language. Clojure draws considerable

influences from Haskell and places a very strong emphasis on immutability.

Clojure is a compiled language, as it compiles directly to JVM bytecode, yet remains

completely dynamic. Every feature supported by Clojure is supported at runtime. Clojure

provides access to Java frameworks and libraries, with optional type hints and type

inference, so that calls to Java can avoid reflection and enable fast primitive operations.

In addition, Lisp dialects are used as scripting languages in a number of applications,

with the most well-known being Emacs Lisp in the Emacs editor, AutoLisp and later

Visual Lisp in AutoCAD, Nyquist in Audacity. The small size of a minimal but useful

Scheme interpreter makes it particularly popular for embedded scripting. Examples

include SIOD and TinyScheme, both of which have been successfully embedded in the

GIMP image processor under the generic name "Script-fu".[27] LIBREP, a Lisp interpreter

by John Harper originally based on the Emacs Lisp language, has been embedded in the

Sawfish window manager.[28] The Guile interpreter is used in GnuCash. Within GCC, the

MELT plugin provides a Lisp-y dialect, translated into C, to extend the compiler by

coding additional passes (in MELT).

3.2 LANGUAGE INNOVATIONS

Lisp was the first homoiconic programming language: the primary representation

of program code is the same type of list structure that is also used for the main data

structures. As a result, Lisp functions can be manipulated, altered or even created within

a Lisp program without extensive parsing or manipulation of binary machine code. This

is generally considered one of the primary advantages of the language with regard to its

expressive power, and makes the language amenable to metacircular evaluation.

The ubiquitous if-then-else structure, now taken for granted as an essential element of any

programming language, was invented by McCarthy for use in Lisp, where it saw its first

appearance in a more general form (the cond structure). It was inherited by ALGOL,

which popularized it.

Lisp deeply influenced Alan Kay, the leader of the research on Smalltalk, and then in turn

Lisp was influenced by Smalltalk, by adopting object-oriented programming features

(classes, instances, etc.) in the late 1970s. The Flavours object system (later CLOS)

introduced multiple inheritance.

Lisp introduced the concept of automatic garbage collection, in which the system walks

the heap looking for unused memory. Most of the modern sophisticated garbage

collection algorithms such as generational garbage collection were developed for Lisp.

Largely because of its resource requirements with respect to early computing hardware

(including early microprocessors), Lisp did not become as popular outside of the AI

community as Fortran and the ALGOL-descended C language. Newer languages such as

Java and Python have incorporated some limited versions of some of the features of Lisp,

but are necessarily unable to bring the coherence and synergy of the full concepts found

in Lisp. Because of its suitability to complex and dynamic applications, Lisp is currently

enjoying some resurgence of popular interest.

3.3 SYNTAX AND SEMANTICS

Symbolic expressions (S-expressions)

Lisp is an expression-oriented language. Unlike most other languages, no

distinction is made between "expressions" and "statements"; all code and data are written

as expressions. When an expression is evaluated, it produces a value (in Common Lisp,

possibly multiple values), which then can be embedded into other expressions. Each

value can be any data type.

McCarthy's 1958 paper introduced two types of syntax: S-expressions (Symbolic

expressions, also called "sexps"), which mirror the internal representation of code and

data; and M-expressions (Meta Expressions), which express functions of S-expressions.

M-expressions never found favor, and almost all Lisps today use S-expressions to

manipulate both code and data.

The use of parentheses is Lisp's most immediately obvious difference from other

programming language families. As a result, students have long given Lisp nicknames

such as Lost In Stupid Parentheses, or Lots of Irritating Superfluous Parentheses.

However, the S-expression syntax is also responsible for much of Lisp's power: the

syntax is extremely regular, which facilitates manipulation by computer. However, the

syntax of Lisp is not limited to traditional parentheses notation. It can be extended to

include alternative notations. XMLisp, for instance, is a Common Lisp extension that

employs the metaobject-protocol to integrate S-expressions with the Extensible Markup

Language (XML).

The reliance on expressions gives the language great flexibility. Because Lisp functions

are themselves written as lists, they can be processed exactly like data. This allows easy

writing of programs which manipulate other programs (metaprogramming).

Many Lisp dialects exploit this feature using macro systems, which enables extension of

the language almost without limit.

3.4 LISTS

A Lisp list is written with its elements separated by whitespace, and surrounded by

parentheses. For example, (1 2 foo) is a list whose elements are three atoms: the values 1,

2, and foo. These values are implicitly typed: they are respectively two integers and a

Lisp-specific data type called a "symbolic atoms", and do not have to be declared as such.

The empty list () is also represented as the special atom nil. This is the only entity in Lisp

which is both an atom and a list.

Expressions are written as lists, using prefix notation. The first element in the list is the

name of a form, i.e., a function, operator, macro, or "special operator" (see below.) The

remainder of the list are the arguments. For example, the function list returns its

arguments as a list, so the expression

(list '1 '2 'foo)

evaluates to the list (1 2 foo). The "quote" before the arguments in the preceding example

is a "special operator" which prevents the quoted arguments from being evaluated (not

strictly necessary for the numbers, since 1 evaluates to 1, etc.). Any unquoted expressions

are recursively evaluated before the enclosing expression is evaluated. For example,

(list 1 2 (list 3 4))

evaluates to the list (1 2 (3 4)). Note that the third argument is a list; lists can be nested.

3.5 OPERATORS

Arithmetic operators are treated similarly. The expression

(+ 1 2 3 4)

evaluates to 10. The equivalent under infix notation would be "1 + 2 + 3 + 4". Arithmetic

operators in Lisp are variadic (or n-ary), able to take any number of arguments.

"Special operators" (sometimes called "special forms") provide Lisp's control structure.

For example, the special operator if takes three arguments. If the first argument is non-

nil, it evaluates to the second argument; otherwise, it evaluates to the third argument.

Thus, the expression

(if nil

(list 1 2 "foo")

(list 3 4 "bar"))

evaluates to (3 4 "bar"). Of course, this would be more useful if a non-trivial expression

had been substituted in place of nil.

3.5.1 Lambda expressions

Another special operator, lambda, is used to bind variables to values which are then

evaluated within an expression. This operator is also used to create functions: the

arguments to lambda are a list of arguments, and the expression or expressions to which

the function evaluates (the returned value is the value of the last expression that is

evaluated). The expression

(lambda (arg) (+ arg 1))

evaluates to a function that, when applied, takes one argument, binds it to arg and returns

the number one greater than that argument. Lambda expressions are treated no differently

from named functions; they are invoked the same way. Therefore, the expression

((lambda (arg) (+ arg 1)) 5)

evaluates to 6.

3.6 ATOMS

In the original LISP there were two fundamental data types: atoms and lists. A list

was a finite ordered sequence of elements, where each element is in itself either an atom

or a list, and an atom was a number or a symbol. A symbol was essentially a unique

named item, written as an Alphanumeric string in source code, and used either as a

variable name or as a data item in symbolic processing. For example, the list (FOO

(BAR 1) 2) contains three elements: the symbol FOO, the list (BAR 1), and the number

2.

The essential difference between atoms and lists was that atoms were immutable and

unique. Two atoms that appeared in different places in source code but were written in

exactly the same way represented the same object, whereas each list was a separate object

that could be altered independently of other lists and could be distinguished from other

lists by comparison operators.

As more data types were introduced in later Lisp dialects, and programming styles

evolved, the concept of an atom lost importance. Many dialects still retained the predicate

atom for legacy compatibility, defining it true for any object which is not a cons.

3.7 CONSES AND LISTS

Box-and-pointer diagram for the list (42 69 613)

A Lisp list is a singly linked list. Each cell of this list is called a cons (in Scheme, a pair),

and is composed of two pointers, called the car and cdr. These are equivalent to the data

and next fields discussed in the article linked list, respectively.

Of the many data structures that can be built out of cons cells, one of the most basic is

called a proper list. A proper list is either the special nil (empty list) symbol, or a cons in

which the car points to a datum (which may be another cons structure, such as a list), and

the cdr points to another proper list.

If a given cons is taken to be the head of a linked list, then its car points to the first

element of the list, and its cdr points to the rest of the list. For this reason, the car and cdr

functions are also called first and rest when referring to conses which are part of a linked

list (rather than, say, a tree).

Thus, a Lisp list is not an atomic object, as an instance of a container class in C++ or Java

would be. A list is nothing more than an aggregate of linked conses. A variable which

refers to a given list is simply a pointer to the first cons in the list. Traversal of a list can

be done by "cdring down" the list; that is, taking successive cdrs to visit each cons of the

list; or by using any of a number of higher-order functions to map a function over a list.

Because conses and lists are so universal in Lisp systems, it is a common misconception

that they are Lisp's only data structures. In fact, all but the most simplistic Lisps have

other data structures – such as vectors (arrays), hash tables, structures, and so forth.

3.7.1 S-expressions represent lists

Parenthesized S-expressions represent linked list structures. There are several ways to

represent the same list as an S-expression. A cons can be written in dotted-pair notation

as (a . b), where a is the car and b the cdr. A longer proper list might be written (a . (b .

(c . (d . nil)))) in dotted-pair notation. This is conventionally abbreviated as (a b c d) in

list notation. An improper list[30] may be written in a combination of the two – as (a b c .

d) for the list of three conses whose last cdr is d (i.e., the list (a . (b . (c . d))) in fully

specified form).

3.7.2 List-processing procedures

Lisp provides many built-in procedures for accessing and controlling lists. Lists can be

created directly with the list procedure, which takes any number of arguments, and

returns the list of these arguments.

(list 1 2 'a 3)

;Output: (1 2 a 3)

(list 1 '(2 3) 4)

;Output: (1 (2 3) 4)

Because of the way that lists are constructed from cons pairs, the [[cons]] procedure can

be used to add an element to the front of a list. Note that the cons procedure is

asymmetric in how it handles list arguments, because of how lists are constructed.

(cons 1 '(2 3))

;Output: (1 2 3)

(cons '(1 2) '(3 4))

;Output: ((1 2) 3 4)

The [[append]] procedure appends two (or more) lists to one another. Because Lisp lists

are linked lists, appending two lists has asymptotic time complexity

(append '(1 2) '(3 4))

;Output: (1 2 3 4)

(append '(1 2 3) '() '(a) '(5 6))

;Output: (1 2 3 a 5 6)

3.7.3 Shared structure

Lisp lists, being simple linked lists, can share structure with one another. That is to say,

two lists can have the same tail, or final sequence of conses. For instance, after the

execution of the following Common Lisp code:

(setf foo (list 'a 'b 'c))

(setf bar (cons 'x (cdr foo)))

the lists foo and bar are (a b c) and (x b c) respectively. However, the tail (b c) is the

same structure in both lists. It is not a copy; the cons cells pointing to b and c are in the

same memory locations for both lists.

Sharing structure rather than copying can give a dramatic performance improvement.

However, this technique can interact in undesired ways with functions that alter lists

passed to them as arguments. Altering one list, such as by replacing the c with a goose,

will affect the other:

(setf (third foo) 'goose)

This changes foo to (a b goose), but thereby also changes bar to (x b goose) – a possibly

unexpected result. This can be a source of bugs, and functions which alter their

arguments are documented as destructive for this very reason.

Aficionados of functional programming avoid destructive functions. In the Scheme

dialect, which favors the functional style, the names of destructive functions are marked

with a cautionary exclamation point, or "bang"—such as set-car! (read set car bang),

which replaces the car of a cons. In the Common Lisp dialect, destructive functions are

commonplace; the equivalent of set-car! is named rplaca for "replace car." This function

is rarely seen however as Common Lisp includes a special facility, setf, to make it easier

to define and use destructive functions. A frequent style in Common Lisp is to write code

functionally (without destructive calls) when prototyping, then to add destructive calls as

an optimization where it is safe to do so.

3.8 SELF-EVALUATING FORMS AND QUOTING

Lisp evaluates expressions which are entered by the user. Symbols and lists

evaluate to some other (usually, simpler) expression – for instance, a symbol evaluates to

the value of the variable it names; (+ 2 3) evaluates to 5. However, most other forms

evaluate to themselves: if you enter 5 into Lisp, it returns 5.

Any expression can also be marked to prevent it from being evaluated (as is necessary for

symbols and lists). This is the role of the quote special operator, or its abbreviation ' (a

single quotation mark). For instance, usually if you enter the symbol foo you will get

back the value of the corresponding variable (or an error, if there is no such variable). If

you wish to refer to the literal symbol, you enter (quote foo) or, usually, 'foo.

Both Common Lisp and Scheme also support the backquote operator (known as

quasiquote in Scheme), entered with the ` character. This is almost the same as the plain

quote, except it allows expressions to be evaluated and their values interpolated into a

quoted list with the comma and comma-at operators. If the variable snue has the value

(bar baz) then `(foo ,snue) evaluates to (foo (bar baz)), while `(foo ,@snue) evaluates to

(foo bar baz). The backquote is most frequently used in defining macro expansions.

Self-evaluating forms and quoted forms are Lisp's equivalent of literals. It may be

possible to modify the values of (mutable) literals in program code. For instance, if a

function returns a quoted form, and the code that calls the function modifies the form,

this may alter the behavior of the function on subsequent iterations.

(defun should-be-constant ()

'(one two three))

(let ((stuff (should-be-constant)))

(setf (third stuff) 'bizarre)) ; bad!

(should-be-constant) ; returns (one two bizarre)

Modifying a quoted form like this is generally considered bad style, and is defined by

ANSI Common Lisp as erroneous (resulting in "undefined" behavior in compiled files,

because the file-compiler can coalesce similar constants, put them in write-protected

memory, etc.).

Lisp's formalization of quotation has been noted by Douglas Hofstadter (in Gödel,

Escher, Bach) and others as an example of the philosophical idea of self-reference.

3.9 SCOPE AND CLOSURE

The modern Lisp family splits over the use of dynamic or static (aka lexical)

scope. Clojure, Common Lisp and Scheme make use of static scoping by default, while

Newlisp, Picolisp and the embedded languages in Emacs and AutoCAD use dynamic

scoping.

3.9.1 List structure of program code; exploitation by macros and compilers

A fundamental distinction between Lisp and other languages is that in Lisp, the textual

representation of a program is simply a human-readable description of the same internal

data structures (linked lists, symbols, number, characters, etc.) as would be used by the

underlying Lisp system.

Lisp uses this to implement a very powerful macro system. Like other macro languages

such as C, a macro returns code that can then be compiled. However, unlike C macros,

the macros are Lisp functions and so can exploit the full power of Lisp.

Further, because Lisp code has the same structure as lists, macros can be built with any of

the list-processing functions in the language. In short, anything that Lisp can do to a data

structure, Lisp macros can do to code. In contrast, in most other languages, the parser's

output is purely internal to the language implementation and cannot be manipulated by

the programmer.

This feature makes it easy to develop efficient languages within languages. For example,

the Common Lisp Object System can be implemented cleanly as a language extension

using macros. This means that if an application requires a different inheritance

mechanism, it can use a different object system. This is in stark contrast to most other

languages; for example, Java does not support multiple inheritance and there is no

reasonable way to add it.

In simplistic Lisp implementations, this list structure is directly interpreted to run the

program; a function is literally a piece of list structure which is traversed by the

interpreter in executing it. However, most substantial Lisp systems also include a

compiler. The compiler translates list structure into machine code or bytecode for

execution. This code can run as fast as code compiled in conventional languages such as

C.

Macros expand before the compilation step, and thus offer some interesting options. If a

program needs a precomputed table, then a macro might create the table at compile time,

so the compiler need only output the table and need not call code to create the table at run

time. Some Lisp implementations even have a mechanism, eval-when, that allows code to

be present during compile time (when a macro would need it), but not present in the

emitted module.

Evaluation and the read–eval–print loop

Lisp languages are frequently used with an interactive command line, which may be

combined with an integrated development environment. The user types in expressions at

the command line, or directs the IDE to transmit them to the Lisp system. Lisp reads the

entered expressions, evaluates them, and prints the result. For this reason, the Lisp

command line is called a "read–eval–print loop", or REPL.

The basic operation of the REPL is as follows. This is a simplistic description which

omits many elements of a real Lisp, such as quoting and macros.

The read function accepts textual S-expressions as input, and parses them into an internal

data structure. For instance, if you type the text (+ 1 2) at the prompt, read translates this

into a linked list with three elements: the symbol +, the number 1, and the number 2. It so

happens that this list is also a valid piece of Lisp code; that is, it can be evaluated. This is

because the car of the list names a function—the addition operation.

Note that a foo will be read as a single symbol. 123 will be read as the number 123. "123"

will be read as the string "123".

The eval function evaluates the data, returning zero or more other Lisp data as a result.

Evaluation does not have to mean interpretation; some Lisp systems compile every

expression to native machine code. It is simple, however, to describe evaluation as

interpretation: To evaluate a list whose car names a function, eval first evaluates each of

the arguments given in its cdr, then applies the function to the arguments. In this case, the

function is addition, and applying it to the argument list (1 2) yields the answer 3. This is

the result of the evaluation.

The symbol foo evaluates to the value of the symbol foo. Data like the string "123"

evaluates to the same string. The list (quote (1 2 3)) evaluates to the list (1 2 3).

It is the job of the print function to represent output to the user. For a simple result such

as 3 this is trivial. An expression which evaluated to a piece of list structure would

require that print traverse the list and print it out as an S-expression.

To implement a Lisp REPL, it is necessary only to implement these three functions and

an infinite-loop function. (Naturally, the implementation of eval will be complicated,

since it must also implement all special operators like if or lambda.) This done, a basic

REPL itself is but a single line of code: (loop (print (eval (read)))).

The Lisp REPL typically also provides input editing, an input history, error handling and

an interface to the debugger.

Lisp is usually evaluated eagerly. In Common Lisp, arguments are evaluated in

applicative order ('leftmost innermost'), while in Scheme order of arguments is undefined,

leaving room for optimization by a compiler.

3.10 CONTROL STRUCTURES

Lisp originally had very few control structures, but many more were added during

the language's evolution. (Lisp's original conditional operator, cond, is the precursor to

later if-then-else structures.)

Programmers in the Scheme dialect often express loops using tail recursion. Scheme's

commonality in academic computer science has led some students to believe that tail

recursion is the only, or the most common, way to write iterations in Lisp, but this is

incorrect. All frequently seen Lisp dialects have imperative-style iteration constructs,

from Scheme's do loop to Common Lisp's complex loop expressions. Moreover, the key

issue that makes this an objective rather than subjective matter is that Scheme makes

specific requirements for the handling of tail calls, and consequently the reason that the

use of tail recursion is generally encouraged for Scheme is that the practice is expressly

supported by the language definition itself. By contrast, ANSI Common Lisp does not

require the optimization commonly referred to as tail call elimination. Consequently, the

fact that tail recursive style as a casual replacement for the use of more traditional

iteration constructs (such as do, dolist or loop) is discouraged in Common Lisp is not just

a matter of stylistic preference, but potentially one of efficiency (since an apparent tail

call in Common Lisp may not compile as a simple jump) and program correctness (since

tail recursion may increase stack use in Common Lisp, risking stack overflow).

Some Lisp control structures are special operators, equivalent to other languages'

syntactic keywords. Expressions using these operators have the same surface appearance

as function calls, but differ in that the arguments are not necessarily evaluated—or, in the

case of an iteration expression, may be evaluated more than once.

In contrast to most other major programming languages, Lisp allows the programmer to

implement control structures using the language itself. Several control structures are

implemented as Lisp macros, and can even be macro-expanded by the programmer who

wants to know how they work.

Both Common Lisp and Scheme have operators for non-local control flow.

The differences in these operators are some of the deepest differences between the two

dialects. Scheme supports re-entrant continuations using the call/cc procedure, which

allows a program to save (and later restore) a particular place in execution. Common

Lisp does not support re-entrant continuations, but does support several ways of

handling escape continuations.

Frequently, the same algorithm can be expressed in Lisp in either an imperative or a

functional style. As noted above, Scheme tends to favor the functional style, using tail

recursion and continuations to express control flow. However, imperative style is still

quite possible. The style preferred by many Common Lisp programmers may seem more

familiar to programmers used to structured languages such as C, while that preferred by

Schemers more closely resembles pure-functional languages such as Haskell.

Because of Lisp's early heritage in list processing, it has a wide array of higher-order

functions relating to iteration over sequences. In many cases where an explicit loop

would be needed in other languages (like a for loop in C) in Lisp the same task can be

accomplished with a higher-order function. (The same is true of many functional

programming languages.)

A good example is a function which in Scheme is called map and in Common Lisp is

called mapcar. Given a function and one or more lists, mapcar applies the function

successively to the lists' elements in order, collecting the results in a new list:

(mapcar #'+ '(1 2 3 4 5) '(10 20 30 40 50))

This applies the + function to each corresponding pair of list elements, yielding the result

(11 22 33 44 55).

Examples

Here are examples of Common Lisp code.

The basic "Hello world" program:

(print "Hello world")

Lisp syntax lends itself naturally to recursion. Mathematical problems such as the

enumeration of recursively defined sets are simple to express in this notation.

Evaluate a number's factorial:

(defun factorial (n)

(if (<= n 1)

1

(* n (factorial (- n 1)))))

An alternative implementation, often faster than the previous version if the Lisp system

has tail recursion optimization:

(defun factorial (n &optional (acc 1))

(if (<= n 1)

acc

(factorial (- n 1) (* acc n))))

Contrast with an iterative version which uses Common Lisp's loop macro:

(defun factorial (n)

(loop for i from 1 to n

for fac = 1 then (* fac i)

finally (return fac)))

The following function reverses a list. (Lisp's built-in reverse function does the same

thing.)

(defun -reverse (list)

(let ((return-value '()))

(dolist (e list) (push e return-value))

return-value))

Object systems

Various object systems and models have been built on top of, alongside, or into Lisp,

including:

The Common Lisp Object System, CLOS, is an integral part of ANSI Common

Lisp. CLOS descended from New Flavors and Common LOOPS. ANSI Common

Lisp was the first standardized object-oriented programming language (1994,

ANSI X3J13).

ObjectLisp or Object Lisp, used by Lisp Machines Incorporated and early

versions of Macintosh Common Lisp

LOOPS (Lisp Object-Oriented Programming System) and the later

CommonLOOPS

Flavors, built at MIT, and its descendant New Flavors (developed by Symbolics).

KR (short for Knowledge Representation), a constraints-based object system

developed to aid the writing of Garnet, a GUI library for Common Lisp.

KEE used an object system called UNITS and integrated it with an inference

engine and a truth maintenance system (ATMS).

MODULE FOUR

(PROGRAMMING LANGUAGE – PROLOG)

4.0 PREAMBLE

Prolog is a general purpose logic programming language associated with artificial

intelligence and computational linguistics.

Prolog has its roots in first-order logic, a formal logic, and unlike many other

programming languages, Prolog is declarative: the program logic is expressed in terms of

relations, represented as facts and rules. A computation is initiated by running a query

over these relations.

The language was first conceived by a group around Alain Colmerauer in Marseille,

France, in the early 1970s and the first Prolog system was developed in 1972 by

Colmerauer with Philippe Roussel.

Prolog was one of the first logic programming languages, and remains the most popular

among such languages today, with many free and commercial implementations available.

While initially aimed at natural language processing, the language has since then

stretched far into other areas like theorem proving, expert systems games, automated

answering systems, ontologies and sophisticated control systems. Modern Prolog

environments support creating graphical user interfaces, as well as administrative and

networked applications.

4.1 SYNTAX AND SEMANTICS

In Prolog, program logic is expressed in terms of relations, and a computation is

initiated by running a query over these relations. Relations and queries are constructed

using Prolog's single data type, the term. Relations are defined by clauses. Given a query,

the Prolog engine attempts to find a resolution refutation of the negated query. If the

negated query can be refuted, i.e., an instantiation for all free variables is found that

makes the union of clauses and the singleton set consisting of the negated query false, it

follows that the original query, with the found instantiation applied, is a logical

consequence of the program. This makes Prolog (and other logic programming

languages) particularly useful for database, symbolic mathematics, and language parsing

applications. Because Prolog allows impure predicates, checking the truth value of

certain special predicates may have some deliberate side effect, such as printing a value

to the screen. Because of this, the programmer is permitted to use some amount of

conventional imperative programming when the logical paradigm is inconvenient. It has a

purely logical subset, called "pure Prolog", as well as a number of extralogical features.

4.2 DATA TYPES

Prolog's single data type is the term. Terms are either atoms, numbers, variables

or compound terms.

An atom is a general-purpose name with no inherent meaning. Examples of atoms

include x, blue, 'Burrito', and 'some atom'.

Numbers can be floats or integers.

Variables are denoted by a string consisting of letters, numbers and underscore

characters, and beginning with an upper-case letter or underscore. Variables

closely resemble variables in logic in that they are placeholders for arbitrary terms.

A compound term is composed of an atom called a "functor" and a number of

"arguments", which are again terms. Compound terms are ordinarily written as a

functor followed by a comma-separated list of argument terms, which is contained

in parentheses. The number of arguments is called the term's arity. An atom can be

regarded as a compound term with arity zero. Examples of compound terms are

truck_year('Mazda', 1986) and 'Person_Friends'(zelda,[tom,jim]).

Special cases of compound terms:

A List is an ordered collection of terms. It is denoted by square brackets with the

terms separated by commas or in the case of the empty list, []. For example [1,2,3]

or [red,green,blue].

Strings: A sequence of characters surrounded by quotes is equivalent to a list of

(numeric) character codes, generally in the local character encoding, or Unicode if

the system supports Unicode. For example, "to be, or not to be".

4.3 RULES AND FACTS

Prolog programs describe relations, defined by means of clauses. Pure Prolog is

restricted to Horn clauses. There are two types of clauses: facts and rules. A rule is of the

form

Head :- Body.

and is read as "Head is true if Body is true". A rule's body consists of calls to predicates,

which are called the rule's goals. The built-in predicate ,/2 (meaning a 2-arity operator

with name ,) denotes conjunction of goals, and ;/2 denotes disjunction. Conjunctions and

disjunctions can only appear in the body, not in the head of a rule.

Clauses with empty bodies are called facts.

An example of a fact is:

cat(tom).

which is equivalent to the rule:

cat(tom) :- true.

The built-in predicate true/0 is always true.

Given the above fact, one can ask:

is tom a cat?

?- cat(tom).

Yes

what things are cats?

?- cat(X).

X = tom

Clauses with bodies are called rules. An example of a rule is:

animal(X):- cat(X).

If we add that rule and ask what things are animals?

?- animal(X).

X = tom

Due to the relational nature of many built-in predicates, they can typically be used in

several directions. For example, length/2 can be used to determine the length of a list

(length(List, L), given a list List) as well as to generate a list skeleton of a given length

(length(X, 5)), and also to generate both list skeletons and their lengths together

(length(X, L)). Similarly, append/3 can be used both to append two lists (append(ListA,

ListB, X) given lists ListA and ListB) as well as to split a given list into parts (append(X,

Y, List), given a list List). For this reason, a comparatively small set of library predicates

suffices for many Prolog programs.

As a general purpose language, Prolog also provides various built-in predicates to

perform routine activities like input/output, using graphics and otherwise communicating

with the operating system. These predicates are not given a relational meaning and are

only useful for the side-effects they exhibit on the system. For example, the predicate

write/1 displays a term on the screen.

4.4 EVALUATION

Execution of a Prolog program is initiated by the user's posting of a single goal,

called the query. Logically, the Prolog engine tries to find a resolution refutation of the

negated query. The resolution method used by Prolog is called SLD resolution.

If the negated query can be refuted, it follows that the query, with the appropriate

variable bindings in place, is a logical consequence of the program. In that case, all

generated variable bindings are reported to the user, and the query is said to have

succeeded. Operationally, Prolog's execution strategy can be thought of as a

generalization of function calls in other languages, one difference being that multiple

clause heads can match a given call. In that case, the system creates a choice-point,

unifies the goal with the clause head of the first alternative, and continues with the goals

of that first alternative. If any goal fails in the course of executing the program, all

variable bindings that were made since the most recent choice-point was created are

undone, and execution continues with the next alternative of that choice-point. This

execution strategy is called chronological backtracking. For example:

mother_child(trude, sally).

father_child(tom, sally).

father_child(tom, erica).

father_child(mike, tom).

sibling(X, Y) :- parent_child(Z, X), parent_child(Z, Y).

parent_child(X, Y) :- father_child(X, Y).

parent_child(X, Y) :- mother_child(X, Y).

This results in the following query being evaluated as true:

?- sibling(sally, erica).

Yes

This is obtained as follows: Initially, the only matching clause-head for the query

sibling(sally, erica) is the first one, so proving the query is equivalent to proving the body

of that clause with the appropriate variable bindings in place, i.e., the conjunction

(parent_child(Z,sally), parent_child(Z,erica)). The next goal to be proved is the leftmost

one of this conjunction, i.e., parent_child(Z, sally). Two clause heads match this goal.

The system creates a choice-point and tries the first alternative, whose body is

father_child(Z, sally). This goal can be proved using the fact father_child(tom, sally), so

the binding Z = tom is generated, and the next goal to be proved is the second part of the

above conjunction: parent_child(tom, erica). Again, this can be proved by the

corresponding fact. Since all goals could be proved, the query succeeds. Since the query

contained no variables, no bindings are reported to the user. A query with variables, like:

?- father_child(Father, Child).

enumerates all valid answers on backtracking.

Notice that with the code as stated above, the query ?- sibling(sally, sally). also succeeds.

One would insert additional goals to describe the relevant restrictions, if desired.

4.4.1 Loops and recursion

Iterative algorithms can be implemented by means of recursive predicates.

4.4.2 Negation

The built-in Prolog predicate \+/1 provides negation as failure, which allows for non-

monotonic reasoning. The goal \+ legal(X) in the rule

illegal(X) :- \+ legal(X).

is evaluated as follows: Prolog attempts to prove the legal(X). If a proof for that goal can

be found, the original goal (i.e., \+ legal(X)) fails. If no proof can be found, the original

goal succeeds. Therefore, the \+/1 prefix operator is called the "not provable" operator,

since the query ?- \+ Goal. succeeds if Goal is not provable. This kind of negation is

sound if its argument is "ground" (i.e. contains no variables). Soundness is lost if the

argument contains variables and the proof procedure is complete. In particular, the query

?- illegal(X). can now not be used to enumerate all things that are illegal.

4.5 EXAMPLES

Here follow some example programs written in Prolog.

4.5.1 Hello world

An example of a query:

?- write('Hello world!'), nl.

Hello world!

true.

?-

4.5.2 Compiler optimization

Any computation can be expressed declaratively as a sequence of state transitions. As an

example, an optimizing compiler with three optimization passes could be implemented as

a relation between an initial program and its optimized form:

program_optimized(Prog0, Prog) :-

optimization_pass_1(Prog0, Prog1),

optimization_pass_2(Prog1, Prog2),

optimization_pass_3(Prog2, Prog).

or equivalently using DCG notation:

program_optimized --> optimization_pass_1, optimization_pass_2, optimization_pass_3.

4.5.3 Quicksort

The Quicksort sorting algorithm, relating a list to its sorted version:

partition([], _, [], []).

partition([X|Xs], Pivot, Smalls, Bigs) :-

( X @< Pivot ->

Smalls = [X|Rest],

partition(Xs, Pivot, Rest, Bigs)

; Bigs = [X|Rest],

partition(Xs, Pivot, Smalls, Rest)

). quicksort([]) --> [].

quicksort([X|Xs]) -->

{ partition(Xs, X, Smaller, Bigger) },

quicksort(Smaller), [X], quicksort(Bigger).

4.6 DESIGN PATTERNS

A design pattern is a general reusable solution to a commonly occurring problem

in software design. In Prolog, design patterns go under various names: skeletons and

techniques, cliches, program schemata, and logic description schemata. An alternative to

design patterns is higher order programming.

4.7 HIGHER-ORDER PROGRAMMING

By definition, first-order logic does not allow quantification over predicates.

A higher-order predicate is a predicate that takes one or more other predicates as

arguments. Prolog already has some built-in higher-order predicates such as call/1, call/2,

call/3, findall/3, setof/3, and bagof/3. Furthermore, since arbitrary Prolog goals can be

constructed and evaluated at run-time, it is easy to write higher-order predicates like

maplist/2, which applies an arbitrary predicate to each member of a given list, and

sublist/3, which filters elements that satisfy a given predicate, also allowing for currying.

To convert solutions from temporal representation (answer substitutions on backtracking)

to spatial representation (terms), Prolog has various all-solutions predicates that collect

all answer substitutions of a given query in a list. This can be used for list

comprehension. For example, perfect numbers equal the sum of their proper divisors:

perfect(N) :-

between(1, inf, N), U is N // 2,

findall(D, (between(1,U,D), N mod D =:= 0), Ds),

sumlist(Ds, N).

This can be used to enumerate perfect numbers, and also to check whether a number is

perfect.

4.7.1 Modules

For programming in the large, Prolog provides a module system. The module system is

standardised by ISO. However, not all Prolog compilers support modules and there are

compatibility problems between the module systems of the major Prolog compilers. [18]

Consequently, modules written on one Prolog compiler will not necessarily work on

others.

4.7.2 Parsing

There is a special notation called definite clause grammars (DCGs). A rule defined via --

>/2 instead of :-/2 is expanded by the preprocessor (expand_term/2, a facility analogous

to macros in other languages) according to a few straightforward rewriting rules,

resulting in ordinary Prolog clauses. Most notably, the rewriting equips the predicate with

two additional arguments, which can be used to implicitly thread state around, analogous

to monads in other languages. DCGs are often used to write parsers or list generators, as

they also provide a convenient interface to difference lists.

4.7.3 Meta-interpreters and reflection

Prolog is a homoiconic language and provides many facilities for reflection. Its implicit

execution strategy makes it possible to write a concise meta-circular evaluator (also

called meta-interpreter) for pure Prolog code. Since Prolog programs are themselves

sequences of Prolog terms (:-/2 is an infix operator) that are easily read and inspected

using built-in mechanisms (like read/1), it is easy to write customized interpreters that

augment Prolog with domain-specific features.

4.8 IMPLEMENTATION

4.8.1 ISO Prolog

The ISO Prolog standard consists of two parts. ISO/IEC 13211-1, published in 1995,

aims to standardize the existing practices of the many implementations of the core

elements of Prolog. It has clarified aspects of the language that were previously

ambiguous and leads to portable programs. There are two corrigenda: Cor.1:2007 and

Cor.2:2012 ISO/IEC 13211-2, published in 2000, adds support for modules to the

standard. The standard is maintained by the ISO/IEC JTC1/SC22/WG17 working group.

ANSI X3J17 is the US Technical Advisory Group for the standard.

4.8.2 Compilation

For efficiency, Prolog code is typically compiled to abstract machine code, often

influenced by the register-based Warren Abstract Machine (WAM) instruction set.

Some implementations employ abstract interpretation to derive type and mode

information of predicates at compile time, or compile to real machine code for high

performance. Devising efficient implementation methods for Prolog code is a field of

active research in the logic programming community, and various other execution

methods are employed in some implementations. These include clause binarization and

stack-based virtual machines.

4.8.3 Tail recursion

Prolog systems typically implement a well-known optimization method called tail call

optimization (TCO) for deterministic predicates exhibiting tail recursion or, more

generally, tail calls: A clause's stack frame is discarded before performing a call in a tail

position. Therefore, deterministic tail-recursive predicates are executed with constant

stack space, like loops in other languages.

4.8.4 Term indexing

Finding clauses that are unifiable with a term in a query is linear in the number of

clauses. Term indexing uses a data structure that enables sublinear-time lookups.

Indexing only affects program performance, it does not affect semantics.

4.8.5 Tabling

Some Prolog systems, (BProlog, XSB and Yap), implement a memoization method called

tabling, which frees the user from manually storing intermediate results.

4.9 IMPLEMENTATION IN HARDWARE

During the Fifth Generation Computer Systems project, there were attempts to

implement Prolog in hardware with the aim of achieving faster execution with dedicated

architectures. Furthermore, Prolog has a number of properties that may allow speed-up

through parallel execution. A more recent approach has been to compile restricted Prolog

programs to a field programmable gate array. However, rapid progress in general-purpose

hardware has consistently overtaken more specialized architectures.

4.10 CRITICISM

Although Prolog is widely used in research and education, Prolog and other logic

programming languages have not had a significant impact on the computer industry in

general. Most applications are small by industrial standards, with few exceeding 100,000

lines of code. Programming in the large is considered to be complicated because not all

Prolog compilers support modules, and there are compatibility problems between the

module systems of the major Prolog compilers. Portability of Prolog code across

implementations has also been a problem, but developments since 2007 have meant: "the

portability within the family of Edinburgh/Quintus derived Prolog implementations is

good enough to allow for maintaining portable real-world applications.

Software developed in Prolog has been criticized for having a high performance penalty

compared to conventional programming languages. However, advances in

implementation methods have reduced the penalties to as little as 25%-50% for some

applications.

Prolog is not purely declarative: because of the cut operator, a procedural reading of a

Prolog program is needed to understand it. The order of clauses in a Prolog program is

significant. Other logic programming languages, such as Datalog, are truly declarative:

clauses can be given in any order.

4.11 EXTENSIONS

Various implementations have been developed from Prolog to extend logic

programming capabilities in numerous directions. These include types, modes, constraint

logic programming (CLP), object-oriented logic programming (OOLP), concurrency,

linear logic (LLP), functional and higher-order logic programming capabilities, plus

interoperability with knowledge bases:

TYPES

Prolog is an untyped language. Attempts to introduce types date back to the 1980s, and as

of 2008 there are still attempts to extend Prolog with types. Type information is useful

not only for type safety but also for reasoning about Prolog programs.

Modes

The syntax of Prolog does not specify which arguments of a predicate are inputs and

which are outputs. However, this information is significant and it is recommended that it

be included in the comments. Modes provide valuable information when reasoning about

Prolog programs and can also be used to accelerate execution.

Constraints

Constraint logic programming extends Prolog to include concepts from constraint

satisfaction. A constraint logic program allows constraints in the body of clauses, such

as: A(X,Y) :- X+Y>0. It is suited to large-scale combinatorial optimization problems, and

is thus useful for applications in industrial settings, such as automated time-tabling and

production scheduling. Most Prolog systems ship with at least one constraint solver for

finite domains, and often also with solvers for other domains like rational numbers.

4.12 HIGHER-ORDER PROGRAMMING

HiLog and λProlog extend Prolog with higher-order programming features. ISO

Prolog now supports the built-in predicates call/2, call/3, ... which facilitate higher-order

programming and lambda abstractions.

maplist(_Cont, [], []).

maplist(Cont, [X1|X1s], [X2|X2s]) :-

call(Cont, X1, X2),

maplist(Cont, X1s, X2s).

4.13 OBJECT-ORIENTATION

Logtalk is an object-oriented logic programming language that can use most

Prolog implementations as a back-end compiler. As a multi-paradigm language, it

includes support for both prototypes and classes.

Oblog is a small, portable, object-oriented extension to Prolog by Margaret McDougall of

EdCAAD, University of Edinburgh.

Objlog was a frame-based language combining objects and Prolog II from CNRS,

Marseille, France.

Graphics

Prolog systems that provide a graphics library are SWI-prolog, Visual-prolog, LPA

Prolog for Windows and B-Prolog.

Concurrency

Prolog-MPI is an open-source SWI-Prolog extension for distributed computing over the

Message Passing Interface. Also there are various concurrent Prolog programming

languages.

Web programming

Some Prolog implementations, notably SWI-Prolog and Ciao, support server-side web

programming with support for web protocols, HTML and XML. There are also

extensions to support semantic web formats such as RDF and OWL. Prolog has also been

suggested as a client-side language.

Adobe Flash

Cedar is a free and basic Prolog interpreter. From version 4 and above Cedar has a FCA

(Flash Cedar App) support. This provides a new platform to programming in Prolog

through ActionScript.

Other

F-logic extends Prolog with frames/objects for knowledge representation.

OW Prolog has been created in order to answer Prolog's lack of graphics and

interface.

4.14 INTERFACES TO OTHER LANGUAGES

Frameworks exist which can bridge between Prolog and other languages:

The LPA Intelligence Server allows the embedding of LPA Prolog within C,

C#, C++, Java, VB, Delphi, .Net, Lua, Python and other languages. It is

exploits the dedicated string data-type which LPA Prolog provides

The Logic Server API allows both the extension and embedding of Prolog in

C, C++, Java, VB, Delphi, .NET and any language/environment which can call

a .dll or .so. It is implemented for Amzi! Prolog Amzi! Prolog + Logic Server

but the API specification can be made available for any implementation.

JPL is a bi-directional Java Prolog bridge which ships with SWI-Prolog by

default, allowing Java and Prolog to call each other (recursively). It is known

to have good concurrency support and is under active development.

InterProlog, a programming library bridge between Java and Prolog,

implementing bi-directional predicate/method calling between both languages.

Java objects can be mapped into Prolog terms and vice-versa. Allows the

development of GUIs and other functionality in Java while leaving logic

processing in the Prolog layer. Supports XSB, with support for SWI-Prolog

and YAP planned for 2013.

Prova provides native syntax integration with Java, agent messaging and

reaction rules. Prova positions itself as a rule-based scripting (RBS) system for

middleware. The language breaks new ground in combining imperative and

declarative programming.

PROL An embeddable Prolog engine for Java. It includes a small IDE and a

few libraries.

GNU Prolog for Java is an implementation of ISO Prolog as a Java library

(gnu.prolog)

Ciao provides interfaces to C, C++, Java, and relational databases.

C#-Prolog is a Prolog interpreter written in (managed) C#. Can easily be

integrated in C# programs. Characteristics: reliable and fairly fast interpreter,

command line interface, Windows-interface, builtin DCG, XML-predicates,

SQL-predicates, extendible. The complete source code is available, including a

parser generator that can be used for adding special purpose extensions.

Jekejeke Prolog API provides tightly coupled concurrent call-in and call-out

facilities between Prolog and Java or Android, with the marked posibility to

create individual knowledge base objects. It can be used to embed the ISO

Prolog interpreter in standalones, applets, servlets, APKs, etc..

A Warren Abstract Machine for PHP A Prolog compiler and interpreter in PHP

5.3. A library that can be used standalone or within Symfony2.1 framework