http://scienceforkids.kidipede.com/chemistry/atoms/proton.htm which of the two appears simple to...

http://scienceforkids.kidipede.com/chemistry/atoms/proton.htm

http://en.wikipedia.org/wiki/Proton

Which of the two appears simple to

you?

1

2

Search for a keyword

Results – Sometimes irrelevantand mixed order of readability

Our Objective

Query

Retrieve web pages(considering relevance)

Re-rank web pages based on readability

Automatically accomplished

An Unsupervised Technical Readability Ranking Model by Building a Conceptual

Terrain in LSI

Shoaib Jameel

Xiaojun Qian

The Chinese University of Hong Kong

This is me!

What has been done so far?

• Heuristic Readability formulae• Unsupervised approaches• Supervised approaches

My focus in this talk would be to cover some popular works in this area. Exhaustive listof references can be found in my paper.

Heuristic Readability Methods

• Have been there since 1940’s

• Semantic Component – Number of syllables per word, length of the syllables per word etc.

• Syntactic Component – Length of sentences etc.

Example – Flesch Reading Ease

Semantic componentSyntactic component

Manually tuned numerical parameters

Supervised Learning Methods

• Language Models• SVMs (Support Vector Machines)• Use of query Log and user profiles

Smoothed Unigram Model [1]

[1] K. Collins-Thompson and J. Callan. (2005.) "Predicting reading difficulty with statistical language models". Journal of the American Society for Information Science and Technology 56(13) (pp. 1448-1462).

•Recast the well-studied problem of readability in terms of text categorization and used straightforward techniques from statistical language modeling.

Smoothed Unigram Model

Limitation of their method: Requires training data, which sometimes may be difficult to obtain

Domain-specific Readability• Jin Zhao and Min-Yen Kan. 2010. Domain-specific iterative readability

computation. In Proceedings of the 10th annual joint conference on Digital libraries (JCDL '10).

Based on web-link structure algorithm HITS and SALSA.

• Xin Yan, Dawei Song, and Xue Li. 2006. Concept-based document readability in domain specific information retrieval. In Proceedings of the 15th ACM international conference on Information and knowledge management (CIKM '06).

Based on an ontology. Tested only in the medical domain

Hypertext Induced Topic Search Stochastic Approach for Link-Structure Analysis

I will focus on this work.

Overview

• The authors state that Document Scope and Document Cohesion are an important parameters in finding simple texts.

• The authors have used a controlled vocabulary thesaurus termed as Medical Subject Headings (MeSH).

• Authors have pointed out the readability based formulae are not directly applicable to web pages.

MeSH Ontology

Concept difficulty increases

Concept difficulty decreases

Overall Concept Based Readability Score

where,DaCw = Dale-Chall Readability MeasurePWD = Percentage of difficult wordsAvgSL = Average sentence length in di

Their work focused on word level readability, hence considered only the PWD

len(ci,cj)=function to compute shortest path between concepts c i cj in the MeSH hierarchyN = total number of domain concepts in document d i

Depth(ci)=depth of the concept ci in the concept hierarchyD= Maximum depth of concept hierarchy

Our “Terrain-based” Method

So, what’s the connection?

Latent Semantic Indexing

• Core component – Singular Value Decomposition

• SVD(C) = USVT

C =

SVD(C) = USVT

U S VT

Three components

• Term Centrality – Is the term central to the document’s theme?

• Term Cohesion – Is the term closely related with the other terms in the document?

• Term Difficulty – Will the reader find it difficult to comprehend the meaning of the term?

Term Centrality

• Closeness of the term vector with the document vector in the latent space.

T1 T2

D

More central

Less central

LSI latent space

Term Centrality = 1 / {Euclidean distance (T1,D)+small constant}

Term Cohesion

T2T1

T3T4

D

LSI Latent SpaceDistance

Normalization is done to standardize the values.

Term cohesion is obtained by computing the Euclidean distance between two consecutive terms T1 and T2

Term Difficulty

• Term Difficulty = Term Centrality x Inverse Document Frequency (idf)

• Idea of idf – If a term is used less often in the document collection, it should be regarded as important.

• For example, ‘’proton’’ will not occur too often but it is an important term.

So, what we now obtain?Te

rm D

ifficu

lty

Term Cohesion

A reader now has to hop from one term to the other in the LSI latent space

Something like this ->

How ranking is done?

• Keep aggregating the individual transition scores.

• Finally, obtain a real number which will be used in ranking.

Experiments and Results

• Collect web pages from domain-specific sites such as Science and Psychology websites.

• Test in two domains.• Used NDCG as an evaluation metric.• Retrieve relevant web pages given a query• Annotate top ten web pages• Re-rank search results based on readability

Results - Psychology

NDCG@3 NDCG@5 NDCG@7 NDCG@100

0.1

0.2

0.3

0.4

0.5

0.6

0.7 ARI Coleman-Liau Flesch Fog LIXSMOG Terrain(Stop) Terrain(No Stop)

Results - Science

NDCG@3 NDCG@5 NDCG@7 NDCG@100

0.1

0.2

0.3

0.4

0.5

0.6

0.7 ARI Coleman-Liau Flesch Fog LIXSMOG Terrain(Stop) Terrain(No Stop)

END