arabic domain names: what’s been done so far?

www.arabic-www.arabic-domains.orgdomains.org

Arabic Domain Names What’s been done so

far?

Euro-SSIG July 25-31 2008Dr. Abdulaziz H. Al-Zoman

Chairman of Steering CommitteeArabic Domain Name Pilot Project

www.arabic-domains.org

AgendaAgendaAgendaAgenda

Introduction Arabic Language Characteristics Our contribution methodology What have we done so far? How to reaching Arabic domains?

If we more time …– From Language to Script … what are the issues?


IntroductionIntroductionWhat is the problem?What is the problem?IntroductionIntroductionWhat is the problem?What is the problem?

Current ASCII-based DNs are incapable of representing Arabic characters

Difficulty to reach Arabic sites using English DNs (pronunciation & spelling problems)

Full Arabic DNs will encourage Arab users to widely use the Internet

Arabic News paperاألوسط الشرق صحيفة

www.al-sharqalawsat.com

www.asharqalawsat.com

www.asharq-alaowsat.com

www.elsharkelaosat.com…

E-government Siteر يس.

www.yasser.gov.sawww.yaser.gov.sawww.yasir.gov.sawww.yassir.gov.sa

…


IntroductionIntroductionInternet in the Arab World Internet in the Arab World IntroductionIntroductionInternet in the Arab World Internet in the Arab World

Population of Arab world: 340 M (5% of world population)

Arab Internet users represent 2.5% of world users

Average Internet penetration in Arab world < 10 %

Less than 10% who can speak English in the Arab world


IntroductionIntroductionObstacles facing Internet in the Arab Obstacles facing Internet in the Arab worldworld

IntroductionIntroductionObstacles facing Internet in the Arab Obstacles facing Internet in the Arab worldworld Low level of telecommunication

infrastructure Lack of adequate regulations High cost Computer Illiteracy Language barrier

– Contents– Tools and applications– Domain names


Arabic Language Arabic Language CharacteristicsCharacteristicsArabic Language Arabic Language CharacteristicsCharacteristics

1. Consists of 28 characters.2. Written from right to left in a cursive style3. Most characters have different shapes

– depending on their position (beginning, middle, or end) within a word

– probably conjugated with preceding and succeeding characters.

– These different shapes for a single character do not count as different code points but they are handled using different fonts

– Letters that can be joined are always joined in both hand-written and printed Arabic.

ن ا ب ج

ـج ج ـجـ جــب ب ـبـ بــا ا

ـن ن ـنـ نـ

جبان


Arabic Language Arabic Language CharacteristicsCharacteristicsArabic Language Arabic Language CharacteristicsCharacteristics4. Tashkeel (diacritic)

– A small sing (not a letter) that is usually put on top or under a character for the purpose of correct pronunciation

– It is not widely used except incase of the possibility of mispronouncing words that have the same letters but with different pronunciations, and hence having different meanings.

5. MAbbreviations are not widely used– When an abbreviation is written (in domain

name) characters will be joined together … leads to a different word and pronunciation

6. Two sets of numerals are used:– Arabic : 0, 1, 2, 3, 4, 5, 6, 7, 8, 9– Arabic-Indic : 0، 1، 2، 3، 4، 5، 6، 7، 8، 9– Numerals are written from left to right

انAبج

BانCجب


Arabic Language Arabic Language CharacteristicsCharacteristicsArabic Language Arabic Language CharacteristicsCharacteristics

7. Words are separated by spaces– A space character is needed between words to get

correct shaping– Connecting words without spaces

• not acceptable• decrease readability • Confused with other words

مدارسخيف

خطأدبي

مدارس خيفخط أدبي

space

space


Contribution MethodologyContribution MethodologyContribution MethodologyContribution Methodology

The work was done based on the following

methodology:

1. Identifying problems & areas of contributions

2. Participating and initiating interest groups & task

forces

3. Conducting web surveys

4. Publishing reports & papers

5. Meeting linguists (face to face)6. Disseminating information to public 7. Testing and building local experiences


4 Arabic root servers

3 Technical solutions3 Technical solutions

4 Arabic root servers

Work Done so farWork Done so farIdentifying areas of Identifying areas of contributionscontributions

Work Done so farWork Done so farIdentifying areas of Identifying areas of contributionscontributions

To define the accepted Arabic character set to be

used for writing Arabic domain names

To define the top-level domains of the Arabic

domain name tree structure (i.e., Arabic gTLDs, and ccTLDs)

1 Linguistic issues

2 Arabic TLDs

IETF, UNICODE… ,

ICANN/IANA… ,

11 Levels of an A-IDN Solution


Work Done so far Work Done so far Identifying areas of Identifying areas of contributionscontributions

Work Done so far Work Done so far Identifying areas of Identifying areas of contributionscontributions

Linguistic issues– ISSUE 1.1: Tashkeel– ISSUE 1.2: Kasheeda– ISSUE 1.3: Taa-Marbota+Haa– ISSUE 1.4: Hamzah– ISSUE 1.5: Alif Maqsura+Ya– ISSUE 1.6: Numbers– ISSUE 1.7: dot or Arabic Zero– ISSUE 1.8: Connecting Multiple

Words– ISSUE 1.9: Space– ISSUE 1.10: Mixing Latin & Arabic

Characters– ISSUE 1.11: Special Charters– ISSUE 1.12: Accepted Character

Set

Arabic TLDs – ISSUE 2.1: Criteria for

selecting an Arabic gTLD

– ISSUE 2.2: Suggested list of Arabic gTLDs

– ISSUE 2.3: Criteria for selecting an Arabic ccTLD

– ISSUE 2.4: Suggested list of Arabic

ccTLDs

11


Work Done so far Work Done so far Participation & Initiation of Participation & Initiation of GroupsGroups

Work Done so far Work Done so far Participation & Initiation of Participation & Initiation of GroupsGroups

MINC: Multilingual Internet Names Consortium, 2000– Arabic Working Group

AINC: Arab Internet Names Consortium, April 2001– Founder and member of the board– Chairman of the Linguistic Committee

ADNTF: Arabic Domain Name Task Force, Q2/2003– Formed under the auspices of ESCWA (UN)– Issuing an Internet Draft for supporting the Arabic

language in domain names GCC ccTLDs Group:

– Formed under the auspices of ITC committee of GCC– GCC Arabic domain name pilot project

Arab Team for Arabic Domain Names, 2005– Formed under the auspices of Arab League– Arabic domain name pilot project

22


Work Done so far Work Done so far Publishing Reports & PapersPublishing Reports & PapersWork Done so far Work Done so far Publishing Reports & PapersPublishing Reports & Papers

5 Scientific research papers published in conference proceedings and journals

Technical reports

Internet drafts– http://www.ietf.org/internet-drafts/draft-farah-

adntf-ling-guidelines-00.txt

33


Work Done so far Work Done so far Publishing Reports & PapersPublishing Reports & PapersWork Done so far Work Done so far Publishing Reports & PapersPublishing Reports & Papers

Scientific Research Papers – "Arabic Top-Level Domain Names", International Journal of Computer

Processing of Oriental Languages, Volume 17 Number 3 September 2004, To Appear.

– "Linguistic Issues in Arabic Domain Names", In Proceedings of the 17th NCC, KAAU, Al-Madina Almunawarah, Saudi Arabia, 5-8 April, 2004, pp 235-250 [in Arabic]

– "Arabic Top-Level Domain Names", In Proceedings of the 17th NCC, KAAU, Al-Madina Almunawarah, Saudi Arabia, 5-8 April, 2004, pp 281-296 [in Arabic]

– "Using Arabic Language in writing domain names", Arab journal of library and information science, Vol 22, No. 3, July 2002, pp. 21-38 [in Arabic].

– " Using Arabic Language in writing domain names ", In Proceedings of IACIT 2001, JUST, Irbid, Jordan, 13-15 Nov., 2001, pp 264-272 [in Arabic]

Technical Reports – “Supporting the Arabic Language in Domain Names”, submitted to

ADNTF-ESCWA, October 2003• The base for the internet draft

– Status Report of the Arabic Linguistic Committee of AINC-September 2001

– Status Report of the Arabic Linguistic Committee of AINC-April 2002

33


Work Done so far Work Done so far Conducting Web SurveysConducting Web SurveysWork Done so far Work Done so far Conducting Web SurveysConducting Web Surveys

3 On-line web surveys – cover most of the linguistic

issues with more than 550 responses

Collected information have been analyzed and compared with the recommendations of the AINC linguistic committee

Results have been published and presented in conferences

44


Work Done so far Work Done so far Meeting Linguistic ExpertsMeeting Linguistic ExpertsWork Done so far Work Done so far Meeting Linguistic ExpertsMeeting Linguistic Experts

SaudiNIC met with 4 Arabic linguists to get their guidance regarding the Arabic linguistic issues in domain names.

55


Work Done so far Work Done so far Information DisseminationInformation DisseminationWork Done so far Work Done so far Information DisseminationInformation Dissemination

Web sites (in Arabic and English)– http://www.arabic-domains.org.sa

Participating in local/regional/international conferences and meetings

Publishing scientific research papers Publishing articles in newspaper and

magazine Radio programs Seminars to public and interested

groups

66


Work Done so far Work Done so far Test ImplementationsTest ImplementationsWork Done so far Work Done so far Test ImplementationsTest Implementations

Country level – Individually done be some Arab countries (ccTLDs)

• Arabic.English , e.g., نطاق.com.sa• Problem of mixing languages (left-to-right and right-to-left)

GCC level (2004-2005)– During the Gulf Cooperation Council (GCC) ccTLDs

group meeting on 7 March 2004, • “A Technical Proposal for Implementing Arabic

Domain names in the GCC Countries” was presented and accepted

– A technical taskforce was formed and assigned the task to implement the proposal within 6 months

Arab world (2005 - now)– The recommendations of the 2nd meeting of the Working

Group on Arabic Domain Names, Cairo, May 2005:• Extend the GCC Pilot Project for Arabic Domain Names to

include all members of the Arab League (22 countries). • Renamed it to be "Arabic Domain Names Pilot Project”.

• Under the auspices of the Arab League.– Implemented a browser plug-in

77

ae, bh, kw, om, qa, sa


Participated in the new ICANN Arabic example.test domains ( إختبار. (مثال– Moderate the Arabic site for the IDNwiki gateway– Published a technical report about the test “IDN

Top Level Domain Evaluations and Testing Report”

Develop many tools and systems that supports Arabic domain names– Browser plug-in “Arabic.Arabic”– Simple IDN registry system– IDN/ADN converter interface and many other tools

Work Done so far Work Done so far Other Testing and tools …Other Testing and tools …77


Reaching Arabic DomainsReaching Arabic DomainsReaching Arabic DomainsReaching Arabic Domains

User can reach Arabic domain names through1. Using ADNPP Root servers

• Through participating ISPs• Using browser that support IDN

2. Using plug-in (Arbic.Arabic)• Work for both MS IE and Mozilla Firefox

ع

AR-ROOT.NIC.NET.SA (Arabic Root Server)

AR-ROOT.NIC.AE(Arabic Root Server)

• Slave for all the Arabic ccTLDs.(Only NS records + any Glue A records)

• Master for all the Arabic ccTLDs.(Only NS records + any Glue A records)

AR-CCTLD.NIC.NET.SA (SA Arabic ccTLD Server)

• Master for .”السعودية“

NS1.UAENIC.AE(AE Arabic ccTLD Server)

• Master for .”اإلمارات“

AR-ROOT.QATAR.NET.QA(QA Arabic ccTLD Server)

• Master for “قطر”.

Ara

bic

Roo

t se

rver

sA

rabi

c cc

TLD

ser

vers

”السعودية“ ”اإلمارات“ ”قطر“

“.”


Examples …Examples …Examples …Examples …


ThanksThanksThanksThanks

شكراxn--mgbti4d

www.arabic-www.arabic-domains.orgdomains.org

Arabic Domain NamesArabic Domain NamesWhat are the issues ifWhat are the issues if

we expand and look at thewe expand and look at thewhole script?whole script?


About Arabic ScriptAbout Arabic ScriptAbout Arabic ScriptAbout Arabic Script

The 2nd most widely used alphabetic writing system in the world

Used by many languages such as: – Persian, Urdu, Turkish, Kurdish, Pashto, Jawi, …

It is widely used by more than 43 countries– more than one billion potential users could be

concerned in using Arabic script domain names.


Accepted characters for Arabic, Persian, Urdu, Pashto, Jawi


Arabic Script IDN - Major Arabic Script IDN - Major IssuesIssuesArabic Script IDN - Major Arabic Script IDN - Major IssuesIssues

1. Acceptable/disallowed characters– IDNA200x table (Pvalid /Disallowed /ContextO)– Language tables

2. Combining Marks3. Diacritics 4. World/label separators (space, ZWNJ, ZWJ,

hyphen)5. Digits6. Confusing similar characters (e.g. variant

tables)7. Bidirectional


Issues Need Further Issues Need Further Investigations Investigations 1. 1. Valid Unicode Codepoints Valid Unicode Codepoints

Issues Need Further Issues Need Further Investigations Investigations 1. 1. Valid Unicode Codepoints Valid Unicode Codepoints 0600..0603 ; CONTEXTO # ARABIC NUMBER SIGN..ARABIC SIGN SAFHA 0604..060A ; UNASSIGNED # <reserved>..<reserved> 060B..060F ; DISALLOWED# AFGHANI SIGN..ARABIC SIGN MISRA 0610..0615 ; PVALID # ARABIC SIGN SALLALLAHOU ALAYHE WASSALLAM..AR 0616..061A ; UNASSIGNED # <reserved>..<reserved> 061B ; DISALLOWED # ARABIC SEMICOLON 061C..061D ; UNASSIGNED # <reserved>..<reserved> 061E..061F ; DISALLOWED # ARABIC TRIPLE DOT PUNCTUATION MARK..ARABIC Q 0620 ; UNASSIGNED # <reserved> 0621..063A ; PVALID # ARABIC LETTER HAMZA..ARABIC LETTER GHAIN 063B..063F ; UNASSIGNED# <reserved>..<reserved> 0640..065E ; PVALID # ARABIC TATWEEL..ARABIC FATHA WITH TWO DOTS 065F ; UNASSIGNED # <reserved> 0660..0669 ; PVALID # ARABIC-INDIC DIGIT ZERO..ARABIC-INDIC DIGIT 066A..066D ; DISALLOWED # ARABIC PERCENT SIGN..ARABIC FIVE POINTED STA 066E..0674 ; PVALID # ARABIC LETTER DOTLESS BEH..ARABIC LETTER HIG 0675..0678 ; DISALLOWED # ARABIC LETTER HIGH HAMZA ALEF..ARABIC LETTER 0679..06D3 ; PVALID # ARABIC LETTER TTEH..ARABIC LETTER YEH BARREE 06D4 ; DISALLOWED # ARABIC FULL STOP 06D5..06DC ; PVALID # ARABIC LETTER AE..ARABIC SMALL HIGH SEEN 06DD ; CONTEXTO # ARABIC END OF AYAH 06DE ; DISALLOWED # ARABIC START OF RUB EL HIZB 06DF..06E8 ; PVALID # ARABIC SMALL HIGH ROUNDED ZERO..ARABIC SMALL 06E9 ; DISALLOWED # ARABIC PLACE OF SAJDAH 06EA..06FC ; PVALID # ARABIC EMPTY CENTRE LOW STOP..ARABIC LETTER 06FD..06FE ; DISALLOWED # ARABIC SIGN SINDHI AMPERSAND..ARABIC SIGN SI 06FF ; PVALID # ARABIC LETTER HEH WITH INVERTED V

Source: draft-faltstrom-idnbis-tables-05.txt


Disallowed characters by IDNA200X


Recommended Disallowed characters


Issues Need Further Issues Need Further InvestigationsInvestigations2. 2. Combining MarksCombining Marks

Issues Need Further Issues Need Further InvestigationsInvestigations2. 2. Combining MarksCombining Marks The use of combining marks with some base

characters would confuse with other character.– Combining maddah and hamza 0653-0655– Other combining marks 0656-065E

≠

ح+

{= wح

is confusing with

ځU+062d U+0654 U+062d U+0654 U+0681

ا+

{=

أis confusing with

أU+0627 U+0654 U+0627 U+0654 U+0623


Issues Need Further Issues Need Further InvestigationsInvestigations3. 3. Diacritics (Tashkeel)Diacritics (Tashkeel)

Issues Need Further Issues Need Further InvestigationsInvestigations3. 3. Diacritics (Tashkeel)Diacritics (Tashkeel) Tashkeel Points: about 9

– 064B-0652, 0670

From user point view, he/she does not know– Whether the label is with or without tashkeel – Whether the label is totally (i.e., all characters)

use tashkeel– The correct pronunciations to put the correct

taskeel– There are many different pronunciations

depending on the local dialogue

There will be huge number of combinations when registering a single label– good playground for phishing

جمعيةةCع{ي جم~ةCجمع{يع{ي.ة جم~جمع{ي.ةجمعية�جمعية�Bجمعيةمعية ج{جم.عية


Issues Need Further Issues Need Further InvestigationsInvestigations4. 4. ZWNJ & ZWJ Control ZWNJ & ZWJ Control CharactersCharacters

Issues Need Further Issues Need Further InvestigationsInvestigations4. 4. ZWNJ & ZWJ Control ZWNJ & ZWJ Control CharactersCharacters The support of ZWJ and ZWNJ in domain

names would result in some confusions Their use and concepts are not known to

regular users



Issues Need Further Issues Need Further InvestigationsInvestigations4. 4. ZWNJ & ZWJ Control ZWNJ & ZWJ Control CharactersCharacters ZWNJ: Visually noticed

ح بلinput[0] = U+062dinput[1] = U+200cinput[2] = U+0628input[3] = U+0644

حبلinput[0] = U+062d input[1] = U+0628 input[2] = U+0644



Issues Need Further Issues Need Further InvestigationsInvestigations4. 4. ZWNJ & ZWJ Control ZWNJ & ZWJ Control CharactersCharacters ZWNJ: Visually Unnoticed

ط بلinput[0] = U+0637input[1] = U+200cinput[2] = U+0628input[3] = U+0644

طبلinput[0] = U+0637 input[1] = U+0628 input[2] = U+0644

≠




-الدولي -الرباط مجمعinput[0] = U+0645 input[1] = U+062c input[2] = U+0645 input[3] = U+0639

input[4] = U+200d input[5] = U+002d input[6] = U+0627 input[7] = U+0644 input[8] = U+0631 input[9] = U+0628

input[10] = U+0627 input[11] = U+0637

input[12] = U+200d input[13] = U+002d input[14] = U+0627 input[15] = U+0644 input[16] = U+062f input[17] = U+0648 input[18] = U+0644 input[19] = U+064a

مجمع-الرباط-الدوليinput[0] = U+0645 input[1] = U+062c input[2] = U+0645 input[3] = U+0639 input[4] = U+002d input[5] = U+0627 input[6] = U+0644 input[7] = U+0631 input[8] = U+0628 input[9] = U+0637

input[10] = U+002d input[11] = U+0627 input[12] = U+0644 input[13] = U+062f input[14] = U+0648 input[15] = U+0644 input[16] = U+064a

ZWJVisually noticed

ZWJVisually Unnoticed


Issues Need Further Issues Need Further InvestigationsInvestigations5. 5. DigitsDigits

Issues Need Further Issues Need Further InvestigationsInvestigations5. 5. DigitsDigits Arabic-Indic VS. Eastern Arabic-Indic digits

– 0 1 2 3 4 5 6 7 8 9– ۹ ۸ ۷ ۶ ۵ ۴ ۳ ۲ ۱ ۰

۱۲۳۷۸۹۰input[0] = U+06f1input[1] = U+06f2input[2] = U+06f3input[3] = U+06f7input[4] = U+06f8input[5] = U+06f9input[6] = U+06f0

١٢٣٧٨٩٠input[0] = U+0661 input[1] = U+0662 input[2] = U+0663 input[3] = U+0667 input[4] = U+0668 input[5] = U+0669

input[6] = U+0660

≠


Issues Need Further Issues Need Further InvestigationsInvestigations5. 5. DigitsDigits

Issues Need Further Issues Need Further InvestigationsInvestigations5. 5. DigitsDigits Arabic-Indic vs. European-Arabic digits

– 0 1 2 3 4 5 6 7 8 9– 0 1 2 3 4 5 6 7 8 9

“Windows has supported number substitution by allowing the representation of different cultural shapes for the same digits while keeping the internal storage of these digits unified among different locales, for example numbers are stored in their well known hexadecimal values, 0x40, 0x41, but displayed according to the selected language.

Source: http://msdn2.microsoft.com/en-us/library/aa350685(VS.85).aspx?PHPSESSID=o1fb21liejulfgrptbmi9dec92

م12مinput[0] = U+0645input[1] = U+0031input[2] = U+0032input[3] = U+0645

م١٢مinput[0] = U+0645input[1] = U+0661input[2] = U+0662input[3] = U+0645

≠


Issues Need Further Issues Need Further InvestigationsInvestigations6. 6. Similar Shape CharactersSimilar Shape Characters

Issues Need Further Issues Need Further InvestigationsInvestigations6. 6. Similar Shape CharactersSimilar Shape Characters There are a number of groups of characters

that have the same shapes, – eg. Kaf, Heh, Yeh, Alef, … groups




ېلمنکلمنيکكلمني

input[0] = U+0643 input[1] = U+0644 input[2] = U+0645 input[3] = U+0646 input[4] = U+064a

input[0] = U+06a9 input[1] = U+0644 input[2] = U+0645 input[3] = U+0646 input[4] = U+064a

input[0] = U+06a9 input[1] = U+0644 input[2] = U+0645 input[3] = U+0646 input[4] = U+06d0

کلمنېکلمنيكلمني




ۍلکیلکكلى

input[0] = U+06a9 input[1] = U+0644 input[2] = U+06cd

input[0] = U+06a9 input[1] = U+0644 input[2] = U+06cc

input[0] = U+0643 input[1] = U+0644 input[2] = U+0649


Issues Need Further Issues Need Further InvestigationsInvestigations7. 7. Bidirectional BehaviorBidirectional Behavior

Issues Need Further Issues Need Further InvestigationsInvestigations7. 7. Bidirectional BehaviorBidirectional Behavior Arabic script domains will include characters

that are LTR (e.g., dash, dot, AE digits) Exiting IDNA2003 (stringprep) assumes that

the first and last characters of the label must be RandALCat characters.


ThanksThanksThanksThanks

شكراxn--mgbti4d

arabic domain names: what’s been done so far?

Technology