Lesson 13: Handling Unicode
Lesson 13: Handling Unicode
Fundamentals of Text Processing for Linguists Na-Rae Han
Objectives
Shameek's presentation:
Object-oriented programming
Handling Unicode
4/9/2014
2
The ASCII chart
CII%20Conversion%20Chart.pdf
Decimal 0 ... 35 36 ... 48 49 50 ...
Binary (7-bit) 000 0000 ... 010 0011 010 0100 ... 011 0000 011 0001 011 0010 ...
4/9/2014
Character (NULL) ... # & ... 0 1 2 ...
Decimal 65 66 67 ... 97 98 99 ... 127
Binary (7-bit) 100 0001 100 0010 100 0011 ... 110 0001 110 0010 110 0011 ... 111 1111
Character A B C ... a b c ...
(DEL)
3
Extending ASCII: ISO-8859, etc.
ASCII (=7 bit, 128 characters) was sufficient for encoding English. But what about characters used in other languages?
Solution: Extend ASCII into 8-bit (=256 characters) and use the additional 128 slots for non-English characters
ISO-8859: has 16 different implementations!
ISO-8859-1
aka Latin-1: French, German, Spanish, etc.
ISO-8859-7
Greek alphabet
ISO-8859-8
Hebrew alphabet
JIS X 0208: Japanese characters
Problem: overlapping character code space.
224dec means ? in Latin-1 but in ISO-8859-8!
4/9/2014
4
Unicode
A character encoding standard developed by the Unicode Consortium
Provides a single representation for all world's writing systems
"Unicode provides a unique number for every character, no matter what the platform, no matter what the program, no matter what the language."
()
4/9/2014
5
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- dataacademy star ying introduction to python
- declare character encoding python
- s python cheat sheet data science free
- writing python 2 3 compatible code
- cs 3101 3 programming languages python lecture 2
- overcoming frustration correctly using unicode in python
- six documentation read the docs
- ppyytthhoonn ssttrriinnggss tutorialspoint
- 1 functions in python
- programming principles in python csci 503 490
Related searches
- cash handling procedures template
- cash handling policy and procedures
- data classification and handling policy
- basic cash handling procedures
- cash handling procedures best practices
- quality material handling inc
- restaurant cash handling policy template
- warehouse material handling procedures
- material handling procedures samples
- manual materials handling procedures
- cash handling procedure manual
- sample material handling policy