Скачать или смотреть S6T1 Why Determining the Length of a String is More Complicated Than You Think

EVENT: Internationalization & Unicode Conference 44 (IUC 44)
DATE: October 14-16, 2020

DESCRIPTION:
There are many ways of determining the “length” of a string, but most of them don’t correlate well with an average person’s expectations. For example, depending on encoding and what’s being measured, the length of the Vietnamese character “ẳ” could be 1, 2, 3, 4, 5, 6, 8, 12 or 16! But with the use of Unicode’s “Extended Grapheme Cluster” algorithm, the length matches user expectation, and is reliably 1. This session will go over the following possible “lengths” of a string, with their pros and cons:

-Bytes, with info on the differences between UTF-8, UTF-16, and UTF-32
-Code Points, and why they don’t always correspond to a “character”
-(Extended) Grapheme Clusters, how their boundaries are determined, and how to implement them in a variety of programming languages
-Considerations for cases beyond Grapheme Clusters

BIO:
Ben Yang, Director of Technology, PanLex The Long Now Foundation
Ben Yang is a linguist specializing in writing systems and the Director of Technology for PanLex. He is the Unicode representative for The Long Now Foundation and is a regular participant in Unicode Technical Committee meetings and the Unicode Script Ad-Hoc. He has a strong interest in making technology more widely available to speakers of underserved languages and the preservation and digitization of traditional writing systems. He has presented at WikidataCon, the Unicode Conference, and the International Multilingual User Group (IMUG).

LINKS
Visit the Unicode Website: https://home.unicode.org
Get Started with Unicode:
Technical Quick Start Guide: https://home.unicode.org/technical-qu...
Overview of Internationalization and Unicode Projects:    • Overview of Internationalization and Unico...  
Become a Member:
Organizations: https://home.unicode.org/membership/m...
Individuals: https://www.unicode.org/consortium/jo...
Learn Different Ways to Support Unicode: https://home.unicode.org/support-unic...