What is Unicode? How does it work and how do you use it?

Unicode is what you use when you use accented or special characters. It also is what you use when you use emoji, these symbols that now have become a universal language. But how does Unicode work, and how did we get to the point where we can use all these symbols and trust them to work in many applications and on many devices?

Let's start with some history. ASCII was the first widely used standard for character representation on computers. It started in 1963 and uses only 7 bit. This means that the maximum number of characters it can represent are 128, and of those only 95 are printable. ASCII is centered around the english language and does not support the characters that many other languages use.

With the growing availability of computers, the next relevant standard showed up in 1987 in the form of ISO 8859. This standard uses 8 bit, allowing 256 characters to be encoded. But that is not nearly enough to cover the many languages in the world, so a number of variations were created. For example, ISO 8859-1 uses the extensions beyond the 7-bit ASCII characters to represent accented characters from western european languages such as german, french, and italian.

As a developer working with Unicode, you have to remember that Unicode has several encodings of varying lengths. When reading Unicode, your development environment very likely will help you to read files that contain Unicode text. When writing Unicode, the most reasonable default to choose is UTF-8, some languages default to UTF-16, and you probably want to stay away from UTF-32.

In summary: Unicode is great because it solves most character representation problems. It's an evolving standard with new characters and symbols being added all the time. Working with Unicode means that you have to deal with its various encodings, but in most cases your development environment will support you with both reading and writing Unicode files.

Additional Resources:

🎥 What are UTF-8 and UTF-16? Working with Unicode encodings -    • What are UTF-8 and UTF-16? Working wi...  
👉 Wikipedia: Unicode - https://en.wikipedia.org/wiki/Unicode
👉 Unicode Consortium - https://home.unicode.org/


00:00 Introduction
00:23 Unicode History: ASCII
01:14 Unicode History: ISO 8859
02:32 Unicode Invention
03:16 Unicode 15
03:51 Unicode Encodings: UTF-8, UTF-16, UTF-32
04:35 Reading Unicode Text
05:19 Writing Unicode Text
05:56 Summary
07:02 Wrap-Up