Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Certain unicode surrogate pair emoji are not recognized #80621

Closed
Koyper opened this issue Aug 14, 2023 · 7 comments · Fixed by #80650
Closed

Certain unicode surrogate pair emoji are not recognized #80621

Koyper opened this issue Aug 14, 2023 · 7 comments · Fixed by #80650
Assignees
Milestone

Comments

@Koyper
Copy link
Contributor

Koyper commented Aug 14, 2023

Godot version

4.2.dev

System information

MacOS

Issue description

These emoji are not recognized correctly by Button, Label, TextEdit or RichTextLabel. They do render correctly in the editor, however:

U+2764 U+FE0F U+200D U+1F525 ; 13.1 # ❤️‍🔥 heart on fire
U+2764 U+FE0F U+200D U+1FA79 ; 13.1 # ❤️‍🩹 mending heart
U+0023 U+FE0F U+20E3 ; 0.6 # #️⃣ keycap: #
U+002A U+FE0F U+20E3 ; 2.0 # *️⃣ keycap: *
U+0030 U+FE0F U+20E3 ; 0.6 # 0️⃣ keycap: 0
U+0031 U+FE0F U+20E3 ; 0.6 # 1️⃣ keycap: 1
U+0032 U+FE0F U+20E3 ; 0.6 # 2️⃣ keycap: 2
U+0033 U+FE0F U+20E3 ; 0.6 # 3️⃣ keycap: 3
U+0034 U+FE0F U+20E3 ; 0.6 # 4️⃣ keycap: 4
U+0035 U+FE0F U+20E3 ; 0.6 # 5️⃣ keycap: 5
U+0036 U+FE0F U+20E3 ; 0.6 # 6️⃣ keycap: 6
U+0037 U+FE0F U+20E3 ; 0.6 # 7️⃣ keycap: 7
U+0038 U+FE0F U+20E3 ; 0.6 # 8️⃣ keycap: 8
U+0039 U+FE0F U+20E3 ; 0.6 # 9️⃣ keycap: 9

They do render if a smiley face precedes them in the text string, so it appears to be a bug in the unicode surrogate parsing.

Screenshot 2023-08-14 at 9 37 24 AM

Steps to reproduce

See MRP.

Minimal reproduction project

emoji_rendering_bugs.zip

@AThousandShips
Copy link
Member

AThousandShips commented Aug 14, 2023

Does your font contain these emoji? They aren't automatically available unless you have a font that contains them

@Calinou
Copy link
Member

Calinou commented Aug 14, 2023

cc @bruvzg

Does your font contain these emoji? They aren't automatically available unless you have a font that contains them

Godot 4.x automatically uses system fonts as a fallback, but support depends on the OS: https://docs.godotengine.org/en/latest/tutorials/ui/gui_using_fonts.html#using-emoji

@bruvzg
Copy link
Member

bruvzg commented Aug 14, 2023

The editor directly includes default system emoji fonts as a fallback. Project use automatic fallback, so this should be an issue with auto fallback.

@bruvzg bruvzg self-assigned this Aug 14, 2023
@Koyper
Copy link
Contributor Author

Koyper commented Aug 14, 2023

Here's a running project with the complete set - you can see only the mentioned emoji fail to render. The failing ones are all available in the OS. By-the-way, this is brilliant that this works in Godot 4 so well already!

Screenshot 2023-08-14 at 10 01 28 AM

@Koyper
Copy link
Contributor Author

Koyper commented Aug 14, 2023

@bruvzg Is there a String method that can return characters whereas an emoji is handled as a single character rather than a compound unicode? For example, simply to find how many emoji's are within a given string? This is related to how in TextEdit, it can take multiple caret shifts to cross certain emoji's.

Also, can the filter type currently applied to fallback system emoji be set somewhere? Godot emoji are slightly blurry as compared to the natively displayed emoji on MacOS (in standard DPI - not hiDPI).

@bruvzg
Copy link
Member

bruvzg commented Aug 14, 2023

Is there a String method that can return characters whereas an emoji is handled as a single character rather than a compound unicode?

No, but I take a look at implementing it, it should be possible to do with ICU.

The issue itself can be fixed by using a full script run to search for fallback (currently it's using first char to select font, that's why it's not working with multichar emojis), but this need to be tested to make sure it works with the strings that require multiple fallback fonts on all systems. So detecting a composite icon and using only it might be a better solution.

diff --git a/modules/text_server_adv/text_server_adv.cpp b/modules/text_server_adv/text_server_adv.cpp
index b92082ee1b..85d7087384 100644
--- a/modules/text_server_adv/text_server_adv.cpp
+++ b/modules/text_server_adv/text_server_adv.cpp
@@ -5444,7 +5444,7 @@ void TextServerAdvanced::_shape_run(ShapedTextDataAdvanced *p_sd, int64_t p_star
 		// Try system fallback.
 		RID fdef = p_fonts[0];
 		if (_font_is_allow_system_fallback(fdef)) {
-			String text = p_sd->text.substr(p_start, 1);
+			String text = p_sd->text.substr(p_start, p_end - p_start);
 			String font_name = _font_get_name(fdef);
 			BitField<FontStyle> font_style = _font_get_style(fdef);
 			int font_weight = _font_get_weight(fdef);

@Koyper
Copy link
Contributor Author

Koyper commented Aug 14, 2023

Can project emoji font fallback also be set to use the default system emoji like the editor? The rendering works correctly if I set the font to the system emoji font, in this case Apple Color Emoji, but then we have to set for every platform.

It's curious that even without the project font fallback set, most of the emoji still render using the default system emoji?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants