Michael Weghorn
76cceb3cad
In order to represent the (codepoint of a) Unicode character
in UTF-16, multiple UTF-16 code units may be needed.
So far, the Korean Hangul/Hanja character conversion
was assuming that each Unicode character was only
consisting of a single UTF-16 code unit, and splitting
into single sal_Unicode characters accordingly in
TextConversion_ko::getCharConversions.
This breaks for characters represented by multiple
code units, like "𡽜" (CJK Unified Ideograph [1],
unicode codepoint: 139100/U+21F5C) which is represented
by 2 UTF-16 code units.
As a consequence, an invalid OUString/UTF-16 string would
be passed to HangulHanjaConversionDialog::SetCurrentString
for some of the suggestions.
Fix this by no longer splitting into single sal_Unicode,
but use o3tl::iterateCodePoints to split into actual
code points.
Example: When using "Tools" -> "Language" ->
"Hangul/Hanja Conversion..." for a Korean text of
"한글맞춤법", the string of all characters for potential
conversions in TextConversion_ko::getCharConversions is:
"韓漢寒限閑恨旱汗翰邯罕悍澣閒瀚桿捍閈暵鼾犴僩嫺熯嫻駻鷳扞狠僴覵瞷猂邗骭撊鷴垾憪橌㸁䛞䦘闲鹇娴汉焊闬韩仠佷傼兯哻娨嫨屽晘橺浫澖皔蓒蔊蛝螒豻貋釬銲雗鶾鷼麲㒏㛠㡾㢨㪋㯗㲦㵄䁂䂅䈨䉯䍐䍑䎯䏷䓍䓳䕿䗙䥜䦥䮧𡽜忓"
Previously, the last 3 of them ("䮧𡽜忓") would incorrectly be
split into (stdout output, showing something is wrong):
"䮧", " " (something wrong, but a valid UTF-16 string), "?" (wrong, and invalid UTF-16 string), "忓"
With the fix in place, they remain intact:
"䮧", "𡽜", "忓"
Also adjust HangulHanjaConversionDialog::OnSuggestionModified
accordingly to compare the number of code points the
string consists of, not the number of UTF-16 code units.
Otherwise, the "Replace"/"Always Replace" buttons in the
"Hangul/Hanja Conversion" dialog would remain greyed
out for the above example when selecting the "𡽜" character
(that now correctly gets displayed with the fix in
TextConversion_ko::getCharConversions in place).
Thanks to Jonathan Clark for pointing out that additional
aspect in review!
Add a corresponding test case.
While the fix is in i18npool, the new test is implemented
in editeng, where the HangulHanjaConversion class uses
the i18npool logic via a UNO service.
The test is implemented there because the component context
availabe in i18npool doesn't supply the required service.
The same test implemented there would fail like this:
An uncaught UNO exception
- com.sun.star.uno.DeploymentException: {Message: "component context fails to supply service com.sun.star.linguistic2.ConversionDictionaryList of type com.sun.star.linguistic2.XConversionDictionaryList at /home/michi/development/git/libreoffice/workdir/UnoApiHeadersTarget/offapi/normal/com/sun/star/linguistic2/ConversionDictionaryList.hpp:45", Context: @0x7fff17acecf0}
[1] https://codepoints.net/U+21F5C
Change-Id: I1ab202cc0d3b5a6ff5d220f4b8e90c578f591158
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/207017
Reviewed-by: Jonathan Clark <jonathan@libreoffice.org>
Tested-by: Jenkins
Internationalisation Pool (i18npool) Framework
Internationalisation pool (i18npool) framework ensures that the suite is adaptable to the requirements of different native languages, their local settings and customs, etc without source code modification. (Wow, that is such marketing-speak...)
Specifically for locale data documentation please see i18npool/source/localedata/data/locale.dtd