Separate the I18N calls for immutable strings and for pluraliseable strings.

Also update tests.
This commit is contained in:
Maurizio M. Gavioli
2024-06-05 10:32:23 +02:00
parent 7064166da2
commit 53755824fb
3 changed files with 177 additions and 77 deletions

View File

@@ -1,31 +1,44 @@
/*
The `i18n` package is flexible and easy to use.
The `i18n` package is a flexible and easy to use way to localise applications.
It has one call to get a translation: `get`, which the user can alias into something like `T`.
It has two calls to get a translation: `get()` amd `get_n()`, which the user can alias into something like `T` and `Tn`
with statements like:
T :: i18n.get
Tn :: i18n.get_n.
`get`, referred to as `T` here, has a few different signatures.
All of them will return the key if the entry can't be found in the active translation catalog.
`get()` is used for retrieving the translation of sentences which **never**change in form,
like for instance "Connection established" or "All temporary files have been deleted".
Note that the number (singular, dual, plural, whatever else) is not relevant: the semtece is fixed and it will have only one possible translation in any other language.
- `T(key)` returns the translation of `key`.
- `T(key, n)` returns a pluralized translation of `key` according to value `n`.
`get_n()` is used for retrieving the translations of sentences which change according to the number of items referenced.
The various signatures of `get_n()` have one more parameter, `n`, which will receive that number,
and which be used to select the correct form according to the pluraliser attached to the message catalogue when initially loaded;
for instance, to summarise a rather complex matter, some languages use the singular form when reerring to 0 items and some use the (only in their case) plural forms;
also, languages may have more or less quantifier forms than a single singular form an a universal plural form:
for istance, Chinese has just one form for any quantity, while Welsh may have up to 6 different forms for specific different quantities.
- `T(section, key)` returns the translation of `key` in `section`.
- `T(section, key, n)` returns a pluralized translation of `key` in `section` according to value `n`.
Both `get()` and `get_n()`, referred to as `T` and `Tn` here, have several different signatures.
All of them will return the key if the entry can't be found in the active translation catalogue.
By default lookup take place in the global `i18n.ACTIVE` catalogue for ease of use, unless a speciic catalogue is supplied
By default lookup take place in the global `i18n.ACTIVE` catalog for ease of use.
If you want to override which translation to use, for example in a language preview dialog, you can use the following:
- `T(key)` returns the translation of `key`.
- `T(key, catalog)` returns the translation of `key` from explictly supplied catalogue.
- `T(section, key)` returns the translation of `key` in `section`.
- `T(section, key, catalog)` returns the translation of `key` in `section` from explictly supplied catalogue.
- `T(key, n, catalog)` returns the pluralized version of `key` from explictly supplied catalog.
- `T(section, key, n, catalog)` returns the pluralized version of `key` in `section` from explictly supplied catalog.
- `Tn(key, n)` returns the translation of `key` according to number of items `n`.
- `Tn(key, n, catalog)` returns the translation of `key` from explictly supplied catalogue.
- `Tn(section, key, n)` returns the translation of `key` in `section` according to number of items `n`.
- `Tn(section, key, n, catalog)` returns the translation of `key` in `section` according to number of items `n` from explictly supplied catalogue.
If a catalog has translation contexts or sections, then omitting it in the above calls looks up in section "".
The default pluralization rule is n != 1, which is to say that passing n == 1 (or not passing n) returns the singular form.
Passing n != 1 returns plural form 1.
The default pluralization rule is `n != 1``, which is to say that passing n == 1 returns the singular form (in slot 0).
Passing n != 1 returns plural form in slot 1 (if any).
Should a language not conform to this rule, you can pass a pluralizer procedure to the catalog parser.
This is a procedure that maps an integer to an integer, taking a value and returning which plural slot should be used.
This is a procedure that maps an integer to an integer, taking a numbe of item and returning which plural slot should be used.
You can also assign it to a loaded catalog after parsing, of course.
@@ -35,6 +48,7 @@ Example:
import "core:text/i18n"
T :: i18n.get
Tn :: i18n.get_n
mo :: proc() {
using fmt
@@ -60,9 +74,9 @@ Example:
println(T("Hellope, World!"))
println("-----")
// We pass 1 into `T` to get the singular format string, then 1 again into printf.
printf(T("There is %d leaf.\n", 1), 1)
printf(Tn("There is %d leaf.\n", 1), 1)
// We pass 42 into `T` to get the plural format string, then 42 again into printf.
printf(T("There is %d leaf.\n", 42), 42)
printf(Tn("There is %d leaf.\n", 42), 42)
/*
This isn't in the translation catalog, so the key is passed back untranslated.
@@ -99,8 +113,8 @@ Example:
println("-----")
println("--- apple_count section ---")
println("apple_count:%d apple(s) =")
println("\t 1 =", T("apple_count", "%d apple(s)", 1))
println("\t 42 =", T("apple_count", "%d apple(s)", 42))
println("\t 1 =", Tn("apple_count", "%d apple(s)", 1))
println("\t 42 =", Tn("apple_count", "%d apple(s)", 42))
}
*/
package i18n

View File

@@ -84,64 +84,149 @@ DEFAULT_PARSE_OPTIONS :: Parse_Options{
merge_sections = false,
}
/*
Several ways to use:
- get(key), which defaults to the singular form and i18n.ACTIVE catalog, or
- get(key, number), which returns the appropriate plural from the active catalog, or
- get(key, number, catalog) to grab text from a specific one.
*/
get_single_section :: proc(key: string, number := 1, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
plural := 1 if number != 1 else 0
// *****************
// get() PROC GROUP
// *****************
if catalog.pluralize != nil {
plural = catalog.pluralize(number)
}
return get_by_slot(key, plural, catalog)
/*
Returns the first translation string for the passed `key`.
It is also aliases with `get( )`.
Two ways to use it:
- get(key), which defaults to the `i18n.ACTIVE`` catalogue, or
- get(key, catalog) to grab text from a specific loaded catalogue.
Inputs:
- key: the string to translate.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_single_section :: proc(key: string, catalog: ^Translation = ACTIVE) -> (value: string) {
return get_by_slot(key, 0, catalog)
}
/*
Several ways to use:
- get(section, key), which defaults to the singular form and i18n.ACTIVE catalog, or
- get(section, key, number), which returns the appropriate plural from the active catalog, or
- get(section, key, number, catalog) to grab text from a specific one.
*/
get_by_section :: proc(section, key: string, number := 1, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
plural := 1 if number != 1 else 0
Returns the first translation string for the passed `key` in a specific section or context.
It is also aliases with `get( )`.
if catalog.pluralize != nil {
plural = catalog.pluralize(number)
}
return get_by_slot(section, key, plural, catalog)
Two ways to use it:
- get(section, key), which defaults to the `i18n.ACTIVE`` catalogue, or
- get(section, key, catalog) to grab text from a specific loaded catalogue.
Inputs:
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
- key: the string to translate.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_by_section :: proc(section, key: string, catalog: ^Translation = ACTIVE) -> (value: string) {
return get_by_slot(section, key, 0, catalog)
}
get :: proc{get_single_section, get_by_section}
// *****************
// get_n() PROC GROUP
// *****************
/*
Several ways to use:
- get_by_slot(key), which defaults to the singular form and i18n.ACTIVE catalog, or
- get_by_slot(key, slot), which returns the requested plural from the active catalog, or
- get_by_slot(key, slot, catalog) to grab text from a specific one.
Returns the translation string for the passed `key` in a specific plural form (if present in the catalogue).
It is also aliases with `get_n( )`.
Two ways to use it:
- get_n(key, quantity), which returns the appropriate plural from the active catalogue, or
- get_n(key, quantity, catalog) to grab text from a specific loaded catalogue.
Inputs:
- key: the string to translate.
- qantity: the quantity of item to be used to select the correct plural form.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_single_section_w_plural :: proc(key: string, quantity: int, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
slot := 1 if quantity != 1 else 0
if catalog.pluralize != nil {
slot = catalog.pluralize(quantity)
}
return get_by_slot(key, slot, catalog)
}
/*
Returns the translation string for the passed `key` in a specific plural form (if present in the catalogue)
in a specific section or context.
It is also aliases with `get_n( )`.
Two ways to use it:
- get(section, key, quantity), which returns the appropriate plural from the active catalogue, or
- get(section, key, quantity, catalog) to grab text from a specific loaded catalogue.
Inputs:
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
- key: the string to translate.
- qantity: the quantity of item to be used to select the correct plural form.
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_by_section_w_plural :: proc(section, key: string, quantity: int, catalog: ^Translation = ACTIVE) -> (value: string) {
/*
A lot of languages use singular for 1 item and plural for 0 or more than 1 items. This is our default pluralize rule.
*/
slot := 1 if quantity != 1 else 0
if catalog.pluralize != nil {
slot = catalog.pluralize(quantity)
}
return get_by_slot(section, key, slot, catalog)
}
get_n :: proc{get_single_section_w_plural, get_by_section_w_plural}
// *****************
// get_by_slot() PROC GROUP
// *****************
/*
Two ways to use:
- get_by_slot(key, slot), which returns the requested plural from the active catalogue, or
- get_by_slot(key, slot, catalog) to grab text from a specific loaded catalogue.
If a file format parser doesn't (yet) support plural slots, each of the slots will point at the same string.
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
Inputs:
- key: the string to translate.
- qantity: the translation slot to choose (slots refer to plural forms specific for each language and their meaning changes from catalogue to catalogue).
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_by_slot_single_section :: proc(key: string, slot := 0, catalog: ^Translation = ACTIVE) -> (value: string) {
get_by_slot_single_section :: proc(key: string, slot: int, catalog: ^Translation = ACTIVE) -> (value: string) {
return get_by_slot_by_section("", key, slot, catalog)
}
/*
Several ways to use:
- get_by_slot(key), which defaults to the singular form and i18n.ACTIVE catalog, or
Two ways to use:
- get_by_slot(key, slot), which returns the requested plural from the active catalog, or
- get_by_slot(key, slot, catalog) to grab text from a specific one.
If a file format parser doesn't (yet) support plural slots, each of the slots will point at the same string.
Inputs:
- section: the catalogue section (sometime also called 'context') from which to lookup the translation
- key: the string to translate.
- qantity: the translation slot to choose (slots refer to plural forms specific for each language and their meaning changes from catalogue to catalogue).
- catalog: the catalogue to use for the translation (defaults to i18n.ACTIVE)
Returns: the translated string or the original `key` if no translation was found.
*/
get_by_slot_by_section :: proc(section, key: string, slot := 0, catalog: ^Translation = ACTIVE) -> (value: string) {
get_by_slot_by_section :: proc(section, key: string, slot: int, catalog: ^Translation = ACTIVE) -> (value: string) {
if catalog == nil || section not_in catalog.k_v {
/*
Return the key if the catalog catalog hasn't been initialized yet, or the section is not present.
@@ -161,7 +246,6 @@ get_by_slot_by_section :: proc(section, key: string, slot := 0, catalog: ^Transl
get_by_slot :: proc{get_by_slot_single_section, get_by_slot_by_section}
/*
Same for destroy:
- destroy(), to clean up the currently active catalog catalog i18n.ACTIVE
- destroy(catalog), to clean up a specific catalog.
*/
@@ -181,4 +265,4 @@ destroy :: proc(catalog: ^Translation = ACTIVE, allocator := context.allocator)
delete(catalog.k_v)
strings.intern_destroy(&catalog.intern)
free(catalog)
}
}

View File

@@ -5,6 +5,7 @@ import "core:testing"
import "core:text/i18n"
T :: i18n.get
Tn :: i18n.get_n
Test :: struct {
section: string,
@@ -47,7 +48,8 @@ test_custom_pluralizer :: proc(t: ^testing.T) {
{"", "Message1/plural", "This is message 1", 1},
{"", "Message1/plural", "This is message 1 - plural A", 1_000_000},
{"", "Message1/plural", "This is message 1 - plural B", 42},
// This isn't in the catalog, so should ruturn the key.
// This isn't in the catalog, so should return the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1},
},
})
@@ -61,11 +63,11 @@ test_mixed_context :: proc(t: ^testing.T) {
plural = nil,
tests = {
// These are in the catalog.
{"", "Message1", "This is message 1 without Context", 1},
{"Context", "Message1", "This is message 1 with Context", 1},
{"", "Message1", "This is message 1 without Context",-1},
{"Context", "Message1", "This is message 1 with Context", -1},
// This isn't in the catalog, so should ruturn the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1},
{"", "Come visit us on Discord!", "Come visit us on Discord!", -1},
},
})
}
@@ -90,15 +92,15 @@ test_nl_mo :: proc(t: ^testing.T) {
plural = nil, // Default pluralizer
tests = {
// These are in the catalog.
{"", "There are 69,105 leaves here.", "Er zijn hier 69.105 bladeren.", 1},
{"", "Hellope, World!", "Hallo, Wereld!", 1},
{"", "There are 69,105 leaves here.", "Er zijn hier 69.105 bladeren.", -1},
{"", "Hellope, World!", "Hallo, Wereld!", -1},
{"", "There is %d leaf.\n", "Er is %d blad.\n", 1},
{"", "There are %d leaves.\n", "Er is %d blad.\n", 1},
{"", "There is %d leaf.\n", "Er zijn %d bladeren.\n", 42},
{"", "There are %d leaves.\n", "Er zijn %d bladeren.\n", 42},
// This isn't in the catalog, so should ruturn the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1},
{"", "Come visit us on Discord!", "Come visit us on Discord!", -1},
},
})
}
@@ -111,15 +113,15 @@ test_qt_linguist :: proc(t: ^testing.T) {
plural = nil, // Default pluralizer
tests = {
// These are in the catalog.
{"Page", "Text for translation", "Tekst om te vertalen", 1},
{"Page", "Also text to translate", "Ook tekst om te vertalen", 1},
{"installscript", "99 bottles of beer on the wall", "99 flessen bier op de muur", 1},
{"Page", "Text for translation", "Tekst om te vertalen", -1},
{"Page", "Also text to translate", "Ook tekst om te vertalen", -1},
{"installscript", "99 bottles of beer on the wall", "99 flessen bier op de muur", -1},
{"apple_count", "%d apple(s)", "%d appel", 1},
{"apple_count", "%d apple(s)", "%d appels", 42},
// These aren't in the catalog, so should ruturn the key.
{"", "Come visit us on Discord!", "Come visit us on Discord!", 1},
{"Fake_Section", "Come visit us on Discord!", "Come visit us on Discord!", 1},
{"", "Come visit us on Discord!", "Come visit us on Discord!", -1},
{"Fake_Section", "Come visit us on Discord!", "Come visit us on Discord!", -1},
},
})
}
@@ -133,16 +135,16 @@ test_qt_linguist_merge_sections :: proc(t: ^testing.T) {
options = {merge_sections = true},
tests = {
// All of them are now in section "", lookup with original section should return the key.
{"", "Text for translation", "Tekst om te vertalen", 1},
{"", "Also text to translate", "Ook tekst om te vertalen", 1},
{"", "99 bottles of beer on the wall", "99 flessen bier op de muur", 1},
{"", "Text for translation", "Tekst om te vertalen", -1},
{"", "Also text to translate", "Ook tekst om te vertalen", -1},
{"", "99 bottles of beer on the wall", "99 flessen bier op de muur", -1},
{"", "%d apple(s)", "%d appel", 1},
{"", "%d apple(s)", "%d appels", 42},
// All of them are now in section "", lookup with original section should return the key.
{"Page", "Text for translation", "Text for translation", 1},
{"Page", "Also text to translate", "Also text to translate", 1},
{"installscript", "99 bottles of beer on the wall", "99 bottles of beer on the wall", 1},
{"Page", "Text for translation", "Text for translation", -1},
{"Page", "Also text to translate", "Also text to translate", -1},
{"installscript", "99 bottles of beer on the wall", "99 bottles of beer on the wall", -1},
{"apple_count", "%d apple(s)", "%d apple(s)", 1},
{"apple_count", "%d apple(s)", "%d apple(s)", 42},
},
@@ -175,7 +177,7 @@ test :: proc(t: ^testing.T, suite: Test_Suite, loc := #caller_location) {
if err == .None {
for test in suite.tests {
val := T(test.section, test.key, test.n, cat)
val := test.n > -1 ? Tn(test.section, test.key, test.n, cat): T(test.section, test.key, cat)
testing.expectf(t, val == test.val, "Expected key `%v` from section `%v`'s form for value `%v` to equal `%v`, got `%v`", test.key, test.section, test.n, test.val, val, loc=loc)
}
}