Fixes an issue that comes up when using strutils.`%` or any other
strutils/strformat feature that uses the unicode lookup tables behind
the scenes, on systems where ints are than 32-bit wide.
Tested with:
```bash
./koch test cat lib
```
Refer to the discussion in #23125.
(cherry picked from commit 4c38569229)
* Add use of Windows Wide CRT API for env. vars
Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.
Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.
Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.
Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.
Add tests that test environment variables with Unicode characters in
their name or value.
* Fix test issues
Fixes
* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
with "cannot importc"
* Fix missing error check on `wcstombs`
* Fix ANSI testing errors
* Separate ANSI-related testing to their own tests, and only executing
them if running process has a specific code page
* Setting locale with `setlocale` was not reliable and didn't work on
certain machines
* Add handling of a "no character representation" error in second
`wcstombs` call
* tests/newruntime_misc: Increment allocCount
Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.
* Revert "tests/newruntime_misc: Increment allocCount"
This reverts commit 4d4fe8bd3e.
* tests/newruntime_misc: Increment allocCount on Windows
Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.
* Refactor, adding suggestions from code review
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Document, adding suggestions
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Remove unnecessary environment tracking
* try to fix windows
* fix delEnv
* make putEnv work on windows even with empty values; improve tests: add tests, add js, vm testing
* [skip ci] fix changelog
Co-authored-by: Caden Haustein <code@brightlysalty.33mail.com>
* Replace calls to C `putenv` with C `setenv` to remove possible memory leaks
* Add test of correct behaviour on invalid input
* Fix style in tests/stdlib/tos.nim
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
* Update tests/stdlib/tos.nim
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
* Update tests/stdlib/tos.nim
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
* Add comment with bug number to tests/stdlib/tos.nim
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
* Fix possible msvc arch issues
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
* osproc: move fork-based code path under the when conditional
* osproc: avoid using the environ global on Haiku
* osenv: import environ from stdlib.h on Haiku
Haiku's environ is declared in `<stdlib.h>` by default, differing from
POSIX and/or Linux. Import it from there to avoid collision with anyone
importing `<stdlib.h>` from Nim.
* update unicode.nim
* create a script to create the needed unicode data
* make unicode.nim compatible with Unicode v12.0.0
* slightly improve unicode.nim documentation (fixes#4795)
* more documentation
