mirrors/Nim
1
0
Fork 0
mirror of https://github.com/nim-lang/Nim.git synced 2026-08-29 01:51:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
16,306 Commits 100 Branches 71 Tags
fec5d04afc4f2e9a22e31bca37ceacf8ac42dca8
Commit Graph

72 Commits

Author SHA1 Message Date
Kaushal Modi
349039678a unicode.split: Fix the splitting when a Rune separator is used [backport] (#13629) 
* unicode.split: Fix the splitting when a Rune separator is used [backport]

- Fixes https://github.com/nim-lang/Nim/issues/13628
- Ref https://irclogs.nim-lang.org/11-03-2020.html#20:01:34

* unicode.split: Remove the sepLen based logic.. resulted in wrong jumps

(cherry picked from commit 64995db4fd)
 2020-04-14 07:10:13 +02:00
narimiran
4675aae90b [backport] run nimpretty on string stuff 
(cherry picked from commit dcf3181bd1)
 2019-09-30 18:43:36 +02:00
Andreas Rumpf
d4af0554c4 remove unicode.Rune16 without deprecation period; fixes https://github.com/nim-lang/RFCs/issues/151 (#12072) 2019-08-28 10:25:13 +02:00
Andreas Rumpf
5a9edb222b unicode.nim: fixed spacing to something sane 2019-08-26 23:30:35 +02:00
Kaushal Modi
d40872bf9b [bugfix] Make std/unicode compile with --styleCheck:error (#11763) 2019-07-17 19:48:15 +02:00
Araq
c94647aeca styleCheck: make the compiler and large parts of the stdlib compatible with --styleCheck:error 2019-07-10 12:42:41 +02:00
Arne Döring
88b5dd3362 right shift is now by default sign preserving (#11322) 
* right shift is now by default sign preserving
* fix hashString and semfold
* enable arithmetic shift right globally for CI
* fix typo
* remove xxx
* use oldShiftRight as flag
* apply feedback
* add changelog entry
 2019-05-29 16:48:00 +02:00
Miran
a63c2a25d0 fix changing case for Unicode characters which change size (#11271) 
Some Unicode characters don't have the same byte length
in their upper- and lower-case variants.
 2019-05-20 09:18:47 +02:00
Miran
aeb30a72c0 update unicode.nim (#10921) 
* update unicode.nim

* create a script to create the needed unicode data
* make unicode.nim compatible with Unicode v12.0.0
* slightly improve unicode.nim documentation (fixes #4795)

* more documentation
 2019-03-31 08:36:04 +02:00
Miran
69149a0e92 [backport] improve unicode docs, fixes #2353 (#10174) 
* as instructed in #2353, provides a short description why
there are no specialized procs for seq[Rune]
* adds several examples to better explain what some functions do
* small fixes (double backticks, add missing dots, etc.)
* use `rune` instead of "unicode characer"
 2019-01-04 13:20:12 +01:00
pgkos
518c72e57a fixes #9800 (#9804) [backport] 
* fixes #9800
* add tests for #9800
 2018-11-26 18:59:24 +01:00
Yuriy Glukhov
d146516008 Fixed wording (#9761) 2018-11-20 12:50:08 +01:00
Araq
3f6168b337 removes deprecated T/P types 2018-11-16 13:27:56 +01:00
Arne Döring
795d7e3217 WIP 2018-11-08 20:52:22 +01:00
Araq
bc26c08862 unicode.nim: minor alignment change 2018-11-02 11:29:30 +01:00
Andreas Rumpf
8960efa340 unicode: added strip, align, alignLeft, repeat procs; refs #6301 2018-10-14 11:00:22 +02:00
Andreas Rumpf
77a8c6ffd8 unicode: add split procs; refs #6301 2018-10-14 10:47:54 +02:00
Andreas Rumpf
7f81519f41 strutils/unicode: deprecate isLower and friends operating on complete strings; fixes #7963 properly 2018-10-14 09:48:54 +02:00
Kaushal Modi
24df909d8a Make isUpper (and variants) work for strings with non-alpha chars 
The other variants are isLower, isUpperAscii and isLowerAscii

Fixes https://github.com/nim-lang/Nim/issues/7963.

This commit changes the behavior and signatures of:

- isUpper, isLower in the unicode module
- isUpperAscii, isLowerAscii in the strutils module

A second mandatory parameter skipNonAlpha is added to these 4 procs.

(This change affects only for the case where the input is a *string*.)

---

With skipNonAlpha set to true, the behavior mimics the Python isupper and
islower behavior i.e. non-alphabetic chars/runes are ignored when checking if
the string is upper-case or lower-case.

    Before this commit:

      doAssert(not isUpper("A B"))

    After this commit:

      doAssert(not isUpper("A B", false))    <-- old behavior
      doAssert isUpper("A B", true)

      Below two are equivalent:

                           isUpper("A B", true)

        isAlpha("A B") and isUpper("A B", false)

.. and the similar for other 3 procs.
 2018-06-08 15:14:29 -04:00
Araq
4e63f0c267 keep unicode.nim warning free 2018-05-18 01:35:53 +02:00
Jacek Sieka
72dfe176f5 remove dead code elimination option (#7669) 2018-04-23 11:02:38 +02:00
Dominik Picheta
b74a5148a9 Fixes #6223. 2017-11-28 14:33:53 +00:00
Dominik Picheta
4789aa979d Little spelling fix in unicode module 2017-10-15 13:12:20 +01:00
Araq
3caf108425 system/unicode: check for buffer overflows; refs #5284 2017-02-08 15:22:36 +01:00
Andreas Rumpf
54cc702351 update stdlib to not use deprecated symbols 2016-08-25 17:21:48 +02:00
Hans Raaf
3cea6e8a96 Added iterator for utf8 strings 2016-07-13 00:25:31 +02:00
Joey Payne
e0203a4463 Add useful unicode procs for string manipulation 
Added: isUpper, isLower, isAlpha, isWhiteSpace, toUpper,
toLower, and capitalize

Renamed strutils procs that are similar to avoid conflicts
 2016-07-01 07:37:35 -06:00
Joey Payne
f6e30981a8 Add new procs for string manipulation 
Add center, isTitle, title, partition, rpartition, rsplit, swapCase,
translate, and expandTabs
 2016-06-13 20:54:23 -06:00
Hans Raaf
2791915d7f Optimized end offsets and added tests. 
I hope this also shows that there are use cases. I still think the user
should get warned about performance issues with those procs, which I
added to the doc comments.
 2016-06-02 17:47:33 +02:00
Hans Raaf
ac6de565ec More work in optimizing, names and added substr(). 
This is work in progress. I added an unicode substring. Tried to handle
edgecases more consistent too.
 2016-06-02 17:43:10 +02:00
Hans Raaf
1138cf5234 Some procs to deal with Rune position base indexing. 
It can't be perfect but at least one can index on rune position
efficiently.
 2016-06-02 17:43:10 +02:00
Parashurama
a98705dddc change 'Rune' type in unicode module to 'int32' 2016-06-02 00:02:27 +02:00
theduke
25b605a3a2 validateUtf8: catch overlong ascii 
Make unicode.validateUtf8() check for overlong ascii representations, which are 2 bytes long and start with c0 or c1.
 2015-11-26 16:05:24 +01:00
Araq
ab6f8f6e5b fixesunicode.lastRune 2015-09-29 19:30:44 +02:00
Araq
73279aba39 added unicode.lastRun, unicode.graphemeLen 2015-09-21 15:49:46 +02:00
Adam Strzelecki
43bddf62dd lib: Trim .nim files trailing whitespace 
via OSX: find . -name '*.nim' -exec sed -i '' -E 's/[[:space:]]+$//' {} +
 2015-09-04 23:03:56 +02:00
apense
48b0de8ab4 Corrected proc name in assertion 2015-07-09 13:49:47 -04:00
apense
5fd7b7850a Corrected documentation 2015-07-09 13:45:20 -04:00
apense
c334e89ee7 Renamed to toRunes 2015-07-04 15:07:29 -04:00
apense
64b3395ade Added new proc 
In reference to #2353
 2015-07-03 21:33:12 -04:00
apense
0ee1672d69 Updated whitespace ranges 
Ranges sourced from <http://www.unicode.org/Public/7.0.0/ucd/PropList.txt>_. Wikipedia also uses these ranges on its information page <http://en.wikipedia.org/wiki/Whitespace_character#Unicode>_. 0xfeff isn't included in the list, but it is a no-break space, so I guess it makes sense. 0x200b is actually a format character, but it is a zero-width space. To fit Unicode, both 0x200b and 0xfeff would be removed.
 2015-06-08 19:48:57 -04:00
Araq
d3fc6e1f28 marshalling can be done at compile-time 2015-04-25 23:17:00 +02:00
def
22b4e4c2f2 Use more Natural and Positive numbers in proc parameters 
- Didn't go through all modules, only the main ones I thought of
- Building the compiler and tests still work
 2015-04-06 02:24:17 +02:00
def
bacb91002a make toUTF8 support up to 6 bytes 2015-03-03 21:25:28 +01:00
def
512db9aea6 Fix documentation a bit in unicode 2015-02-14 19:57:32 +01:00
def
ae7ca46a09 Optimize unicode.reversed 
Runs about 18 times faster:
- combining characters with boolean logic instead of binary search
- No more temporary sequence
- Optimize for ASCII characters
 2015-01-15 23:11:02 +01:00
def
0a82b6eb62 Add reversed proc to unicode module 2015-01-02 23:52:46 +01:00
Araq
11b6958755 big rename 2014-08-27 23:42:51 +02:00
Araq
36afdca87f resolved conflicts with master 2014-01-18 01:16:45 +01:00
Araq
438703f59e case consistency: next steps 2013-12-29 01:13:51 +01:00