With some inputs larger than `BiggestUInt.high`, `parseBiggestUInt` proc
in `parseutils.nim` fails to detect overflow and returns random value.
This is because `rawParseUInt` try to detects overflow with `if prev >
res:` but it doesn't detects the overflow from multiplication.
It is possible that `x *= 10` causes overflow and resulting value is
larger than original value.
Here is example values larger than `BiggestUInt.high` but
`parseBiggestUInt` returns without detecting overflow:
```
22751622367522324480000000
41404969074137497600000000
20701551093035827200000000000000000
22546225502460313600000000000000000
204963831854661632000000000000000000
```
Following code search for values larger than `BiggestUInt.high` and
`parseBiggestUInt` cannot detect overflow:
```nim
import std/[strutils]
const
# Increase this to extend search range
NBits = 34'u
NBitsMax1 = 1'u shl NBits
NBitsMax = NBitsMax1 - 1'u
# Increase this when there are too many results and want to see only larger result.
MinMultiply10 = 14
var nfound = 0
for i in (NBitsMax div 10'u + 1'u) .. NBitsMax:
var
x = i
n10 = 0
for j in 0 ..< NBits:
let px = x
x = (x * 10'u) and NBitsMax
if x < px:
break
inc n10
if n10 >= MinMultiply10:
echo "i = ", i
echo "uint: ", (i shl (64'u - NBits)), '0'.repeat n10
inc nfound
if nfound > 15:
break
echo "found: ", nfound
```
fixes#23513
Also, the old `runnableExample` is just a copy of `proc
parseInt(openArray[char], var int, int)` variant (in Line 1000).
---------
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
* This adds `parseutils.parseSize`, an inverse to `strutils.formatSize`
which has existed since 2017.
It is useful for parsing the compiler's own output logs (like SuccessX)
or many other scenarios where "human readable" units have been chosen.
The doc comment and tests explain accepted syntax in detail.
Big units lead to small numbers, often with a fractional part, but we
parse into an `int64` since that is what `formatSize` stringifies and
this is an inverse over partial function slots. Although metric
prefixes z & y for zettabyte & yottabyte are accepted, these will
saturate the result at `int64.high` unless the qualified number is a
small fraction. This should not be much of a problem until such sizes
are common (at which point another overload with the parse result
either `float64` or `int128` could be added).
Tests avoids `test()` because of a weakly related static: test() failure
as mentioned in https://github.com/nim-lang/Nim/pull/21325. This is a
more elemental VM failure. As such, it needs its own failure exhibition
issue that is a smaller test case. (I am working on that, but unless
there is a burning need to `parseSize` at compile-time before run-time
it need not hold up this PR.)
* This worked with `int` but fails with `int64`. Try for green tests.
* Lift 2-result matching into a `checkParseSize` template and format as a
table of input & 2 expected outputs which seems nicer and to address
https://github.com/nim-lang/Nim/pull/21349#pullrequestreview-1294407679
* Fix (probably) the i386 trouble by using `int64` consistently.
* Improve documentation by mentioning saturation.
* Improve documentation with `runnableExamples` and a little more detail in
the main doc comment based on excellent code review by @juancarlospaco:
https://github.com/nim-lang/Nim/pull/21349#pullrequestreview-1294564155
* Address some more @juancarlospaco code review concerns.
* Remove a stray space.
* Mention milli-bytes in docs to maybe help clarify why wild conventions
are so prone to going case-insensitive-metric.
* Add some parens.
* Added 'openarray[char]' overloads to 'std/parseutils'
* Removed redundant `start` and `last` params from slice using procs
* Fixed type for parseIdent overload
* fixed one by off with 'substr'
* removed missed start parameters for procedures
* Added 'openarray[char]' overloads to 'std/parseutils'
* Removed redundant `start` and `last` params from slice using procs
* Fixed type for parseIdent overload
* fixed one by off with 'substr'
* removed missed start parameters for procedures
* Fixed VM op to work with new 'opcSlice'
* Corrected captureBetween's logic to work with openarray
* js sys's parsefloat logic now uses openarray
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* I don't care about observable stores
* enforce explicit initializations
* cleaner code for the stdlib
* stdlib: use explicit initializations
* make tests green
* algorithm.nim: set result explicitly
* remove out parameters and bring the PR into a mergable state
* updated the changelog
* Error -> Defect for defects
The distinction between Error and Defect is subjective,
context-dependent and somewhat arbitrary, so when looking at an
exception, it's hard to guess what it is - this happens often when
looking at a `raises` list _without_ opening the corresponding
definition and digging through layers of inheritance.
With the help of a little consistency in naming, it's at least possible
to start disentangling the two error types and the standard lib can set
a good example here.
* make parsutils.parseBin generic and improve runnableExamples
* reimplement parseBin/Oct/Hex and improve runnableExamples
* update changelog.md file with parseBin/Oct/Hex and fix a typo
* clarify `parseHexInt`, `parseOctInt` docstring and exception msgs
* add `parseBinInt` based on `parseutil.parseBin` implementation
Adds a `parseBinInt`, which parses a binary integer string and returns
it as an integer. This is based on the implementation of
`parseutil.parseBin`, removing the unnecessary parts.
* add tests for all `parse(Hex|Oct|Bin)Int` procs
* replace `parse*Int` proc impls by call to parseutil procs
Replaces the `parse(Hex|Oct|Bin)Int` procedure implementation by calls
to the `parseutil` procs, which receive a mutable argument.
Has the main advantage that the empty string as well as a "prefix
only" string, e.g. "0x" counts as an invalid integer.
Also moves the `parseOctInt` proc further up in the file so that all
`parse` procs are below one another.
* replace `var L` by `let L` in `parse` procs
There's no reason for the usage of `var` here.
* add `maxLen` optional arg for `parseutil.parse(Oct|Bin)`
Plus small change to test cases.
* update changelog about `parse*Int` procs
* fix `rejectParse` template in `tstrutils`
* make sure only `s.len` chars are parsed, if `maxLen+start` > s.len
Fixes a previous bug in `parseHex` (and now affected `parseOct` and
`parseBin`), which allowed to set `start + maxLen` to be larger than
the strings length. This resulted in an out of bounds access.
* move `parse*Int` proc change to breaking changes, add double `
