mirrors/Nim
1
0
Fork 0
mirror of https://github.com/nim-lang/Nim.git synced 2026-04-19 22:10:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

remove a condition that table size must be passed as power of 2 (#14926)

Browse Source 
* remove a condition that table size must be passed as power of 2
* remove power-of-2 condition from sets and sharedtables
* remove power-of-2 condition from deques
* use 'correctSize' for both branches
* prettify changelog.md and fix typos
* add a changelog entry
* fix double-call of 'right-size'
* fix the same thing in sets.nim
* introduce a new internal proc `slotsNeeded`

Deprecate the public proc `rightSize`, which is not needed anymore.
Now it is an identity function, allowing the old code to work
correctly and without extra allocations.
This commit is contained in:
Miran
2020-07-08 15:01:47 +02:00
committed by GitHub
parent 06d776a582
commit 3de5296337
10 changed files with 83 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
changelog.md
View File

@@ -5,7 +5,7 @@
## Standard library additions and changes
- Added `bindParams`, `bindParam` to `db_sqlite` for binding parameters into a `SqlPrepared` statement.
- Add `tryInsert`,`insert` procs to db_* libs accept primary key column name.
- Add `tryInsert`,`insert` procs to `db_*` libs accept primary key column name.
- Added `xmltree.newVerbatimText` support create `style`'s,`script`'s text.
- `uri` adds Data URI Base64, implements RFC-2397.
- Add [DOM Parser](https://developer.mozilla.org/en-US/docs/Web/API/DOMParser)
@@ -24,7 +24,7 @@
ease the writing of multi-process servers, where sockets inheritance is
desired.
For a transistion period, define `nimInheritHandles` to enable file handle
For a transition period, define `nimInheritHandles` to enable file handle
inheritance by default. This flag does **not** affect the `selectors` module
due to the differing semantics between operating systems.
@@ -83,11 +83,14 @@
proc foo(x: int, y: int): auto {.noSideEffect.} = x + y
```
- The fields of `times.DateTime` are now private, and are accessed with getters and deprecated setters.
- The `times` module now handles the default value for `DateTime` more consistently. Most procs raise an assertion error when given
an uninitialized `DateTime`, the exceptions are `==` and `$` (which returns `"Uninitialized DateTime"`). The proc `times.isInitialized`
has been added which can be used to check if a `DateTime` has been initialized.
- The `times` module now handles the default value for `DateTime` more consistently.
Most procs raise an assertion error when given
an uninitialized `DateTime`, the exceptions are `==` and `$` (which returns `"Uninitialized DateTime"`).
The proc `times.isInitialized` has been added which can be used to check if
a `DateTime` has been initialized.
- Fix a bug where calling `close` on io streams in osproc.startProcess was a noop and led to
hangs if a process had both reads from stdin and writes (eg to stdout).
@@ -105,14 +108,17 @@
["Intermediate compatibility"](https://wiki.mozilla.org/Security/Server_Side_TLS#Intermediate_compatibility_.28recommended.29)
per Mozilla's recommendation instead of `ALL`. This change should protect
users from the use of weak and insecure ciphers while still provides
adequate compatiblity with the majority of the Internet.
adequate compatibility with the majority of the Internet.
- new module `std/jsonutils` with hookable `jsonTo,toJson,fromJson` for json serialization/deserialization of custom types.
- new module `std/jsonutils` with hookable `jsonTo,toJson,fromJson` for json
serialization/deserialization of custom types.
- new proc `heapqueue.find[T](heap: HeapQueue[T], x: T): int` to get index of element ``x``.
- Add `rstgen.rstToLatex` convenience proc for `renderRstToOut` and `initRstGenerator` with `outLatex` output.
- Add `os.normalizeExe`, eg: `koch` => `./koch`.
- `macros.newLit` now preserves named vs unnamed tuples; use `-d:nimHasWorkaround14720` to keep old behavior
- Add `rstgen.rstToLatex` convenience proc for `renderRstToOut` and `initRstGenerator`
with `outLatex` output.
- Add `os.normalizeExe`, e.g.: `koch` => `./koch`.
- `macros.newLit` now preserves named vs unnamed tuples; use `-d:nimHasWorkaround14720`
to keep old behavior.
- Add `random.gauss`, that uses the ratio of uniforms method of sampling from a Gaussian distribution.
- Add `typetraits.elementType` to get element type of an iterable.
- `typetraits.$` changes: `$(int,)` is now `"(int,)"` instead of `"(int)"`;
@@ -120,14 +126,20 @@
`$((int, float), int)` is now `"((int, float), int)"` instead of `"(tuple of (int, float), int)"`
- add `macros.extractDocCommentsAndRunnables` helper
- `strformat.fmt` and `strformat.&` support `= specifier`. `fmt"{expr=}"` now expands to `fmt"expr={expr}"`.
- `strformat.fmt` and `strformat.&` support `= specifier`. `fmt"{expr=}"` now
expands to `fmt"expr={expr}"`.
- deprecations: `os.existsDir` => `dirExists`, `os.existsFile` => `fileExists`
- Add `jsre` module, [Regular Expressions for the JavaScript target.](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions)
- Made `maxLines` argument `Positive` in `logging.newRollingFileLogger`,
because negative values will result in a new file being created for each logged line which doesn't make sense.
because negative values will result in a new file being created for each logged
line which doesn't make sense.
- Changed `log` in `logging` to use proper log level on JavaScript target,
eg. `debug` uses `console.debug`, `info` uses `console.info`, `warn` uses `console.warn`, etc.
e.g. `debug` uses `console.debug`, `info` uses `console.info`, `warn` uses `console.warn`, etc.
- Tables, HashSets, SharedTables and deques don't require anymore that the passed
initial size must be a power of two - this is done internally.
Proc `rightSize` for Tables and HashSets is deprecated, as it is not needed anymore.
## Language changes
@@ -165,7 +177,7 @@
x.val = nil
```
- `getImpl` on enum type symbols now returns field syms instead of idents. This helps
with writing typed macros. Old behavior for backwards compatiblity can be restored
with writing typed macros. Old behavior for backwards compatibility can be restored
with command line switch `--useVersion:1.0`.
- ``let`` statements can now be used without a value if declared with
``importc``/``importcpp``/``importjs``/``importobjc``.
@@ -223,24 +235,29 @@ proc mydiv(a, b): int {.raises: [].} =
(likewise with other hints and warnings), which is consistent with all other bool flags.
(since 1.3.3).
- `nim doc -r main` and `nim rst2html -r main` now call openDefaultBrowser
- new hint: `--hint:msgOrigin` will show where a compiler msg (hint|warning|error) was generated; this
helps in particular when it's non obvious where it came from either because multiple locations generate
the same message, or because the message involves runtime formatting.
- new flag `--backend:js|c|cpp|objc (or -b:js etc), to change backend; can be used with any command
(eg nim r, doc, check etc); safe to re-assign.
- new flag `--doccmd:cmd` to pass additional flags for runnableExamples, eg: `--doccmd:-d:foo --threads`
- new hint: `--hint:msgOrigin` will show where a compiler msg (hint|warning|error)
was generated; this helps in particular when it's non obvious where it came from
either because multiple locations generate the same message, or because the
message involves runtime formatting.
- new flag `--backend:js|c|cpp|objc` (or -b:js etc), to change backend; can be
used with any command (eg nim r, doc, check etc); safe to re-assign.
- new flag `--doccmd:cmd` to pass additional flags for runnableExamples,
e.g.: `--doccmd:-d:foo --threads`
use `--doccmd:skip` to skip runnableExamples and rst test snippets.
- new flag `--usenimcache` to output to nimcache (whatever it resolves to after all commands are processed)
- new flag `--usenimcache` to output to nimcache (whatever it resolves to after
all commands are processed)
and avoids polluting both $pwd and $projectdir. It can be used with any command.
- `runnableExamples "-b:cpp -r:off": code` is now supported, allowing to override how an example is compiled and run,
for example to change backend or compile only.
- `nim doc` now outputs under `$projectPath/htmldocs` when `--outdir` is unspecified (with or without `--project`);
passing `--project` now automatically generates an index and enables search.
- `runnableExamples "-b:cpp -r:off": code` is now supported, allowing to override
how an example is compiled and run, for example to change backend or compile only.
- `nim doc` now outputs under `$projectPath/htmldocs` when `--outdir` is unspecified
(with or without `--project`); passing `--project` now automatically generates
an index and enables search.
See [docgen](docgen.html#introduction-quick-start) for details.
- Removed the `--oldNewlines` switch.
- Removed the `--laxStrings` switch for mutating the internal zero terminator on strings.
- Removed the `--oldast` switch.
- `$getType(untyped)` is now "untyped" instead of "expr", `$getType(typed)` is now "typed" instead of "stmt"
- `$getType(untyped)` is now "untyped" instead of "expr", `$getType(typed)` is
now "typed" instead of "stmt".
## Tool changes

11
lib/pure/collections/deques.nim
View File

@@ -67,9 +67,9 @@ const
defaultInitialSize* = 4
template initImpl(result: typed, initialSize: int) =
assert isPowerOfTwo(initialSize)
result.mask = initialSize-1
newSeq(result.data, initialSize)
let correctSize = nextPowerOfTwo(initialSize)
result.mask = correctSize-1
newSeq(result.data, correctSize)
template checkIfInitialized(deq: typed) =
when compiles(defaultInitialSize):
@@ -82,11 +82,6 @@ proc initDeque*[T](initialSize: int = 4): Deque[T] =
## Optionally, the initial capacity can be reserved via `initialSize`
## as a performance optimization.
## The length of a newly created deque will still be 0.
##
## ``initialSize`` must be a power of two (default: 4).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_.
result.initImpl(initialSize)
proc len*[T](deq: Deque[T]): int {.inline.} =

14
lib/pure/collections/hashcommon.nim
View File

@@ -30,17 +30,23 @@ proc nextTry(h, maxHash: Hash): Hash {.inline.} =
result = (h + 1) and maxHash
proc mustRehash[T](t: T): bool {.inline.} =
# If this is changed, make sure to synchronize it with `slotsNeeded` below
assert(t.dataLen > t.counter)
result = (t.dataLen * 2 < t.counter * 3) or (t.dataLen - t.counter < 4)
proc rightSize*(count: Natural): int {.inline.} =
proc slotsNeeded(count: Natural): int {.inline.} =
# Make sure to synchronize with `mustRehash` above
result = nextPowerOfTwo(count * 3 div 2 + 4)
proc rightSize*(count: Natural): int {.inline, deprecated: "Deprecated since 1.4.0".} =
## **Deprecated since Nim v1.4.0**, it is not needed anymore
## because picking the correct size is done internally.
##
## Return the value of ``initialSize`` to support ``count`` items.
##
## If more items are expected to be added, simply add that
## expected extra amount to the parameter before calling this.
#
# Make sure to synchronize with `mustRehash`
result = nextPowerOfTwo(count * 3 div 2 + 4)
result = count
template rawGetKnownHCImpl() {.dirty.} =
if t.dataLen == 0:

4
lib/pure/collections/setimpl.nim
View File

@@ -16,12 +16,12 @@ template dataLen(t): untyped = len(t.data)
include hashcommon
template initImpl(s: typed, size: int) =
assert isPowerOfTwo(size)
let correctSize = slotsNeeded(size)
when s is OrderedSet:
s.first = -1
s.last = -1
s.counter = 0
newSeq(s.data, size)
newSeq(s.data, correctSize)
template rawInsertImpl() {.dirty.} =
if data.len == 0:

30
lib/pure/collections/sets.nim
View File

@@ -94,11 +94,6 @@ include setimpl
proc init*[A](s: var HashSet[A], initialSize = defaultInitialSize) =
## Initializes a hash set.
##
## The `initialSize` parameter needs to be a power of two (default: 64).
## If you need to accept runtime values for this, you can use
## `math.nextPowerOfTwo proc <math.html#nextPowerOfTwo,int>`_ or
## `rightSize proc <#rightSize,Natural>`_ from this module.
##
## Starting from Nim v0.20, sets are initialized by default and it is
## not necessary to call this function explicitly.
##
@@ -222,7 +217,7 @@ proc toHashSet*[A](keys: openArray[A]): HashSet[A] =
assert len(b) == 5
## b == {'a', 'b', 'c', 'd', 'r'}
result = initHashSet[A](rightSize(keys.len))
result = initHashSet[A](keys.len)
for key in items(keys): result.incl(key)
iterator items*[A](s: HashSet[A]): A =
@@ -628,11 +623,6 @@ template forAllOrderedPairs(yieldStmt: untyped) {.dirty.} =
proc init*[A](s: var OrderedSet[A], initialSize = defaultInitialSize) =
## Initializes an ordered hash set.
##
## The `initialSize` parameter needs to be a power of two (default: 64).
## If you need to accept runtime values for this, you can use
## `math.nextPowerOfTwo proc <math.html#nextPowerOfTwo,int>`_ or
## `rightSize proc <#rightSize,Natural>`_ from this module.
##
## Starting from Nim v0.20, sets are initialized by default and it is
## not necessary to call this function explicitly.
##
@@ -685,7 +675,7 @@ proc toOrderedSet*[A](keys: openArray[A]): OrderedSet[A] =
assert len(b) == 5
## b == {'a', 'b', 'r', 'c', 'd'} # different than in HashSet
result = initOrderedSet[A](rightSize(keys.len))
result = initOrderedSet[A](keys.len)
for key in items(keys): result.incl(key)
proc contains*[A](s: OrderedSet[A], key: A): bool =
@@ -980,7 +970,7 @@ when isMainModule and not defined(release):
block toSeqAndString:
var a = toHashSet([2, 7, 5])
var b = initHashSet[int](rightSize(a.len))
var b = initHashSet[int](a.len)
for x in [2, 7, 5]: b.incl(x)
assert($a == $b)
#echo a
@@ -1098,20 +1088,6 @@ when isMainModule and not defined(release):
b.incl(2)
assert b.len == 1
block:
type FakeTable = object
dataLen: int
counter: int
countDeleted: int
var t: FakeTable
for i in 0 .. 32:
var s = rightSize(i)
t.dataLen = s
t.counter = i
doAssert s > i and not mustRehash(t),
"performance issue: rightSize() will not elide enlarge() at: " & $i
block missingOrExcl:
var s = toOrderedSet([2, 3, 6, 7])
assert s.missingOrExcl(4) == true

8
lib/pure/collections/sharedtables.nim
View File

@@ -207,15 +207,11 @@ proc len*[A, B](t: var SharedTable[A, B]): int =
withLock t:
result = t.counter
proc init*[A, B](t: var SharedTable[A, B], initialSize = 64) =
proc init*[A, B](t: var SharedTable[A, B], initialSize = 32) =
## creates a new hash table that is empty.
##
## This proc must be called before any other usage of `t`.
##
## `initialSize` needs to be a power of two. If you need to accept runtime
## values for this you could use the ``nextPowerOfTwo`` proc from the
## `math <math.html>`_ module or the ``rightSize`` proc from this module.
assert isPowerOfTwo(initialSize)
let initialSize = slotsNeeded(initialSize)
t.counter = 0
t.dataLen = initialSize
t.data = cast[KeyValuePairSeq[A, B]](allocShared0(

6
lib/pure/collections/tableimpl.nim
View File

@@ -121,12 +121,12 @@ template ctAnd(a, b): bool =
else: false
template initImpl(result: typed, size: int) =
let correctSize = slotsNeeded(size)
when ctAnd(declared(SharedTable), type(result) is SharedTable):
init(result, size)
init(result, correctSize)
else:
assert isPowerOfTwo(size)
result.counter = 0
newSeq(result.data, size)
newSeq(result.data, correctSize)
when compiles(result.first):
result.first = -1
result.last = -1

48
lib/pure/collections/tables.nim
View File

@@ -239,7 +239,7 @@ type
## <#newTable,int>`_.
const
defaultInitialSize* = 64
defaultInitialSize* = 32
# ------------------------------ helpers ---------------------------------
@@ -288,12 +288,6 @@ proc enlarge[A, B](t: var Table[A, B]) =
proc initTable*[A, B](initialSize = defaultInitialSize): Table[A, B] =
## Creates a new hash table that is empty.
##
## ``initialSize`` must be a power of two (default: 64).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_ or the `rightSize proc<#rightSize,Natural>`_
## from this module.
##
## Starting from Nim v0.20, tables are initialized by default and it is
## not necessary to call this function explicitly.
##
@@ -335,7 +329,7 @@ proc toTable*[A, B](pairs: openArray[(A, B)]): Table[A, B] =
let b = toTable(a)
assert b == {'a': 5, 'b': 9}.toTable
result = initTable[A, B](rightSize(pairs.len))
result = initTable[A, B](pairs.len)
for key, val in items(pairs): result[key] = val
proc `[]`*[A, B](t: Table[A, B], key: A): B =
@@ -780,12 +774,6 @@ iterator allValues*[A, B](t: Table[A, B]; key: A): B =
proc newTable*[A, B](initialSize = defaultInitialSize): <//>TableRef[A, B] =
## Creates a new ref hash table that is empty.
##
## ``initialSize`` must be a power of two (default: 64).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_ or the `rightSize proc<#rightSize,Natural>`_
## from this module.
##
## See also:
## * `newTable proc<#newTable,openArray[]>`_ for creating a `TableRef`
## from a collection of `(key, value)` pairs
@@ -1260,12 +1248,6 @@ template forAllOrderedPairs(yieldStmt: untyped) {.dirty.} =
proc initOrderedTable*[A, B](initialSize = defaultInitialSize): OrderedTable[A, B] =
## Creates a new ordered hash table that is empty.
##
## ``initialSize`` must be a power of two (default: 64).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_ or the `rightSize proc<#rightSize,Natural>`_
## from this module.
##
## Starting from Nim v0.20, tables are initialized by default and it is
## not necessary to call this function explicitly.
##
@@ -1309,7 +1291,7 @@ proc toOrderedTable*[A, B](pairs: openArray[(A, B)]): OrderedTable[A, B] =
let b = toOrderedTable(a)
assert b == {'a': 5, 'b': 9}.toOrderedTable
result = initOrderedTable[A, B](rightSize(pairs.len))
result = initOrderedTable[A, B](pairs.len)
for key, val in items(pairs): result[key] = val
proc `[]`*[A, B](t: OrderedTable[A, B], key: A): B =
@@ -1771,12 +1753,6 @@ iterator mvalues*[A, B](t: var OrderedTable[A, B]): var B =
proc newOrderedTable*[A, B](initialSize = defaultInitialSize): <//>OrderedTableRef[A, B] =
## Creates a new ordered ref hash table that is empty.
##
## ``initialSize`` must be a power of two (default: 64).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_ or the `rightSize proc<#rightSize,Natural>`_
## from this module.
##
## See also:
## * `newOrderedTable proc<#newOrderedTable,openArray[]>`_ for creating
## an `OrderedTableRef` from a collection of `(key, value)` pairs
@@ -1803,7 +1779,7 @@ proc newOrderedTable*[A, B](pairs: openArray[(A, B)]): <//>OrderedTableRef[A, B]
let b = newOrderedTable(a)
assert b == {'a': 5, 'b': 9}.newOrderedTable
result = newOrderedTable[A, B](rightSize(pairs.len))
result = newOrderedTable[A, B](pairs.len)
for key, val in items(pairs): result.add(key, val)
@@ -2251,12 +2227,6 @@ proc inc*[A](t: var CountTable[A], key: A, val: Positive = 1)
proc initCountTable*[A](initialSize = defaultInitialSize): CountTable[A] =
## Creates a new count table that is empty.
##
## ``initialSize`` must be a power of two (default: 64).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_ or the `rightSize proc<#rightSize,Natural>`_
## from this module.
##
## Starting from Nim v0.20, tables are initialized by default and it is
## not necessary to call this function explicitly.
##
@@ -2269,7 +2239,7 @@ proc initCountTable*[A](initialSize = defaultInitialSize): CountTable[A] =
proc toCountTable*[A](keys: openArray[A]): CountTable[A] =
## Creates a new count table with every member of a container ``keys``
## having a count of how many times it occurs in that container.
result = initCountTable[A](rightSize(keys.len))
result = initCountTable[A](keys.len)
for key in items(keys): result.inc(key)
proc `[]`*[A](t: CountTable[A], key: A): int =
@@ -2617,12 +2587,6 @@ proc inc*[A](t: CountTableRef[A], key: A, val = 1)
proc newCountTable*[A](initialSize = defaultInitialSize): <//>CountTableRef[A] =
## Creates a new ref count table that is empty.
##
## ``initialSize`` must be a power of two (default: 64).
## If you need to accept runtime values for this you could use the
## `nextPowerOfTwo proc<math.html#nextPowerOfTwo,int>`_ from the
## `math module<math.html>`_ or the `rightSize proc<#rightSize,Natural>`_
## from this module.
##
## See also:
## * `newCountTable proc<#newCountTable,openArray[A]>`_ for creating
## a `CountTableRef` from a collection
@@ -2634,7 +2598,7 @@ proc newCountTable*[A](initialSize = defaultInitialSize): <//>CountTableRef[A] =
proc newCountTable*[A](keys: openArray[A]): <//>CountTableRef[A] =
## Creates a new ref count table with every member of a container ``keys``
## having a count of how many times it occurs in that container.
result = newCountTable[A](rightSize(keys.len))
result = newCountTable[A](keys.len)
for key in items(keys): result.inc(key)
proc `[]`*[A](t: CountTableRef[A], key: A): int =

10
tests/collections/ttables.nim
View File

@@ -446,3 +446,13 @@ block: # https://github.com/nim-lang/Nim/issues/13496
testDel(): (let t = newOrderedTable[int, int]())
testDel(): (var t: CountTable[int])
testDel(): (let t = newCountTable[int]())
block testNonPowerOf2:
var a = initTable[int, int](7)
a[1] = 10
assert a[1] == 10
var b = initTable[int, int](9)
b[1] = 10
assert b[1] == 10

3
tests/manyloc/argument_parser/argument_parser.nim
View File

@@ -301,8 +301,7 @@ template build_specification_lookup():
OrderedTable[string, ptr Tparameter_specification] =
## Returns the table used to keep pointers to all of the specifications.
var result {.gensym.}: OrderedTable[string, ptr Tparameter_specification]
result = initOrderedTable[string, ptr Tparameter_specification](
tables.rightSize(expected.len))
result = initOrderedTable[string, ptr Tparameter_specification](expected.len)
for i in 0..expected.len-1:
for param_to_detect in expected[i].names:
if result.hasKey(param_to_detect):