a0b33f9408
* Unwind just the "pseudorandom probing" (whole hash-code-keyed variable stride double hashing) part of recent sets & tables changes (which has still been causing bugs over a month later (e.g., two days ago https://github.com/nim-lang/Nim/issues/13794) as well as still having several "figure this out" implementation question comments in them (see just diffs of this PR). This topic has been discussed in many places: https://github.com/nim-lang/Nim/issues/13393 https://github.com/nim-lang/Nim/pull/13418 https://github.com/nim-lang/Nim/pull/13440 https://github.com/nim-lang/Nim/issues/13794 Alternative/non-mandatory stronger integer hashes (or vice-versa opt-in identity hashes) are a better solution that is more general (no illusion of one hard-coded sequence solving all problems) while retaining the virtues of linear probing such as cache obliviousness and age-less tables under delete-heavy workloads (still untested after a month of this change). The only real solution for truly adversarial keys is a hash keyed off of data unobservable to attackers. That all fits better with a few families of user-pluggable/define-switchable hashes which can be provided in a separate PR more about `hashes.nim`. This PR carefully preserves the better (but still hard coded!) probing of the `intsets` and other recent fixes like `move` annotations, hash order invariant tests, `intsets.missingOrExcl` fixing, and the move of `rightSize` into `hashcommon.nim`. * Fix `data.len` -> `dataLen` problem. * This is an alternate resolution to https://github.com/nim-lang/Nim/issues/13393 (which arguably could be resolved outside the stdlib). Add version1 of Wang Yi's hash specialized to 8 byte integers. This gives simple help to users having trouble with overly colliding hash(key)s. I.e., A) `import hashes; proc hash(x: myInt): Hash = hashWangYi1(int(x))` in the instantiation context of a `HashSet` or `Table` or B) more globally, compile with `nim c -d:hashWangYi1`. No hash can be all things to all use cases, but this one is A) vetted to scramble well by the SMHasher test suite (a necessarily limited but far more thorough test than prior proposals here), B) only a few ALU ops on many common CPUs, and C) possesses an easy via "grade school multi-digit multiplication" fall back for weaker deployment contexts. Some people might want to stampede ahead unbridled, but my view is that a good plan is to A) include this in the stdlib for a release or three to let people try it on various key sets nim-core could realistically never access/test (maybe mentioning it in the changelog so people actually try it out), B) have them report problems (if any), C) if all seems good, make the stdlib more novice friendly by adding `hashIdentity(x)=x` and changing the default `hash() = hashWangYi1` with some `when defined` rearranging so users can `-d:hashIdentity` if they want the old behavior back. This plan is compatible with any number of competing integer hashes if people want to add them. I would strongly recommend they all *at least* pass the SMHasher suite since the idea here is to become more friendly to novices who do not generally understand hashing failure modes. * Re-organize to work around `when nimvm` limitations; Add some tests; Add a changelog.md entry. * Add less than 64-bit CPU when fork. * Fix decl instead of call typo. * First attempt at fixing range error on 32-bit platforms; Still do the arithmetic in doubled up 64-bit, but truncate the hash to the lower 32-bits, but then still return `uint64` to be the same. So, type correct but truncated hash value. Update `thashes.nim` as well. * A second try at making 32-bit mode CI work. * Use a more systematic identifier convention than Wang Yi's code. * Fix test that was wrong for as long as `toHashSet` used `rightSize` (a very long time, I think). `$a`/`$b` depend on iteration order which varies with table range reduced hash order which varies with range for some `hash()`. With 3 elements, 3!=6 is small and we've just gotten lucky with past experimental `hash()` changes. An alternate fix here would be to not stringify but use the HashSet operators, but it is not clear that doesn't alter the "spirit" of the test. * Fix another stringified test depending upon hash order. * Oops - revert the string-keyed test. * Fix another stringify test depending on hash order. * Add a better than always zero `defined(js)` branch. * It turns out to be easy to just work all in `BigInt` inside JS and thus guarantee the same low order bits of output hashes (for `isSafeInteger` input numbers). Since `hashWangYi1` output bits are equally random in all their bits, this means that tables will be safely scrambled for table sizes up to 2**32 or 4 gigaentries which is probably fine, as long as the integer keys are all < 2**53 (also likely fine). (I'm unsure why the infidelity with C/C++ back ends cut off is 32, not 53 bits.) Since HashSet & Table only use the low order bits, a quick corollary of this is that `$` on most int-keyed sets/tables will be the same in all the various back ends which seems a nice-to-have trait. * These string hash tests fail for me locally. Maybe this is what causes the CI hang for testament pcat collections? * Oops. That failure was from me manually patching string hash in hashes. Revert. * Import more test improvements from https://github.com/nim-lang/Nim/pull/13410 * Fix bug where I swapped order when reverting the test. Ack. * Oh, just accept either order like more and more hash tests. * Iterate in the same order. * `return` inside `emit` made us skip `popFrame` causing weird troubles. * Oops - do Windows branch also. * `nimV1hash` -> multiply-mnemonic, type-scoped `nimIntHash1` (mnemonic resolutions are "1 == identity", 1 for Nim Version 1, 1 for first/simplest/fastest in a series of possibilities. Should be very easy to remember.) * Re-organize `when nimvm` logic to be a strict `when`-`else`. * Merge other changes. * Lift constants to a common area. * Fall back to identity hash when `BigInt` is unavailable. * Increase timeout slightly (probably just real-time perturbation of CI system performance).
Nim 
This repository contains the Nim compiler, Nim's stdlib, tools and documentation. For more information about Nim, including downloads and documentation for the latest release, check out Nim's website or bleeding edge docs.
Community
- The forum - the best place to ask questions and to discuss Nim.
- #nim IRC Channel (Freenode) - a place to discuss Nim in real-time. Also where most development decisions get made.
- Gitter - an additional place to discuss Nim in real-time. There is a bridge between Gitter and the IRC channel.
- Telegram - an additional place to discuss Nim in real-time. There is the official Telegram channel.
- Stack Overflow - a popular Q/A site for programming related topics that includes posts about Nim.
- Github Wiki - Misc user-contributed content.
Compiling
The compiler currently officially supports the following platform and architecture combinations:
- Windows (Windows XP or greater) - x86 and x86_64
- Linux (most, if not all, distributions) - x86, x86_64, ppc64 and armv6l
- Mac OS X (10.04 or greater) - x86, x86_64 and ppc64
More platforms are supported, however they are not tested regularly and they may not be as stable as the above-listed platforms.
Compiling the Nim compiler is quite straightforward if you follow these steps:
First, the C source of an older version of the Nim compiler is needed to
bootstrap the latest version because the Nim compiler itself is written in the
Nim programming language. Those C sources are available within the
nim-lang/csources repository.
Next, to build from source you will need:
- A C compiler such as
gcc3.x/later or an alternative such asclang,Visual C++orIntel C++. It is recommended to usegcc3.x or later. - Either
gitorwgetto download the needed source repositories. - The
build-essentialpackage when usinggccon Ubuntu (and likely other distros as well).
Then, if you are on a *nix system or Windows, the following steps should compile
Nim from source using gcc, git and the koch build tool.
Note: The following commands are for the development version of the compiler. For most users, installing the latest stable version is enough. Check out the installation instructions on the website to do so: https://nim-lang.org/install.html.
For package maintainers: see packaging guidelines.
First get Nim from github:
git clone https://github.com/nim-lang/Nim.git
cd Nim
Next run the appropriate build shell script for your platform:
build_all.sh(Linux, Mac)build_all.bat(Windows)
Finally, once you have finished the build steps (on Windows, Mac or Linux) you
should add the bin directory to your PATH.
Koch
koch is the build tool used to build various parts of Nim and to generate
documentation and the website, among other things. The koch tool can also
be used to run the Nim test suite.
Assuming that you added Nim's bin directory to your PATH, you may execute
the tests using ./koch tests. The tests take a while to run, but you
can run a subset of tests by specifying a category (for example
./koch tests cat async).
For more information on the koch build tool please see the documentation
within the doc/koch.rst file.
Nimble
nimble is Nim's package manager. To learn more about it, see the
nim-lang/nimble repository.
Contributors
This project exists thanks to all the people who contribute.
Contributing
See detailed contributing guidelines. We welcome all contributions to Nim regardless of how small or large they are. Everything from spelling fixes to new modules to be included in the standard library are welcomed and appreciated. Before you start contributing, you should familiarize yourself with the following repository structure:
bin/,build/- these directories are empty, but are used when Nim is built.compiler/- the compiler source code. Also includes nimfix, and plugins withincompiler/nimfixandcompiler/pluginsrespectively.nimsuggest- the nimsuggest tool that previously lived in thenim-lang/nimsuggestrepository.config/- the configuration for the compiler and documentation generator.doc/- the documentation files in reStructuredText format.lib/- the standard library, including:pure/- modules in the standard library written in pure Nim.impure/- modules in the standard library written in pure Nim with dependencies written in other languages.wrappers/- modules which wrap dependencies written in other languages.
tests/- contains categorized tests for the compiler and standard library.tools/- the tools includingniminstandnimweb(mostly invoked viakoch).koch.nim- tool used to bootstrap Nim, generate C sources, build the website, and generate the documentation.
If you are not familiar with making a pull request using GitHub and/or git, please read this guide.
Ideally you should make sure that all tests pass before submitting a pull request.
However, if you are short on time, you can just run the tests specific to your
changes by only running the corresponding categories of tests. Travis CI verifies
that all tests pass before allowing the pull request to be accepted, so only
running specific tests should be harmless.
Integration tests should go in tests/untestable.
If you're looking for ways to contribute, please look at our issue tracker.
There are always plenty of issues labelled Easy; these should
be a good starting point for an initial contribution to Nim.
You can also help with the development of Nim by making donations. Donations can be made using:
If you have any questions feel free to submit a question on the Nim forum, or via IRC on the #nim channel.
Backers
Thank you to all our backers! [Become a backer]
Sponsors
Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]
You can also see a list of all our sponsors/backers from various payment services on the sponsors page of our website.
License
The compiler and the standard library are licensed under the MIT license, except for some modules which explicitly state otherwise. As a result you may use any compatible license (essentially any license) for your own programs developed with Nim. You are explicitly permitted to develop commercial applications using Nim.
Please read the copying.txt file for more details.
Copyright © 2006-2020 Andreas Rumpf, all rights reserved.