Rearrange nre files

This commit is contained in:
Flaviu Tamas
2015-05-26 19:14:04 -04:00
parent 0d4aeec9be
commit 0527e1f9fe
47 changed files with 12 additions and 343 deletions

View File

@@ -1,5 +1,5 @@
import private.pcre as pcre
import private.util
import nre.private.pcre as pcre
import nre.private.util
import tables
import unsigned
from future import lc, `[]`

View File

@@ -1,19 +0,0 @@
Copyright (c) 2015 Flaviu Tamas
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View File

@@ -1,258 +0,0 @@
What is NRE?
============
A regular expression library for Nim using PCRE to do the hard work.
Why?
----
The `re.nim <http://nim-lang.org/re.html>`__ module that
`Nim <http://nim-lang.org/>`__ provides in its standard library is
inadequate:
- It provides only a limited number of captures, while the underling
library (PCRE) allows an unlimited number.
- Instead of having one proc that returns both the bounds and
substring, it has one for the bounds and another for the substring.
- If the splitting regex is empty (``""``), then it returns the input
string instead of following `Perl <https://ideone.com/dDMjmz>`__,
`Javascript <http://jsfiddle.net/xtcbxurg/>`__, and
`Java <https://ideone.com/hYJuJ5>`__'s precedent of returning a list
of each character (``"123".split(re"") == @["1", "2", "3"]``).
Other Notes
-----------
By default, NRE compiles its own PCRE. If this is undesirable, pass
``-d:pcreDynlib`` to use whatever dynamic library is available on the
system. This may have unexpected consequences if the dynamic library
doesnt have certain features enabled.
Types
-----
``type Regex* = ref object``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Represents the pattern that things are matched against, constructed with
``re(string)``. Examples: ``re"foo"``, ``re(r"(*ANYCRLF)(?x)foo #
comment".``
``pattern: string``
the string that was used to create the pattern.
``captureCount: int``
the number of captures that the pattern has.
``captureNameId: Table[string, int]``
a table from the capture names to their numeric id.
Options
.......
The following options may appear anywhere in the pattern, and they affect
the rest of it.
- ``(?i)`` - case insensitive
- ``(?m)`` - multi-line: ``^`` and ``$`` match the beginning and end of
lines, not of the subject string
- ``(?s)`` - ``.`` also matches newline (*dotall*)
- ``(?U)`` - expressions are not greedy by default. ``?`` can be added
to a qualifier to make it greedy
- ``(?x)`` - whitespace and comments (``#``) are ignored (*extended*)
- ``(?X)`` - character escapes without special meaning (``\w`` vs.
``\a``) are errors (*extra*)
One or a combination of these options may appear only at the beginning
of the pattern:
- ``(*UTF8)`` - treat both the pattern and subject as UTF-8
- ``(*UCP)`` - Unicode character properties; ``\w`` matches ``я``
- ``(*U)`` - a combination of the two options above
- ``(*FIRSTLINE*)`` - fails if there is not a match on the first line
- ``(*NO_AUTO_CAPTURE)`` - turn off auto-capture for groups;
``(?<name>...)`` can be used to capture
- ``(*CR)`` - newlines are separated by ``\r``
- ``(*LF)`` - newlines are separated by ``\n`` (UNIX default)
- ``(*CRLF)`` - newlines are separated by ``\r\n`` (Windows default)
- ``(*ANYCRLF)`` - newlines are separated by any of the above
- ``(*ANY)`` - newlines are separated by any of the above and Unicode
newlines:
single characters VT (vertical tab, U+000B), FF (form feed, U+000C),
NEL (next line, U+0085), LS (line separator, U+2028), and PS
(paragraph separator, U+2029). For the 8-bit library, the last two
are recognized only in UTF-8 mode.
— man pcre
- ``(*JAVASCRIPT_COMPAT)`` - JavaScript compatibility
- ``(*NO_STUDY)`` - turn off studying; study is enabled by default
For more details on the leading option groups, see the `Option
Setting <http://man7.org/linux/man-pages/man3/pcresyntax.3.html#OPTION_SETTING>`__
and the `Newline
Convention <http://man7.org/linux/man-pages/man3/pcresyntax.3.html#NEWLINE_CONVENTION>`__
sections of the `PCRE syntax
manual <http://man7.org/linux/man-pages/man3/pcresyntax.3.html>`__.
``type RegexMatch* = object``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Usually seen as Option[RegexMatch], it represents the result of an
execution. On failure, it is ``None[RegexMatch]``, but if you want
automated derefrence, import ``optional_t.nonstrict``. The available
fields are as follows:
``pattern: Regex``
the pattern that is being matched
``str: string``
the string that was matched against
``captures[]: string``
the string value of whatever was captured at that id. If the value
is invalid, then behavior is undefined. If the id is ``-1``, then
the whole match is returned. If the given capture was not matched,
``nil`` is returned.
- ``"abc".match(re"(\w)").captures[0] == "a"``
- ``"abc".match(re"(?<letter>\w)").captures["letter"] == "a"``
- ``"abc".match(re"(\w)\w").captures[-1] == "ab"``
``captureBounds[]: Option[Slice[int]]``
gets the bounds of the given capture according to the same rules as
the above. If the capture is not filled, then ``None`` is returned.
The bounds are both inclusive.
- ``"abc".match(re"(\w)").captureBounds[0] == 0 .. 0``
- ``"abc".match(re"").captureBounds[-1] == 0 .. -1``
- ``"abc".match(re"abc").captureBounds[-1] == 0 .. 2``
``match: string``
the full text of the match.
``matchBounds: Slice[int]``
the bounds of the match, as in ``captureBounds[]``
``(captureBounds|captures).toTable``
returns a table with each named capture as a key.
``(captureBounds|captures).toSeq``
returns all the captures by their number.
``$: string``
same as ``match``
``type RegexInternalError* = ref object of RegexException``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Internal error in the module, this probably means that there is a bug
``type InvalidUnicodeError* = ref object of RegexException``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Thrown when matching fails due to invalid unicode in strings
``type SyntaxError* = ref object of RegexException``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Thrown when there is a syntax error in the
regular expression string passed in
``type StudyError* = ref object of RegexException``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Thrown when studying the regular expression failes
for whatever reason. The message contains the error
code.
Operations
----------
``proc match*(str: string, pattern: Regex, start = 0, endpos = int.high): Option[RegexMatch]``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Like ```find(...)`` <#proc-find>`__, but anchored to the start of the
string. This means that ``"foo".match(re"f") == true``, but
``"foo".match(re"o") == false``.
``iterator findIter*(str: string, pattern: Regex, start = 0, endpos = int.high): RegexMatch``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Works the same as ```find(...)`` <#proc-find>`__, but finds every
non-overlapping match. ``"2222".find(re"22")`` is ``"22", "22"``, not
``"22", "22", "22"``.
Arguments are the same as ```find(...)`` <#proc-find>`__
Variants:
- ``proc findAll(...)`` returns a ``seq[string]``
``proc find*(str: string, pattern: Regex, start = 0, endpos = int.high): Option[RegexMatch]``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Finds the given pattern in the string between the end and start
positions.
``start``
The start point at which to start matching. ``|abc`` is ``0``;
``a|bc`` is ``1``
``endpos``
The maximum index for a match; ``int.high`` means the end of the
string, otherwise its an inclusive upper bound.
``proc split*(str: string, pattern: Regex, maxSplit = -1, start = 0): seq[string]``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Splits the string with the given regex. This works according to the
rules that Perl and Javascript use:
- If the match is zero-width, then the string is still split:
``"123".split(r"") == @["1", "2", "3"]``.
- If the pattern has a capture in it, it is added after the string
split: ``"12".split(re"(\d)") == @["", "1", "", "2", ""]``.
- If ``maxsplit != -1``, then the string will only be split
``maxsplit - 1`` times. This means that there will be ``maxsplit``
strings in the output seq.
``"1.2.3".split(re"\.", maxsplit = 2) == @["1", "2.3"]``
``start`` behaves the same as in ```find(...)`` <#proc-find>`__.
``proc replace*(str: string, pattern: Regex, subproc: proc (match: RegexMatch): string): string``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Replaces each match of Regex in the string with ``sub``, which should
never be or return ``nil``.
If ``sub`` is a ``proc (RegexMatch): string``, then it is executed with
each match and the return value is the replacement value.
If ``sub`` is a ``proc (string): string``, then it is executed with the
full text of the match and and the return value is the replacement
value.
If ``sub`` is a string, the syntax is as follows:
- ``$$`` - literal ``$``
- ``$123`` - capture number ``123``
- ``$foo`` - named capture ``foo``
- ``${foo}`` - same as above
- ``$1$#`` - first and second captures
- ``$#`` - first capture
- ``$0`` - full match
If a given capture is missing, a ``ValueError`` exception is thrown.
``proc escapeRe*(str: string): string``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Escapes the string so it doesnt match any special characters.
Incompatible with the Extra flag (``X``).

View File

@@ -1,39 +0,0 @@
dependencies:
pre:
- |
if [ ! -x ~/nim/bin/nim ]; then
sudo apt-get install gcc
git clone -b devel --depth 1 git://github.com/araq/nim ~/nim/
git clone -b devel --depth 1 git://github.com/nim-lang/csources ~/nim/csources/
cd ~/nim/csources; sh build.sh; cd ..
rm -rf csources
bin/nim c koch
./koch boot -d:release
ln -fs ~/nim/bin/nim ~/bin/nim
else
cd ~/nim
git fetch origin
if ! git merge FETCH_HEAD | grep "Already up-to-date"; then
bin/nim c koch
./koch boot -d:release
fi
fi
- |
if [ ! -x ~/.nimble/bin/nimble ]; then
git clone --depth 1 git://github.com/nim-lang/nimble ~/nimble/
cd ~/nimble/
nim c src/nimble.nim
./src/nimble install
ln -fs ~/.nimble/bin/nimble ~/bin/nimble
fi
- nimble update
- nimble build
cache_directories:
- "~/bin/"
- "~/nim/"
- "~/.nimble/"
test:
override:
- ./runtests.sh

View File

@@ -1,11 +0,0 @@
[Package]
name = "nre"
author = "Flaviu Tamas"
version = "0.6.1"
description = "Yet another PCRE library"
license = "MIT"
srcDir = "src"
[Deps]
Requires: "nim >= 0.10.0"
Requires: "optional_t >= 1.2.0"

View File

@@ -1,4 +1,4 @@
when defined(pcreDynlib):
when not defined(pcreStaticlib):
const pcreHeader = "<pcre.h>"
when not defined(pcreDll):
when hostOS == "windows":

View File

@@ -1,3 +0,0 @@
#!/bin/sh
nim c --path:src -r --verbosity:0 --hints:off --linedir:on --debuginfo \
--stacktrace:on --linetrace:on "$@" ./test/testall.nim

View File

@@ -1,9 +0,0 @@
import nre
import init
import captures
import find
import split
import match
import replace
import escape
import misc

Binary file not shown.

Before

Width:  |  Height:  |  Size: 41 KiB

View File

@@ -1 +0,0 @@
<svg xmlns:svg="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg" width="627" height="140" version="1.1"><style>.s0{-inkscape-font-specification:DejaVu Sans Mono;fill:#c17d11;font-family:DejaVu Sans Mono;font-size:35.7;}.s1{-inkscape-font-specification:DejaVu Sans Mono;fill:#c00;font-family:DejaVu Sans Mono;font-size:35.7;}.s2{-inkscape-font-specification:DejaVu Sans Mono;fill:#4e9a06;font-family:DejaVu Sans Mono;font-size:35.7;}</style><g transform="translate(0,-912.36217)"><g transform="matrix(3.3601391,0,0,3.3601391,-52.384026,-75.180678)" style="fill:#000;font-family:Sans;font-size:40;letter-spacing:0;line-height:125;word-spacing:0"><path d="m77.2 293.9 125 0 0 41.7-125 0z" style="fill:#e9b96e;opacity:0.5"/><path d="m91.3 296 0 9.7-3 0 0-9.7 3 0 -6.7 0 0 9.7-3 0 0-9.7 3 0" class="s0"/><path d="m93.5 298.6 7.4 0 0 2.5-4.2 0 0 26.9 4.2 0 0 2.5-7.4 0 0-31.8" class="s1"/><path d="m109.4 314.4c0-0.6 0.2-1.2 0.7-1.7 0.5-0.5 1-0.7 1.6-0.7 0.7 0 1.2 0.2 1.7 0.7 0.5 0.5 0.7 1 0.7 1.7 0 0.7-0.2 1.2-0.7 1.7-0.5 0.5-1 0.7-1.7 0.7-0.7 0-1.2-0.2-1.6-0.7-0.4-0.4-0.6-1-0.6-1.7m2.3-10.6c-1.6 0-2.9 0.9-3.7 2.7-0.8 1.8-1.2 4.5-1.2 8.1 0 3.6 0.4 6.3 1.2 8.1 0.8 1.8 2 2.7 3.7 2.7 1.7 0 2.9-0.9 3.7-2.7 0.8-1.8 1.2-4.5 1.2-8.1 0-3.6-0.4-6.3-1.2-8.1-0.8-1.8-2-2.7-3.7-2.7m0-2.8c2.8 0 4.9 1.1 6.3 3.4 1.4 2.3 2.1 5.6 2.1 10.1 0 4.4-0.7 7.8-2.1 10.1-1.4 2.3-3.5 3.4-6.3 3.4-2.8 0-4.9-1.1-6.3-3.4-1.4-2.3-2.1-5.6-2.1-10.1 0-4.5 0.7-7.8 2.1-10.1 1.4-2.3 3.5-3.4 6.3-3.4" class="s2"/><path d="m124.6 313.1 9.1 0 0 2.9-9.1 0 0-2.9" class="s1"/><path d="m146.4 316.1c1.5 0 2.7-0.5 3.5-1.6 0.9-1.1 1.3-2.6 1.3-4.5 0-1.9-0.4-3.4-1.3-4.5-0.8-1.1-2-1.6-3.5-1.6-1.6 0-2.7 0.5-3.5 1.6-0.8 1-1.2 2.5-1.2 4.6 0 2 0.4 3.5 1.2 4.6 0.8 1 2 1.5 3.5 1.5m-6.3 11 0-3.2c0.7 0.4 1.5 0.8 2.3 1 0.8 0.2 1.7 0.3 2.6 0.3 2.2 0 3.9-0.8 5.1-2.5 1.2-1.7 1.7-4.2 1.7-7.4-0.5 1.2-1.3 2.1-2.3 2.7-1 0.6-2.1 0.9-3.4 0.9-2.5 0-4.5-0.8-5.8-2.3-1.4-1.5-2.1-3.7-2.1-6.6 0-2.8 0.7-5 2.1-6.5 1.4-1.5 3.4-2.3 6-2.3 3 0 5.2 1.1 6.6 3.3 1.4 2.2 2.1 5.6 2.1 10.3 0 4.4-0.8 7.7-2.5 10-1.7 2.3-4.1 3.5-7.4 3.5-0.8 0-1.7-0.1-2.6-0.3-0.9-0.2-1.7-0.4-2.5-0.8" class="s2"/><path d="m165.2 298.6 0 31.8-7.4 0 0-2.5 4.2 0 0-26.9-4.2 0 0-2.5 7.4 0M179.9 305.3l0 7.7 7.8 0 0 3-7.8 0 0 7.7-2.9 0 0-7.7-7.7 0 0-3 7.7 0 0-7.7 2.9 0" class="s1"/><path d="m197.7 296 0 9.7-3 0 0-9.7 3 0 -6.7 0 0 9.7-3 0 0-9.7 3 0" class="s0"/><g transform="translate(-2.0396636,0.18413477)" style="fill:#000;font-family:Sans;font-size:35.7"><path d="m38.6 313.9 0 11.8-3.2 0 0-11.7c0-1.8-0.4-3.2-1.1-4.2-0.7-0.9-1.8-1.4-3.2-1.4-1.7 0-3.1 0.6-4.1 1.7-1 1.1-1.5 2.6-1.5 4.5l0 11-3.2 0 0-19.5 3.2 0 0 3c0.8-1.2 1.7-2.1 2.7-2.6 1-0.6 2.2-0.9 3.6-0.9 2.2 0 3.9 0.7 5.1 2.1 1.2 1.4 1.7 3.4 1.7 6.1M56.4 309.1c-0.4-0.2-0.8-0.4-1.2-0.5-0.4-0.1-0.9-0.2-1.4-0.2-1.8 0-3.2 0.6-4.2 1.8-1 1.2-1.4 2.9-1.4 5.1l0 10.3-3.2 0 0-19.5 3.2 0 0 3c0.7-1.2 1.6-2.1 2.6-2.6 1.1-0.6 2.4-0.9 3.9-0.9 0.2 0 0.5 0 0.7 0.1 0.3 0 0.6 0.1 0.9 0.1l0 3.3M75.7 315.1l0 1.6-14.8 0c0.1 2.2 0.8 3.9 2 5.1 1.2 1.2 2.9 1.7 5 1.7 1.2 0 2.4-0.2 3.6-0.5 1.2-0.3 2.3-0.8 3.5-1.4l0 3c-1.2 0.5-2.3 0.9-3.5 1.1-1.2 0.3-2.4 0.4-3.7 0.4-3.1 0-5.6-0.9-7.4-2.7-1.8-1.8-2.7-4.3-2.7-7.4 0-3.2 0.9-5.7 2.6-7.6 1.7-1.9 4.1-2.8 7-2.8 2.6 0 4.7 0.8 6.2 2.5 1.5 1.7 2.3 4 2.3 6.9m-3.2-0.9c0-1.8-0.5-3.2-1.5-4.2-1-1-2.2-1.6-3.8-1.6-1.8 0-3.2 0.5-4.3 1.5-1.1 1-1.7 2.4-1.8 4.3l11.4 0"/></g></g></g></svg>

Before

Width:  |  Height:  |  Size: 3.3 KiB

9
tests/stdlib/tnre.nim Normal file
View File

@@ -0,0 +1,9 @@
import nre
import nre.init
import nre.captures
import nre.find
import nre.split
import nre.match
import nre.replace
import nre.escape
import nre.misc