mirrors/Odin
1
0
Fork 0
mirror of https://github.com/odin-lang/Odin.git synced 2026-01-07 21:43:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #4467 from clankill3r/patch-1

Browse Source 
Update scanner.odin
This commit is contained in:
gingerBill
2024-11-08 13:38:29 +00:00
committed by GitHub
parent 20a8c97d68 b492b73ff2
commit 4890c3681c
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
core/text/scanner/scanner.odin
View File

@@ -2,7 +2,7 @@
// It takes a string providing the source, which then can be tokenized through
// repeated calls to the scan procedure.
// For compatibility with existing tooling and languages, the NUL character is not allowed.
// If an UTF-8 encoded byte order mark (BOM) is the first character in the first character in the source, it will be discarded.
// If an UTF-8 encoded byte order mark (BOM) is the first character in the source, it will be discarded.
//
// By default, a Scanner skips white space and Odin comments and recognizes all literals defined by the Odin programming language specification.
// A Scanner may be customized to recognize only a subset of those literals and to recognize different identifiers and white space characters.