Commit Graph

380 Commits

Author SHA1 Message Date
Jeroen van Rijn
248b0fe9e1 More package lines 2025-10-09 16:11:33 +02:00
Jeroen van Rijn
aec7d6480b Package lines for base32, move its tests to tests" 2025-10-09 15:27:53 +02:00
Courtney Strachan
69ad94d8a8 Followup fix to #4895 2025-10-05 19:22:04 -07:00
Courtney Strachan
6de2d6e8ca Merge branch 'odin-lang:master' into master 2025-10-06 02:41:44 +01:00
gingerBill
d3b8770318 Add #+test to base32_test.odin 2025-09-28 20:48:13 +01:00
gingerBill
7642e0a0e0 Require @(init) and @(fini) to be proc "contextless" () 2025-08-08 12:10:01 +01:00
gingerBill
af3184adc9 Change is_utf16 field to encoding and use an enum 2025-08-05 15:12:54 +01:00
gingerBill
ae02d3d02d Begin supporting string16 across the core library 2025-08-02 11:55:16 +01:00
Jeroen van Rijn
179e5b8835 Fix #4705
Allocate `doc.tokenizer`, and free it in `destroy`.
2025-06-09 15:09:04 +02:00
Feoramund
b7de15caa3 Clarify strconv.append_* to strconv.write_* 2025-06-05 16:56:00 -04:00
Laytan Laats
9eefa2006e encoding/cbor: support simd vectors 2025-06-04 22:00:02 +02:00
Laytan Laats
85224b21e6 encoding/cbor: support the matrix type 2025-06-04 21:51:12 +02:00
Wesley Kerfoot
cac18b4aba Fix incorrect CSV reader settings for example, fix typo in docs 2025-05-27 14:27:22 -04:00
Feoramund
a9df1b1cde Rename core:encoding/ansi to core:terminal/ansi 2025-05-20 19:28:06 -04:00
Laytan
d3159f74cd Merge pull request #4944 from dozn/master
[core:encoding/json] When Unmarshalling, Only Match On Struct Tags If Present
2025-05-15 22:31:34 +02:00
Barinzaya
3f5e09a0df Fixed an overflow when decoding a large CBOR slice.
The initial allocation for the slice is limited to prevent untrusted
data from forcing a huge allocation, but then the dynamic array was
created with a capacity of the unlimited length, rather than the actual
capacity of the allocation. This was causing a buffer overrun.
2025-04-29 08:19:43 -04:00
Jeroen van Rijn
4f00224dd2 Add cbor.unmarshal_from_bytes taking a []byte 2025-04-29 01:10:15 +02:00
Ely Alon
13566306d2 Fix typo in private function 2025-04-28 22:03:20 +03:00
Jeroen van Rijn
ab5ca087a7 Add comment 2025-04-19 23:44:02 +02:00
Jeroen van Rijn
062a3c2fae Fix parsing of CDATA tags (#5059)
Fixes #5054
2025-04-19 20:25:44 +02:00
Jeroen van Rijn
3287e1b0f0 Fix HXA defer warning 2025-04-07 13:19:00 +02:00
Jeroen van Rijn
f7c4c80ef3 Fix broken examples in documentation tester.
No more:
```
We could not find the procedure "pkg_foo_example :: proc()" needed to test the example created for "pkg.foo"
The following procedures were found:
   bar()
```
2025-04-05 16:36:26 +02:00
Laytan Laats
e4bc9677af fix unmarshalling bit sets in json
Fixes #4761
2025-03-22 00:20:00 +01:00
dozn
3148acf6a6 [core:encoding/json] When Unmarshalling, Only Match Struct Tags If Present 2025-03-17 01:51:04 -07:00
Courtney Strachan
dbbe96ae5c Fixed json null value parsing 2025-03-02 07:57:23 -08:00
jkenda
51b80c5a20 encoding/json: marshal enumerated arrays to objects with key-value pairs 2025-02-01 10:32:07 +01:00
Jeroen van Rijn
600e0ebed0 Fix stray space vs. tab 2025-01-12 12:13:29 +01:00
gingerBill
c7739de891 Merge pull request #4634 from dozn/patch-1
Use Struct Tags For Embedded (with `using`) Structs When Unmarshalling JSON
2025-01-06 09:33:42 +00:00
Zoltán Kéri
a4a1562905 encoding/base32: Add @(rodata) attribute to default tables
Add `@(rodata)` attribute to `ENC_TABLE` and `DEC_TABLE` to mark
them as read-only data. This places these tables in the read-only
section of the executable, protecting them from modification
during program execution.
2025-01-03 19:16:56 +01:00
Zoltán Kéri
fe88c22b1f encoding/base32: Fix RFC 4648 references and add RFC reference URL
Fix incorrect RFC 4648 section references:
- Add RFC URL reference at package level
- Update Error enum documentation to reference correct sections:
  - Invalid_Character: Section 3.3 (non-alphabet characters)
  - Invalid_Length: Section 6 (base32 block size requirements)
  - Malformed_Input: Section 3.2 (padding)
- Fix test file section references to match correct sections

This ensures all RFC references are accurate and adds a link to the
source RFC for reference.
2024-12-31 23:47:33 +01:00
Zoltán Kéri
d6f4412dc3 encoding/base32: Fix style issues for CI 2024-12-31 18:59:39 +01:00
Zoltán Kéri
3d25128520 encoding/base32: Convert files to UTF-8 with Unix line endings 2024-12-30 15:31:57 +01:00
Zoltán Kéri
5ce6990077 encoding/base32: Add proper cleanup for encoded strings in tests
Add defer delete for encoded strings across all test procedures to ensure
proper cleanup and prevent memory leaks. This completes the memory
management improvements started in 591dd876.
2024-12-30 15:26:42 +01:00
Zoltán Kéri
8292509769 encoding/base32: Add custom alphabet test case
Add test case to verify custom alphabet support. The test uses a
decimal-uppercase alphabet (0-9, A-V) to test both encoding and decoding
with custom tables, including validation. This ensures the encode and
decode functions work correctly with custom encoding tables and
validation functions as documented.
2024-12-30 15:18:38 +01:00
Zoltán Kéri
591dd8765a encoding/base32: Remove incorrect defer delete in encode()
Remove premature deallocation of the output buffer which was causing
use-after-free behavior. The returned string needs to take ownership
of this memory, but the defer delete was freeing it before the
string could be used. This fixes issues with encoding that were
introduced by overly aggressive memory cleanup in 93238db2.
2024-12-30 12:00:38 +01:00
Zoltán Kéri
0d4c0064d9 encoding/base32: Add encode->decode roundtrip test
Add test_base32_roundtrip() to verify the encode->decode roundtrip
preserves data integrity. This test helps ensure our base32 implementation
correctly handles the full encode->decode cycle without data loss or
corruption.
2024-12-30 03:03:50 +01:00
Zoltán Kéri
c9c59edc64 encoding/base32: Move tests to base32_test.odin
Move existing test procedures to a dedicated test file for better
code organization and maintainability.
2024-12-29 23:35:01 +01:00
dozn
87c159c69f Remove unnecessary string() conversion. 2024-12-28 08:13:38 -08:00
dozn
d22cb20d85 Use Struct Tags For Embedded (with using) Structs When Unmarshalling JSON
A fix for https://github.com/odin-lang/Odin/issues/4539
2024-12-28 07:48:09 -08:00
Zoltán Kéri
490f527005 encoding/base32: Expand DEC_TABLE to full 256 bytes
The decoding table was only 224 bytes which caused type mismatches when
using custom alphabets, so expand with zeroes to cover full byte range
while maintaining the same decoding logic.
2024-12-26 19:20:46 +01:00
Zoltán Kéri
88c0e62095 encoding/base32: Use ENC_TBL parameter consistently in encode()
Fix encoding to properly use provided encoding table parameter instead of
hardcoded `ENC_TABLE`.

This makes encode properly support custom alphabets as documented.
2024-12-26 14:48:02 +01:00
Zoltán Kéri
e7fb02a84a encoding/base32: Add custom validation support
Add support for custom alphabet validation through an optional validation
function parameter. The default validation follows RFC 4648 base32
alphabet rules (A-Z, 2-7).

This properly supports the documented ability to use custom alphabets.
2024-12-25 16:15:41 +01:00
Zoltán Kéri
8211a911db encoding/base32: Replace padding map with switch statement
Replace package-level map with a simple switch statement for padding
validation. This eliminates allocations we can't properly free while
maintaining the same validation logic.
2024-12-24 20:46:38 +01:00
Zoltán Kéri
e75a49f095 encoding/base32: Set optimization mode for decode() 2024-12-24 16:07:01 +01:00
Zoltán Kéri
93238db202 encoding/base32: Use consistent allocator and add proper cleanup
Fix memory handling throughout base32 package:
- Make padding map package-level constant (to avoid repeated allocs)
- Use passed allocator in encode's make() call
- Add defer delete for allocated memory in encode
- Add proper cleanup in test cases
- Fix memory cleanup of output buffers

The changes ensure consistent allocator usage and cleanup in both
implementation and tests.
2024-12-24 16:00:00 +01:00
Zoltán Kéri
f1f2ed3194 encoding/base32: Fix decode implementation per RFC 4648
Rework base32.decode() to properly handle all cases per RFC 4648:

- Fix error detection order:
  - Check minimum length first (Invalid_Length)
  - Check character validity (Invalid_Character)
  - Check padding and structure (Malformed_Input)

- Fix padding validation:
  - Add required padding length checks (2=6, 4=4, 5=3, 7=1 chars)
  - Ensure padding only appears at end
  - Fix handling of unpadded inputs

- Fix buffer handling:
  - Proper output buffer size calculation
  - Add bounds checking for buffer access
  - Add proper buffer validation

For example:
- "M" correctly returns Invalid_Length (too short)
- "mzxq====" correctly returns Invalid_Character (lowercase)
- "MZXQ=" correctly returns Malformed_Input (wrong padding)
- Unpadded input lengths must be multiples of 8

These changes make the decode function fully compliant with RFC 4648
requirements while providing proper error handling.
2024-12-24 15:52:33 +01:00
Zoltán Kéri
7672ac945a encoding/base32: Add RFC 4648 test suite
Add test suite based on RFC 4648 test vectors and validation rules:
- Add section 10 test vectors for valid encoding/decoding
- Add test cases for invalid character handling (section 3.2)
- Add test cases for padding validation (section 4)
- Add test cases for length requirements (section 6)

The test vectors verify that:
- Empty string encodes/decodes correctly
- Standard cases like "foo" -> "MZXW6===" work
- Invalid characters are rejected
- Missing or malformed padding is detected
- Invalid lengths are caught
2024-12-24 15:28:34 +01:00
Zoltán Kéri
b9338777e3 encoding/base32: Fix buffer allocation and bounds checking
Fix buffer allocation size calculation and add proper bounds checking to
ensure output buffer has sufficient space. This fixes crashes that could
occur with inputs like "AA" and other edge cases where the output buffer
was too small.

Remove #no_bounds_check as proper bounds checking is necessary for safe
error handling. The small performance trade-off is worth the improved
robustness.
2024-12-24 02:20:32 +01:00
Zoltán Kéri
8c761627c8 encoding/base32: Replace assertions with error returns
Replace assertions with proper error handling in base32.decode() to allow
programs to handle invalid input gracefully rather than crashing.

The function now returns ([]byte, Error) instead of just []byte.
2024-12-24 02:17:57 +01:00
Laytan Laats
7edd332993 fix #4536 - stack buffer overflow for size_of 0 types in struct unmarshal 2024-12-06 22:12:52 +01:00