Commit Graph

225 Commits

Author SHA1 Message Date
Feoramund
f0f3d72818 Fix syscall-based virtual memory allocation on NetBSD 2025-05-12 12:29:20 -04:00
Feoramund
4b08e8cfff Support getting page sizes at runtime 2025-05-11 14:29:58 -04:00
Feoramund
97a08e5c15 Pass argc and argv for shared libraries on UNIX-likes
This is to support getting `auxv` on SysV platforms.
2025-05-11 07:24:41 -04:00
Feoramund
674853f16c Remove heap coverage code in favor of external tooling 2025-05-09 18:15:22 -04:00
Feoramund
acc877b3a7 Don't free virtual memory when ThreadSanitizer is active
This is a workaround for a lack of an ability to tell TSan that it
should clear any state it has about certain memory ranges to prevent
false positives.
2025-05-09 17:56:57 -04:00
Feoramund
78006adf67 Use AddressSanitizer with new heap allocator 2025-05-09 17:56:55 -04:00
Feoramund
b056b9e130 Rewrite the heap allocator
This is a total rewrite after having experimented with a variety of
techniques including a dual-allocator strategy for small and large size
classes where the large class allocator had a coalescing mechanism.

This allocator design is much closer to `mimalloc` in spirit and
performs almost twice as fast as the original bitmap-based design.

Additionally, several bugs have been fixed. The most important one being
the lock-free synchronization method used for remote frees. This design
uses a single atomic pointer that is doubly-tagged to pass remote frees
either to the heap or to the slab, depending on the ownership status.
2025-05-09 17:56:36 -04:00
Feoramund
2a33e1c1bb Forbid freeing of superpages that were not fully used once 2025-05-08 11:06:03 -04:00
Feoramund
94bfdd489f Use cache-friendly array to update next free slab index 2025-05-08 11:06:03 -04:00
Feoramund
920487be04 Zero only needed part of memory when changing size categories 2025-05-08 11:06:03 -04:00
Feoramund
92fdb4d79a Cache contiguous free slabs per heap superpage 2025-05-08 11:06:03 -04:00
Feoramund
e038c3cf4e Make superpage caching linear time instead of quadratic 2025-05-08 11:06:03 -04:00
Feoramund
dcd9bd7888 Make slab map caching linear time instead of quadratic 2025-05-08 11:06:03 -04:00
Feoramund
fa1f6abe78 Don't atomically subtract zero from remote_free_count 2025-05-08 11:06:03 -04:00
Feoramund
a0425ca220 Do not return superpages that had remote frees to the OS 2025-05-08 11:06:03 -04:00
Feoramund
acd5d0d40b Fix slab iteration bug in the remote free collection phase
Previously, a wide slab could be freed during this phase which could
trigger the superpage itself to be freed, resulting in iteration across
potentially invalid memory if the superpage was returned to the host and
not placed in the orphanage.

`heap_free_wide_slab` has had its responsibilities reduced to just
freeing the wide slab, and `heap_free_superpage_if_empty_and_unused` has
been added to better clarify what is happening where.
2025-05-08 11:06:02 -04:00
Feoramund
2d0547dcaf Remove unneeded call to min
At this point, we know we're expanding the wide slab, so `old_size` will
always be the right size.
2025-05-08 11:06:02 -04:00
Feoramund
44a7015609 Fix in-place expansion of wide slabs 2025-05-08 11:06:02 -04:00
Feoramund
8638bd3b89 Fix slab iteration bug in heap_cache_register_superpage
Increment of `i` is handled near the start of this loop.
2025-05-08 11:06:02 -04:00
Feoramund
ee3d859c6b Remove HEAP_PANIC_ON_FREE_NIL 2025-05-08 11:06:02 -04:00
Feoramund
f5226d01f7 Revert to unoptimized behavior on Darwin 2025-05-08 11:06:02 -04:00
Feoramund
4a31e1f635 Ensure remapped memory has desired protection on Darwin 2025-05-08 11:06:02 -04:00
Feoramund
0947914e0d Use mach_task_self_ global instead 2025-05-08 11:06:02 -04:00
Feoramund
247ce3cc97 Favor rescheduling superpage cache entry instead
This should be more robust during merges that happen simultaneously with
remote frees.

Previously, we would preserve the cache entry, but there is an edge case
where a slab of a lower index could be remotely freed and missed during
a merge.
2025-05-08 11:06:02 -04:00
Feoramund
e86bd35387 Optimize virtual memory resizing on Darwin 2025-05-08 11:06:02 -04:00
Feoramund
c52c29856c Unify heap allocators
- Simplify the malloc-based allocator
- Correct the type signatures on malloc/calloc/realloc
- Make `heap_alloc/heap_free/heap_resize` API consistent across platforms
2025-05-08 11:06:02 -04:00
Feoramund
30e600699d Strengthen order to prevent reordering 2025-05-08 11:06:01 -04:00
Feoramund
0532b73563 Simplify loop 2025-05-08 11:06:01 -04:00
Feoramund
541443fe33 Don't build feoramalloc on web platforms
Their allocation pathways are already tuned for their respective
platforms.
2025-05-08 11:06:01 -04:00
Feoramund
e9bf350cca Add heap allocator exception for Orca
This is a copy of the libc malloc-based allocator that we previously
had. It's preferable to use the Orca runtime allocator over our own.
2025-05-08 11:06:01 -04:00
Feoramund
ef72a373b2 Use VM_INHERIT_COPY instead
This aligns with the use of `MAP_PRIVATE` in `mmap`-based
implementations.
2025-05-08 11:06:01 -04:00
Feoramund
4c59019a57 Restrict superpage allocation on Darwin to AMD64 and 2MiB
There's a possibility that Darwin does not support anything else, in
terms of size or architecture.
2025-05-08 11:06:01 -04:00
Feoramund
1e62f2c8c4 Rearrange slab iteration conditionals 2025-05-08 11:06:01 -04:00
Feoramund
76a2877d7d Hoist reused calculations out into variables 2025-05-08 11:06:01 -04:00
Feoramund
17b38c1bca Strengthen Consume to Acquire in heap allocator 2025-05-08 11:06:01 -04:00
Feoramund
6d99d9571c Fix indentation 2025-05-08 11:06:01 -04:00
Feoramund
a4a776634c Add native heap allocator
- Add the test bench for the allocator
- Move old allocator code to the test bench
- Fix `heap_resize` usage in `os2/env_linux.odin` to fit new API
  requiring `old_size`
2025-05-08 11:06:00 -04:00
Feoramund
f142043db9 Add virtual memory procedures to base 2025-05-08 11:06:00 -04:00
Feoramund
13dca77d3d Add get_current_thread_id to base 2025-05-08 11:06:00 -04:00
gingerBill
92df892f25 Merge pull request #5064 from harold-b/hb/objc-classes
Add support for Objective-C class implementation
2025-05-08 12:58:33 +01:00
Lucas Perlind
46e0c7ad74 Cleanup 2025-05-07 11:30:58 +10:00
Lucas Perlind
fd05f55691 Fix wasi_wasm 2025-05-06 15:23:44 +10:00
Lucas Perlind
83bc2d3c4a Add asan support for various allocators 2025-05-06 14:55:50 +10:00
Harold Brenes
a3de9c8de4 Add initial support for Objective-C class implementation 2025-04-20 21:53:46 -04:00
gingerBill
3dcc22fa6d Change hashing rules for float-like types to make 0 == -0 2025-04-16 10:52:35 +01:00
Jeroen van Rijn
bb38775fb1 Remove Type_Info_Tuple 2025-04-13 21:51:57 +02:00
gingerBill
1f814c33dc Support subtargets in build tags: #build darwin:generic and #build linux:android, darwin:ios 2025-04-10 11:36:03 +01:00
Dave Voutila
18988b5f94 Fix use of errno on OpenBSD. 2025-03-27 10:36:40 -04:00
gingerBill
539e9bd2e3 Merge pull request #4836 from laytan/fix-wrong-out-of-memory
fix wrong out of memory in edge cases, just try allocate from block for one source of truth
2025-03-20 17:20:26 +00:00
gingerBill
21a6648cde Fix #4903 2025-03-03 15:16:50 +00:00