Post by @mcc@mastodon.social

I have been called on to write a "Partial Free" function for previously allocated virtual memory. EG, someone allocated a large block of memory with mmap(), and now they want to free a section *out of the middle or end* of it.

My reading of the POSIX munmap doc https://pubs.opengroup.org/onlinepubs/7908799/xsh/munmap.html is that munmap does this already, because it deallocates "pages" not previous allocations, and I only need to call munmap() with the requested-deallocated range (assuming the range to be page-aligned)

Correct?

munmap

0 Boosts

0 Likes

HP van Braam

@hp@mastodon.tmm.cx 17 min. ago

@mcc depending on the arch, the protections may be a finite resource, particularly embedded riscv.

So splitting a range, and then adding a new protection range may fail, I don't know how important that is in your environment, but there's not necessarily a guarantee that the "partial free" is possible.

mcc

@mcc@mastodon.social 15 min. ago

@hp Thank you for this information. We have written a hygienic failure path in already, so I think we are safe.

yuubi

@yuubi@furry.engineer 49 min. ago

@mcc i've seen glibc on linux turn malloc into "mmap a large block and munmap the last about half of it", so yes that's a common usage pattern. the munmap took most of the time.

David Smith

@Catfish_Man@mastodon.social 50 min. ago

@mcc correct. In case it matters, some kernels do not handle this especially efficiently.

mcc

@mcc@mastodon.social 22 min. ago

@Catfish_Man Good to know.

John Ripley

@jripley@mastodon.social 53 min. ago

@mcc That is also my understanding, and what I recall is also the implementation in various kernel sources (including Linux). mmap is not malloc, operates on independent pages, and munmap is not required to use only values returned by a previous mmap. You can even munmap pages which were present at process launch, and never explicitly mmap'd.

mcc

@mcc@mastodon.social 58 min. ago

For clarity, it is more important my function run correctly on BSD derived systems than Linux derived systems

mcc

@mcc@mastodon.social 23 min. ago

2nd odd question.

The person calling on me to write "partial deallocate" also wants me to return the "allocate page size" and "commit page size". They provided a Win32 implementation, I must write UNIX/Linux. here is what I believe is happening.

- Windows vmem has two operations, "allocate" and "commit". These operations have different requirements for the range size.

- On UNIX/Linux, these operations correspond to "mmap" and "mprotect". They BOTH use sysconf(_SC_PAGE_SIZE) for size multiple.

John Ripley

@jripley@mastodon.social 12 min. ago

@mcc I think "commit" is closer to Linux "mlock" than "mprotect". What it's doing is making a page deterministically accessible. In Linux, a page may fault non-deterministically on first access, even if you allocated backing store (anonymous or otherwise). You can either trigger allocation by striding through the range, and taking access faults (annoying to handle), or by locking the range (enforces pages present).

John Ripley

@jripley@mastodon.social 6 min. ago

@mcc To be clear, there's only a distinction here when you are in *OOM* condition. In Linux, first access is when you discover you are OOM, and your process takes a page fault, or is simply killed. You can make this deterministic by calling mlock() but then (depending on your global OOM config) that may only serve to make your process get killed at that function call.

1 more reply

mcc

@mcc@mastodon.social 6 min. ago

@jripley That's very interesting. However, would you agree this mlock operation occurs on whole pages (eg the same page size used for mprotect and mmap(MAP_ANONYMOUS)?

1 more reply

mcc

@mcc@mastodon.social 22 min. ago

(The above post glosses over MAP_ANONYMOUS for lack of space, but it is not important to my question.)

Federation Bot

@Federation_Bot 51 min. ago

@mcc Lol I replied to this as if it was the other thread, sorry

Federation Bot

@Federation_Bot 54 min. ago

@mcc Several that I know of have proprietary languages, but a lot are simply HPGL and the interfaces are serial, and you can literally pipe the HPGL text saved from Inkscape onto the USB TTY file at least on Linux. The Silhouette has a plugin for Inkscape, but say, the Cricut you'd have to gut the electronics entirely and figure that out