This question is an extension of why do we need to pass buffers to system calls in order to have information returned? Why can't system calls allocate the buffer internally? since memory allocation is one of many motivations to pass callbacks to syscalls.
I would like to to pass an allocate callback to a syscall variant of the original that requires a buffer and since the original syscall would likely know the size of the buffer to allocate before writing to the buffer.
Many file system APIs (not just posix/*nix, but Windows/NT does this too) always ask that you pass a x byte path buffer (x would be 256 or 1024 or 4096 or MAX_PATH). You still have to pass x into the function to indicate buffer size and the function may fail if it needs more space than the buffer allows.
Is there a restriction where a syscall cannot call user space functions? Is it possible, but complicated since the syscall must retain the calling userspace context and the callback userspace context? Are all syscalls supposed to be atomic from a thread switching perspective and calling userspace code from within a syscall would break that atomicity?