It is not possible because system call table (called
sys_call_table) is a static size array. And its size is determined at compile time by the number of registered syscalls. This means there is no space for another one.
You can check implementation for example for x86 architecture in
arch/x86/kernel/syscall_64.c file, where
sys_call_table is defined. Its size is exactly
__NR_syscall_max is defined in
sizeof(syscalls) - 1 (it's the number of last syscall), wher
syscall is a table with all the syscalls.
The only possible thing is to replace some existing syscall with yours as this won't require more space in memory. It's not really easy from kernel module as kernel does not export
sys_call_table to modules as of version 2.6 (the last kernel version that had this symbol exported was
2.5.41. You can use this technique if you are developing new syscall and just want to avoid rebooting while experimenting. You will have to define your new system call and then replace it from module. To do this, you will have to change your kernel to export
sys_call_table symbol to modules. To do this, you have to add following to lines to
kernel/kallsyms.c (don't do this on production machines):
extern void *sys_call_table;