I am currently reading the "Malware Analyst's Cookbook and DVD". There is a chapter "Dynamic Analysis" and there are also some recipes about hooking and monitoring API calls of process but it is for Windows.
I want to do the same thing like recipe 9-10 explains but for Linux. 9-10 is called "Capturing process, Thread, and Image Load Events". In this receipe it is showed "how to implement a driver that alerts you when any events occure on the system while your malware samlpe executes". It uses the API functions of the Windows Driver Kit (WDK) to call a user-defined callback function. It uses the callback functions:
- Process creation callback function called PsSetCreateProcessNotifyRoutine(...)
- Thread creation callback function called PsSetCreateThreadNotifyRoutine(...)
- Image load callback function called PsSetLoadImageNotifyRoutine(...).
And when any events occur it will display them as a debug message which can then be viewed in e.g. DebugView.
It seems well documented for Windows and it is easy to find information for this, but I have a bit of a problem in finding information for Linux.
I've found some general introduction to drivers and a one for hooking, but I still haven't found any that are not so general or at least are a bit more focused on malware analysis.
I would be happy for tips for further readings or recommended tutorials on this topic.