0

The socket API provides an SO_INCOMING_CPU option, the details of which are provided in the man page:

   SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux 4.4)
         Sets or gets the CPU affinity of a socket.  Expects an integer
         flag.

             int cpu = 1;
             setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, sizeof(cpu));

         Because all of the packets for a single stream (i.e., all
         packets for the same 4-tuple) arrive on the single RX queue
         that is associated with a particular CPU, the typical use case
         is to employ one listening process per RX queue, with the
         incoming flow being handled by a listener on the same CPU that
         is handling the RX queue.  This provides optimal NUMA behavior
         and keeps CPU caches hot.

I have a server that creates a UDP socket, binds to the interface, and recv()s data. I want to get the SO_INCOMING_CPU of this socket. Is that possible?

I wrote a test program, and the test program consistently returns -1.

I did some printk() debugging, and it looks like the problem is that there isn't an inet_daddr associated with the socket: https://github.com/torvalds/linux/blob/v4.20-rc2/net/ipv4/udp.c#L1870

I think this makes sense: UDP is connectionless and as such, the socket could receive packets from any address. And I think I read somewhere that the CPU affinity of a socket is a function of the 4-tuple, and as such, the socket doesn't necessarily have a fixed CPU affinity.

However, given the fact that there's code to initialize the CPU affinity in udp.c, I think I might be missing something.

Is there any way to get SO_INCOMING_CPU working with a server-side UDP socket?

Here's the (slightly sloppy) test program I mentioned:

#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>

int main()
{
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);

    if (sockfd < 0)
    {  
        perror("Error opening socket");
        return 0;
    }

    struct sockaddr_in myaddr;

    memset((char *)&myaddr, 0, sizeof(myaddr));
    myaddr.sin_family = AF_INET;
    myaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    myaddr.sin_port = htons(4242);

    if (bind(sockfd, (struct sockaddr *)&myaddr, sizeof(myaddr)) < 0) {
        perror("bind failed");
        return 0;
    }

    #define BUFSIZE 20
    unsigned char buf[BUFSIZE];     /* receive buffer */
    struct sockaddr_in remaddr;     /* remote address */
    socklen_t addrlen = sizeof(remaddr);            /* length of addresses */
    int recvlen = recvfrom(sockfd, buf, BUFSIZE, 0, (struct sockaddr *)&remaddr, &addrlen);

    sleep(1);

    int cpu = 0;
    socklen_t len = sizeof(cpu);
    int ret = getsockopt(sockfd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);

    // Sample Output: Incoming CPU is -1 (ret = 0, recvlen = 5)
    printf("Incoming CPU is %d (ret = %d, recvlen = %d)\n", cpu, ret, recvlen);
}
  • where are the calls to listen() and accept()? – user3629249 Nov 20 '18 at 7:57
  • you probably want to examine blog to gain a more detailed prospective on CPU selection, etc of network data packets – user3629249 Nov 20 '18 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.