[LINUX] Asynchronous I / O and non-blocking I / O

background

I wrote about synchronous / asynchronous and blocking / non-blocking, and I personally wanted to summarize it.

Features of asynchronous IO

The I / O model that notifies when the I / O processing is completed is called asynchronous I / O. I / O completion notification to the user is done by signal or callback. The process can proceed with other processing until notified.

It can be implemented using io_prep_pread (3), io_prep_pwrite (3), io_submit (2). There are also library functions such as aio_write and aio_read in the POSIX implementation.

struct {
    pthread_cond_t cond;
    pthread_mutex_t mtx;
    int flag;
} notified = {
    .cond   = PTHREAD_COND_INITIALIZER,
    .mtx    = PTHREAD_MUTEX_INITIALIZER,
    .flag   = 0
};

void thread_func(union sigval sv)
{
    printf("%s : aio_read from fd %d completed \n",
        __func__, sv.sival_int);

    pthread_mutex_lock(&notified.mtx);
    notified.flag = 1;
    pthread_cond_signal(& notified.cond);
    pthread_mutex_unlock(& notified.mtx);
}

main ()
{
    char a[BUFSIZ];
    struct aiocb aio = {
        .aio_offset   = 0,
        .aio_buf      = a,
        .aio_nbytes   = sizeof(a),
        .aio_reqprio  = 0,
        .aio_sigevent = {
            .sigev_notify            = SIGEV_THREAD,
            .sigev_notify_function   = thread_func,
            .sigev_notify_attributes = NULL
        }
    };

    aio.aio_fildes = open(argv[1], O_RDONLY);
    aio.aio_sigevent.sigev_value.sival_int = aio.aio_fildes;
    aio_read(&aio);

    /* do other jobs */

    pthread_mutex_lock(&notified.mtx);
    while (!notified.flag)
        pthread_cond_wait(&notified.cond, &notified.mtx);
    pthread_mutex_unlock(&notified.mtx);
}

A summary of the functions used. The contents of each are as follows

io_queue_init (2)-Preparing for asynchronous I / O io_prep_pwrite(3) - io_submit (2) --Subscribe an asynchronous I / O block to the pending queue io_getevents (2)-Read asynchronous I / O events from the completion queue io_queue_release (3)-Release userspace-related context

Features of non-blocking IO

In non-blocking I / O, if the file descriptor to be I / O is not ready, the user Immediate error returned (EAGAIN)

It can be implemented by specifying the O_NONBLOCK flag in open (2). Operations on file descriptors opened with O_NONBLOCK no longer cause the process to wait.

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

void set_fl(int fd, int flags);
void clr_fl(int fd, int flags);

char buf[100000];

int main(void) {
  int ntowrite, nwrite;
  char *ptr;

  ntowrite = read(STDIN_FILENO, buf, sizeof(buf));
  fprintf(stderr, "read %d byts\n", ntowrite);
  set_fl(STDOUT_FILENO, O_NONBLOCK);

  for (ptr = buf; ntowrite > 0; ) {
    errno = 0;
    nwrite = write(STDOUT_FILENO, ptr, ntowrite);
    fprintf(stderr, "nwrite = %d, errno = %d, err_message = '%s'\n", nwrite, errno, strerror(errno));
    if (nwrite > 0) {
      ptr += nwrite;
      ntowrite -= nwrite;
    }
  }

  clr_fl(STDOUT_FILENO, O_NONBLOCK);
  return 0;
}

void set_fl(int fd, int flags) {
  int val;

  if ((val = fcntl(fd, F_GETFL, 0)) < 0) {
    fprintf(stderr, "fcntl F_GETFL error");
    exit(1);
  }

  val |= flags;  /* turn on flags */

  if (fcntl(fd, F_SETFL, val) < 0) {
    fprintf(stderr, "fcntl F_SETFL error");
    exit(1);
  }
}

void clr_fl(int fd, int flags) {
  int val;

  if ((val = fcntl(fd, F_GETFL, 0)) < 0) {
    fprintf(stderr, "fcntl F_GETFL error");
    exit(1);
  }

  val &= ~flags;  /* turn flags off */

  if (fcntl(fd, F_SETFL, val) < 0) {
    fprintf(stderr, "fcntl F_SETFL error");
    exit(1);
  }
}

Difference between the two

Asynchronous waiting for notifications in the background. It is non-blocking to go to check the status at fixed intervals.

Making full use of these and using I / O Multiplexing will lead to an event-driven architecture.

C10K problem

The "C10K problem" (10,000 clients problem) is a problem that the server punctures when the number of clients becomes too large, even if there is no problem in terms of hardware performance. It seems to happen when a large number of threads with a certain amount of memory are created.

Reference: https://wa3.i-3-i.info/word11592.html

About spinlock

A little derailment. About the term spinlock that comes up several times when I research asynchronous.

When each CPU accesses the same resource at the same time in a multiprocessor environment Exclusive control mechanism used. Prepare one lock variable in memory for one resource. And only the CPU that can get the lock variable can access the resource.

Spinlock is an easy way to achieve exclusive control in a multiprocessor system. It is often used. However, the CPU during busy waiting cannot execute the process. Since it will be in a waiting state, processing efficiency will be improved if busy weights occur frequently. It gets worse. Considering processing efficiency, resources that are exclusively controlled by spinlock should be used as much as possible. It is necessary to devise such as subdividing.

A type of spinlock is a "read / write spinlock". In this spin lock, if there is a conflict in read processing from the CPU, the lock is not performed. You can access resources from both. Exclusive control is performed as usual only when writing. Read / write spinlocks are useful when there is a lot of reference processing to the resource.

Summary

Reference link

About asynchronous IO [Understanding the difference between non-blocking I / O and asynchronous I / O](https://blog.takanabe.tokyo/2015/03/%E3%83%8E%E3%83%B3%E3%83%96% E3% 83% AD% E3% 83% 83% E3% 82% AD% E3% 83% B3% E3% 82% B0i / o% E3% 81% A8% E9% 9D% 9E% E5% 90% 8C% E6% 9C% 9Fi / o% E3% 81% AE% E9% 81% 95% E3% 81% 84% E3% 82% 92% E7% 90% 86% E8% A7% A3% E3% 81% 99% E3% 82% 8B /)