| Age | Commit message (Collapse) | Author |
|---|
|
Ring memory is released by umapping {cq,sq}->khead, which assumes that
p->{cq,sq}_off.head will always be 0.
Add another field to for the mmaped region and use that instead of
->khead when umapping.
Signed-off-by: Kornilios Kourtis <kkourt@kkourt.io>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
io_uring_cqe_seen() contains write memory barrier (at least for !x86),
so calling it can be very expensive for a big number of cqes.
Would be better to have a loop over all available cqes and only then
advance the cq ring once, e.g.:
struct io_uring_cqe *cqe;
unsigned nr, head;
nr = 0;
io_uring_for_each_cqe(&ring, head, cqe) {
/* handle cqe */
if (++nr == max)
break;
}
io_uring_cq_advance(&ring, nr);
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Enter kernel only if SQ thread is off or wakeup is needed.
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
io_uring_prep_{readv,writev,read_fixed,write_fixed}() take
non-const arguments that are const in the native functions
that they mimic. Make those arguments const.
Signed-off-by: Ian Gulliver <git@flamingcow.io>
|
|
liburing.h uses memset() but doesn't include string.h.
If nothing else includes it, this causes compile errors.
Add the include.
Signed-off-by: Ian Gulliver <git@flamingcow.io>
|
|
Make header includable from C++ projects.
Signed-off-by: Kevin Vigor <kvigor@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
We have io_uring_get_sqe() on the submission side, yet the completion
side is named _completion. Rename as follows:
io_uring_get_completion() io_uring_peek_cqe()
iO_uring_wait_completion() io_uring_wait_cqe()
This better tells the user what the _get variant does by calling it
_peek instead, and we move to using _cqe() as the postfix instead
of _completion.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
There's a failure case where an application gets a cqe entry, but
the kernel can then overwrite it before the application is done
reading it. This can happen since the io_uring_{get,wait}_completion()
interface both returns a CQE pointer AND increments the ring index.
If the kernel reuses this entry before the applications is done reading
it, the contents may be corrupted.
Remove the CQ head increment from the CQE retrieval, and put it into
a separate helper, io_uring_cqe_seen(). The application must call this
helper when it got a new CQE entry through one of the above calls, and
it's now done reading it.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Zach Bjornson <zbbjornson@gmail.com>
Add uintptr_t cast to avoid 32-bit warnings.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Also changes the fsync prep helper to require passing in the actual
flag, not just a boolean for fsync vs fdatasync.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Also unify the setup, all the read/write variants can use the same
helper.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
io_uring_queue_init does not allow the caller to specify sq_thread_cpu
or sq_thread_idle. Users that want to specify those parameters need to
call io_uring_setup(2) themselves. Add a helper so that they don't also
have to hand-craft the code to map the submission and completion queues,
and setup the sqe ring. This allows those applications to still make
use of io_uring_submit and get/wait_completion.
Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Applications should not need to care about this, we can pass it in
ourselves. Once the libc support is there, we won't expose this
parameter either.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Update liburing and io_uring_enter.2 to match the kernel.
Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Some older installs don't have __kernel_rwf_t in linux/fs.h, so
add a check for that.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
We don't need any of the information in there in the caller, and
this makes it harder to abuse as we don't require the caller to
have memset() the struct first.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
- Fixed buffers are now available through io_uring_register()
- Various thread/wq options are now dead and automatic instead
- sqe->index is now sqe->buf_index
- Fixed buffers require flag, not separate opcode
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
- io_uring_sqe added a data field that's passed back at completion
- io_uring_sqe added an index field, for fixed buffer locations
- io_uring_setup(2) system call added a 'nr_iovecs' field
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
No point in keeping these separate.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
This is cleaner than having the app juggle an SQ and CQ ring,
just wrap them in struct io_uring and have the API always take
that. This means the app doesn't need to worry about the different
types of rings, and that we only need to pass in one argument
for setup/teardown.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Returns the iocb associated with a completion event.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
This now exposes two helpers:
io_uring_get_completion()
Return a completion, if we have one (or more) available in
the ring
io_uring_wait_completion()
Return a completion, waiting for it if necessary
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
