| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
io_uring: add zc notification infrastructure Add internal part of send zerocopy notifications. There are two main structures, the first one is struct io_notif, which carries inside struct ubuf_info and maps 1:1 to it. io_uring will be binding a number of zerocopy send requests to it and ask to complete (aka flush) it. When flushed and all attached requests and skbs complete, it'll generate one and only one CQE. There are intended to be passed into the network layer as struct msghdr::msg_ubuf. The second concept is notification slots. The userspace will be able to register an array of slots and subsequently addressing them by the index in the array. Slots are independent of each other. Each slot can have only one notifier at a time (called active notifier) but many notifiers during the lifetime. When active, a notifier not going to post any completion but the userspace can attach requests to it by specifying the corresponding slot while issueing send zc requests. Eventually, the userspace will want to "flush" the notifier losing any way to attach new requests to it, however it can use the next atomatically added notifier of this slot or of any other slot. When the network layer is done with all enqueued skbs attached to a notifier and doesn't need the specified in them user data, the flushed notifier will post a CQE. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/3ecf54c31a85762bf679b0a432c9f43ecf7e61cc.1657643355.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: always go async for unsupported fadvise flags No point in issuing -> return -EAGAIN -> go async, when it can be done upfront. Signed-off-by: Dylan Yudaken <dylany@meta.com> Link: https://lore.kernel.org/r/20230127135227.3646353-4-dylany@meta.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: split out fadvise/madvise operations Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/rsrc: consolidate node caching We store one pre-allocated rsrc node in ->rsrc_backup_node, merge it with ->rsrc_node_cache. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/6d5410e51ccd29be7a716be045b51d6b371baef6.1681210788.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: use fget/fput consistently stable inclusion from stable-v6.6.13 commit 0c7df8c241ff066b19782100f6bfc288144d7a38 bugzilla: https://gitee.com/openeuler/kernel/issues/I99TJK Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=0c7df8c241ff066b19782100f6bfc288144d7a38 -------------------------------- [ Upstream commit 73363c262d6a7d26063da96610f61baf69a70f7c ] Normally within a syscall it's fine to use fdget/fdput for grabbing a file from the file table, and it's fine within io_uring as well. We do that via io_uring_enter(2), io_uring_register(2), and then also for cancel which is invoked from the latter. io_uring cannot close its own file descriptors as that is explicitly rejected, and for the cancel side of things, the file itself is just used as a lookup cookie. However, it is more prudent to ensure that full references are always grabbed. For anything threaded, either explicitly in the application itself or through use of the io-wq worker threads, this is what happens anyway. Generalize it and use fget/fput throughout. Also see the below link for more details. Link: https://lore.kernel.org/io-uring/CAG48ez1htVSO3TqmrF8QcX2WFuYTRM-VZ_N10i-VZgbtg=NNqw@mail.gmail.com/ Suggested-by: Jann Horn <jannh@google.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: ZhangPeng <zhangpeng362@huawei.com> | 2 年前 | |
io_uring/cancel: de-unionize file and user_data in struct io_cancel_data stable inclusion from stable-v6.6.128 commit fb2d7c30d03016f75b5c9535d537a5ca1b8145fe category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/ Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=fb2d7c30d03016f75b5c9535d537a5ca1b8145fe -------------------------------- commit fb2d7c30d03016f75b5c9535d537a5ca1b8145fe upstream. [ Upstream commit 22dbb0987bd1e0ec3b1e4ad20756a98f99aa4a08 ] By having them share the same space in struct io_cancel_data, it ends up disallowing IORING_ASYNC_CANCEL_FD|IORING_ASYNC_CANCEL_USERDATA from working. Eg you cannot match on both a file and user_data for cancelation purposes. This obviously isn't a common use case as nobody has reported this, but it does result in -ENOENT potentially being returned when trying to match on both, rather than actually doing what the API says it would. Fixes: 4bf94615b888 ("io_uring: allow IORING_OP_ASYNC_CANCEL with 'fd' key") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 个月前 | |
io_uring: undeprecate epoll_ctl support Libuv recently started using it so there is at least one consumer now. Cc: stable@vger.kernel.org Fixes: 61a2732af4b0 ("io_uring: deprecate epoll_ctl support") Link: https://github.com/libuv/libuv/pull/3979 Signed-off-by: Ben Noordhuis <info@bnoordhuis.nl> Link: https://lore.kernel.org/r/20230506095502.13401-1-info@bnoordhuis.nl Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: move epoll handler to its own file Would be nice to sort out Kconfig for this and don't even compile epoll.c if we don't have epoll configured. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/fdinfo: annotate racy sq/cq head/tail reads stable inclusion from stable-v6.6.93 commit b72952c8c3668142843acde6749cfb1fd3eff720 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/8365 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=b72952c8c3668142843acde6749cfb1fd3eff720 -------------------------------- [ Upstream commit f024d3a8ded0d8d2129ae123d7a5305c29ca44ce ] syzbot complains about the cached sq head read, and it's totally right. But we don't need to care, it's just reading fdinfo, and reading the CQ or SQ tail/head entries are known racy in that they are just a view into that very instant and may of course be outdated by the time they are reported. Annotate both the SQ head and CQ tail read with data_race() to avoid this syzbot complaint. Link: https://lore.kernel.org/io-uring/6811f6dc.050a0220.39e3a1.0d0e.GAE@google.com/ Reported-by: syzbot+3e77fd302e99f5af9394@syzkaller.appspotmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> (cherry picked from commit b72952c8c3668142843acde6749cfb1fd3eff720) Signed-off-by: Wentao Guan <guanwentao@uniontech.com> | 5 个月前 | |
io_uring: move fdinfo helpers to its own file This also means moving a bit more of the fixed file handling to the filetable side, which makes sense separately too. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/filetable: clamp alloc_hint to the configured alloc range stable inclusion from stable-v6.6.128 commit 6eb571a376312faf7122d2759a5513129f8f5ec0 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/ Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=6eb571a376312faf7122d2759a5513129f8f5ec0 -------------------------------- commit 6eb571a376312faf7122d2759a5513129f8f5ec0 upstream. [ Upstream commit a6bded921ed35f21b3f6bd8e629bf488499ca442 ] Explicit fixed file install/remove operations on slots outside the configured alloc range can corrupt alloc_hint via io_file_bitmap_set() and io_file_bitmap_clear(), which unconditionally update alloc_hint to the bit position. This causes subsequent auto-allocations to fall outside the configured range. For example, if the alloc range is [10, 20) and a file is removed at slot 2, alloc_hint gets set to 2. The next auto-alloc then starts searching from slot 2, potentially returning a slot below the range. Fix this by clamping alloc_hint to [file_alloc_start, file_alloc_end) at the top of io_file_bitmap_get() before starting the search. Cc: stable@vger.kernel.org Fixes: 6e73dffbb93c ("io_uring: let to set a range for file slot allocation") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 个月前 | |
io_uring: add helpers to decode the fixed file file_ptr Remove all the open coded magic on slot->file_ptr by introducing two helpers that return the file pointer and the flags instead. Signed-off-by: Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/r/20230620113235.920399-9-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/fs: consider link->flags when getting path for LINKAT stable inclusion from stable-v6.6.4 commit ed869aee2adcf2b0ea38bf52f9660f0f5b7e9faf category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8N1WC Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ed869aee2adcf2b0ea38bf52f9660f0f5b7e9faf -------------------------------- commit 8479063f1fbee201a8739130e816cc331b675838 upstream. In order for AT_EMPTY_PATH to work as expected, the fact that the user wants that behavior needs to make it to getname_flags or it will return ENOENT. Fixes: cf30da90bc3a ("io_uring: add support for IORING_OP_LINKAT") Cc: <stable@vger.kernel.org> Link: https://github.com/axboe/liburing/issues/995 Signed-off-by: Charles Mirabile <cmirabil@redhat.com> Link: https://lore.kernel.org/r/20231120105545.1209530-1-cmirabil@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> | 2 年前 | |
io_uring: split out filesystem related operations This splits out renameat, unlinkat, mkdirat, symlinkat, and linkat. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io-wq: check that the predecessor is hashed in io_wq_remove_pending() stable inclusion from stable-v6.6.141 commit d6bda9df0c0a3080804181464d5c0f4d78a4e769 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/ Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=d6bda9df0c0a3080804181464d5c0f4d78a4e769 -------------------------------- commit d6a2d7b04b5a093021a7a0e2e69e9d5237dfa8cc upstream. io_wq_remove_pending() needs to fix up wq->hash_tail[] if the cancelled work was the tail of its hash bucket. When doing this, it checks whether the preceding entry in acct->work_list has the same hash value, but never checks that the predecessor is hashed at all. io_get_work_hash() is simply atomic_read(&work->flags) >> IO_WQ_HASH_SHIFT, and the hash bits are never set for non-hashed work, so it returns 0. Thus, when a hashed bucket-0 work is cancelled while a non-hashed work is its list predecessor, the check spuriously passes and a pointer to the non-hashed io_kiocb is stored in wq->hash_tail[0]. Because non-hashed work is dequeued via the fast path in io_get_next_work(), which never touches hash_tail[], the stale pointer is never cleared. Therefore, after the non-hashed io_kiocb completes and is freed back to req_cachep, wq->hash_tail[0] is a dangling pointer. The io_wq is per-task (tctx->io_wq) and survives ring open/close, so the dangling pointer persists for the lifetime of the task; the next hashed bucket-0 enqueue dereferences it in io_wq_insert_work() and wq_list_add_after() writes through freed memory. Add the missing io_wq_is_hashed() check so a non-hashed predecessor never inherits a hash_tail[] slot. Cc: stable@vger.kernel.org Fixes: 204361a77f40 ("io-wq: fix hang after cancelling pending hashed work") Signed-off-by: Nicholas Carlini <nicholas@carlini.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 个月前 | |
io_uring: break out of iowq iopoll on teardown io-wq will retry iopoll even when it failed with -EAGAIN. If that races with task exit, which sets TIF_NOTIFY_SIGNAL for all its workers, such workers might potentially infinitely spin retrying iopoll again and again and each time failing on some allocation / waiting / etc. Don't keep spinning if io-wq is dying. Fixes: 561fb04a6a225 ("io_uring: replace workqueue usage with io-wq") Cc: stable@vger.kernel.org Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> | 2 年前 | |
fs: Rename anon_inode_getfile_secure() and anon_inode_getfd_secure() mainline inclusion from mainline-v6.8-rc1 commit 4f0b9194bc119a9850a99e5e824808e2f468c348 category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/126 CVE: NA Reference: https://github.com/torvalds/linux/commit/4f0b9194bc119a9850a99e5e824808e2f468c348 -------------------------------- commit 4f0b9194bc119a9850a99e5e824808e2f468c348 upstream The call to the inode_init_security_anon() LSM hook is not the sole reason to use anon_inode_getfile_secure() or anon_inode_getfd_secure(). For example, the functions also allow one to create a file with non-zero size, without needing a full-blown filesystem. In this case, you don't need a "secure" version, just unique inodes; the current name of the functions is confusing and does not explain well the difference with the more "standard" anon_inode_getfile() and anon_inode_getfd(). Of course, there is another side of the coin; neither io_uring nor userfaultfd strictly speaking need distinct inodes, and it is not that clear anymore that anon_inode_create_get{file,fd}() allow the LSM to intercept and block the inode's creation. If one was so inclined, anon_inode_getfile_secure() and anon_inode_getfd_secure() could be kept, using the shared inode or a new one depending on CONFIG_SECURITY. However, this is probably overkill, and potentially a cause of bugs in different configurations. Therefore, just add a comment to io_uring and userfaultfd explaining the choice of the function. While at it, remove the export for what is now anon_inode_create_getfd(). There is no in-tree module that uses it, and the old name is gone anyway. If anybody actually needs the symbol, they can ask or they can just use anon_inode_create_getfile(), which will be exported very soon for use in KVM. [Backport Changes] 1. In file “io_uring/io_uring.c” retained the changes from Euler commit a7d46ae176b8a (io_uring: drop any code related to SCM_RIGHTS), which dropped support for passing io_uring file descriptors over SCM_RIGHTS, so in function io_uring_get_file() instead of returning dropped variable with file pointer, the existing logic was retained where function directly returns result of anon_inode_create_getfile(). This approach aligns with the upstream version in merge commit 09d1c6a80f2cf from v6.8-rc1, which is retained in future kernel releases. Suggested-by: Christian Brauner <brauner@kernel.org> Reviewed-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Jeevan deep J <j.jeevandeep@amd.com> Signed-off-by: PrithivishS <sprithiv@amd.com> | 30 天前 | |
io_uring: include dying ring in task_work "should cancel" state stable inclusion from stable-v6.6.108 commit abc9f859a9ea3d24ccaab2a2f4b1850084d6900f category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/8521 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=abc9f859a9ea3d24ccaab2a2f4b1850084d6900f -------------------------------- Commit 3539b1467e94336d5854ebf976d9627bfb65d6c3 upstream. When running task_work for an exiting task, rather than perform the issue retry attempt, the task_work is canceled. However, this isn't done for a ring that has been closed. This can lead to requests being successfully completed post the ring being closed, which is somewhat confusing and surprising to an application. Rather than just check the task exit state, also include the ring ref state in deciding whether or not to terminate a given request when run from task_work. Cc: stable@vger.kernel.org # 6.1+ Link: https://github.com/axboe/liburing/discussions/1459 Reported-by: Benedek Thaler <thaler@thaler.hu> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> (cherry picked from commit abc9f859a9ea3d24ccaab2a2f4b1850084d6900f) Signed-off-by: Wentao Guan <guanwentao@uniontech.com> | 4 个月前 | |
io_uring/kbuf: check if target buffer list is still legacy on recycle stable inclusion from stable-v6.6.130 commit 439a6728ec4641ffad1ca796622c19bc525e570f category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/14925 CVE: CVE-2026-43366 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=439a6728ec4641ffad1ca796622c19bc525e570f -------------------------------- Commit c2c185be5c85d37215397c8e8781abf0a69bec1f upstream. There's a gap between when the buffer was grabbed and when it potentially gets recycled, where if the list is empty, someone could've upgraded it to a ring provided type. This can happen if the request is forced via io-wq. The legacy recycling is missing checking if the buffer_list still exists, and if it's of the correct type. Add those checks. Cc: stable@vger.kernel.org Fixes: c7fb19428d67 ("io_uring: add support for ring mapped supplied buffers") Reported-by: Keenan Dong <keenanat2000@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Zizhi Wo <wozizhi@huawei.com> | 1 个月前 | |
io_uring/kbuf: hold io_buffer_list reference over mmap stable inclusion from stable-v6.6.26 commit 65938e81df2197203bda4b9a0c477e7987218d66 bugzilla: https://gitee.com/openeuler/kernel/issues/I9MPZ8 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=65938e81df2197203bda4b9a0c477e7987218d66 -------------------------------- commit 561e4f9451d65fc2f7eef564e0064373e3019793 upstream. If we look up the kbuf, ensure that it doesn't get unregistered until after we're done with it. Since we're inside mmap, we cannot safely use the io_uring lock. Rely on the fact that we can lookup the buffer list under RCU now and grab a reference to it, preventing it from being unregistered until we're done with it. The lookup returns the io_buffer_list directly with it referenced. Cc: stable@vger.kernel.org # v6.4+ Fixes: 5cf4f52e6d8a ("io_uring: free io_buffer_list entries via RCU") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: ZhangPeng <zhangpeng362@huawei.com> | 2 年前 | |
io_uring: use io_file_from_index in io_msg_grab_file Use io_file_from_index instead of open coding it. Signed-off-by: Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/r/20230620113235.920399-8-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: get rid of double locking We don't need to take both uring_locks at once, msg_ring can be split in two parts, first getting a file from the filetable of the first ring and then installing it into the second one. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/a80ecc2bc99c3b3f2cf20015d618b7c51419a797.1670384893.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
| 5 个月前 | ||
io_uring: Add KASAN support for alloc_caches Add support for KASAN in the alloc_caches (apoll and netmsg_cache). Thus, if something touches the unused caches, it will raise a KASAN warning/exception. It poisons the object when the object is put to the cache, and unpoisons it when the object is gotten or freed. Signed-off-by: Breno Leitao <leitao@debian.org> Reviewed-by: Gabriel Krisman Bertazi <krisman@suse.de> Link: https://lore.kernel.org/r/20230223164353.2839177-2-leitao@debian.org Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: fail NOP if non-zero op flags is passed in stable inclusion from stable-v6.6.33 commit a9886aad5dfdf7570e4fdfc4f97baaffc92c6a9c category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAD6H2 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=a9886aad5dfdf7570e4fdfc4f97baaffc92c6a9c -------------------------------- commit 3d8f874bd620ce03f75a5512847586828ab86544 upstream. The NOP op flags should have been checked from beginning like any other opcode, otherwise NOP may not be extended with the op flags. Given both liburing and Rust io-uring crate always zeros SQE op flags, just ignore users which play raw NOP uring interface without zeroing SQE, because NOP is just for test purpose. Then we can save one NOP2 opcode. Suggested-by: Jens Axboe <axboe@kernel.dk> Fixes: 2b188cc1bb85 ("Add io_uring IO interface") Cc: stable@vger.kernel.org Signed-off-by: Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/20240510035031.78874-2-ming.lei@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 年前 | |
io_uring: move nop into its own file Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/notif: add constant for ubuf_info flags Add a constant IO_NOTIF_UBUF_FLAGS for struct ubuf_info flags that notifications use. That should minimise merge conflicts for planned changes touching both io_uring and net at the same time. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/notif: add constant for ubuf_info flags Add a constant IO_NOTIF_UBUF_FLAGS for struct ubuf_info flags that notifications use. That should minimise merge conflicts for planned changes touching both io_uring and net at the same time. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: make fallocate be hashed work stable inclusion from stable-v6.6.99 commit 50b1e01aa1233fc36364e2d3e1855e7fb7723db5 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/8365 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=50b1e01aa1233fc36364e2d3e1855e7fb7723db5 -------------------------------- [ Upstream commit 88a80066af1617fab444776135d840467414beb6 ] Like ftruncate and write, fallocate operations on the same file cannot be executed in parallel, so it is better to make fallocate be hashed work. Signed-off-by: Fengnan Chang <changfengnan@bytedance.com> Link: https://lore.kernel.org/r/20250623110218.61490-1-changfengnan@bytedance.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> (cherry picked from commit 50b1e01aa1233fc36364e2d3e1855e7fb7723db5) Signed-off-by: Wentao Guan <guanwentao@uniontech.com> | 5 个月前 | |
io_uring: Split io_issue_def struct This patch removes some "cold" fields from struct io_issue_def. The plan is to keep only highly used fields into struct io_issue_def, so, it may be hot in the cache. The hot fields are basically all the bitfields and the callback functions for .issue and .prep. The other less frequently used fields are now located in a secondary and cold struct, called io_cold_def. This is the size for the structs: Before: io_issue_def = 56 bytes After: io_issue_def = 24 bytes; io_cold_def = 40 bytes Signed-off-by: Breno Leitao <leitao@debian.org> Reviewed-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/20230112144411.2624698-2-leitao@debian.org Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: fix filename leak in __io_openat_prep() stable inclusion from stable-v6.6.120 commit 18b99fa603d0df5e1c898699c17d3b92ddc80746 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/13367 CVE: CVE-2025-68814 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=18b99fa603d0df5e1c898699c17d3b92ddc80746 -------------------------------- commit b14fad555302a2104948feaff70503b64c80ac01 upstream. __io_openat_prep() allocates a struct filename using getname(). However, for the condition of the file being installed in the fixed file table as well as having O_CLOEXEC flag set, the function returns early. At that point, the request doesn't have REQ_F_NEED_CLEANUP flag set. Due to this, the memory for the newly allocated struct filename is not cleaned up, causing a memory leak. Fix this by setting the REQ_F_NEED_CLEANUP for the request just after the successful getname() call, so that when the request is torn down, the filename will be cleaned up, along with other resources needing cleanup. Reported-by: syzbot+00e61c43eb5e4740438f@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=00e61c43eb5e4740438f Tested-by: syzbot+00e61c43eb5e4740438f@syzkaller.appspotmail.com Cc: stable@vger.kernel.org Signed-off-by: Prithvi Tambewagh <activprithvi@gmail.com> Fixes: b9445598d8c6 ("io_uring: openat directly into fixed fd table") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Long Li <leo.lilong@huawei.com> | 5 个月前 | |
io_uring: split out fixed file installation and removal Put it with the filetable code, which is where it belongs. While doing so, have the helpers take a ctx rather than an io_kiocb. It doesn't make sense to use a request, as it's not an operation on the request itself. It applies to the ring itself. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/poll: ensure EPOLL_ONESHOT is propagated for EPOLL_URING_WAKE stable inclusion from stable-v6.6.140 commit fafab8b3cd57a3389182f884b879caa632da40d7 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/ Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=fafab8b3cd57a3389182f884b879caa632da40d7 -------------------------------- commit 1967f0b1cafdde37aa9e08e6021c14bcc484b7a5 upstream. Commit: aacf2f9f382c ("io_uring: fix req->apoll_events") fixed an issue where poll->events and req->apoll_events weren't synchronized, but then when the commit referenced in Fixes got added, it didn't ensure the same thing. If we mask in EPOLLONESHOT in the regular EPOLL_URING_WAKE path, then ensure it's done for both. Including a link to the original report below, even though it's mostly nonsense. But it includes a reproducer that does show that IORING_CQE_F_MORE is set in the previous CQE, while no more CQEs will be generated for this request. Just ignore anything that pretends this is security related in any way, it's just the typical AI nonsense. Cc: stable@vger.kernel.org Link: https://lore.kernel.org/io-uring/CAM0zi7yQzF3eKncgHo4iVM5yFLAjsiob_ucqyWKs=hyd_GqiMg@mail.gmail.com/ Reported-by: Azizcan Daştan <azizcan.d@mileniumsec.com> Fixes: 4464853277d0 ("io_uring: pass in EPOLL_URING_WAKE for eventfd signaling and wakeups") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 个月前 | |
io_uring: avoid indirect function calls for the hottest task_work We use task_work for a variety of reasons, but doing completions or triggering rety after poll are by far the hottest two. Use the indirect funtion call wrappers to avoid the indirect function call if CONFIG_RETPOLINE is set. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: always do atomic put from iowq stable inclusion from stable-v6.6.89 commit c5d4d103005d8926cdad344f9fc947e651c9f2f7 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/ID73WA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=c5d4d103005d8926cdad344f9fc947e651c9f2f7 -------------------------------- [ Upstream commit 390513642ee6763c7ada07f0a1470474986e6c1c ] io_uring always switches requests to atomic refcounting for iowq execution before there is any parallilism by setting REQ_F_REFCOUNT, and the flag is not cleared until the request completes. That should be fine as long as the compiler doesn't make up a non existing value for the flags, however KCSAN still complains when the request owner changes oter flag bits: BUG: KCSAN: data-race in io_req_task_cancel / io_wq_free_work ... read to 0xffff888117207448 of 8 bytes by task 3871 on cpu 0: req_ref_put_and_test io_uring/refs.h:22 [inline] Skip REQ_F_REFCOUNT checks for iowq, we know it's set. Reported-by: syzbot+903a2ad71fb3f1e47cf5@syzkaller.appspotmail.com Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/d880bc27fb8c3209b54641be4ff6ac02b0e5789a.1743679736.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> (cherry picked from commit c5d4d103005d8926cdad344f9fc947e651c9f2f7) Signed-off-by: Wentao Guan <guanwentao@uniontech.com> | 7 个月前 | |
io_uring: check for overflows in io_pin_pages mainline inclusion from mainline-v6.10-rc2 commit 0c0a4eae26ac78379d0c1db053de168a8febc6c9 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IBEAFR CVE: CVE-2024-53187 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c0a4eae26ac78379d0c1db053de168a8febc6c9 -------------------------------- WARNING: CPU: 0 PID: 5834 at io_uring/memmap.c:144 io_pin_pages+0x149/0x180 io_uring/memmap.c:144 CPU: 0 UID: 0 PID: 5834 Comm: syz-executor825 Not tainted 6.12.0-next-20241118-syzkaller #0 Call Trace: <TASK> __io_uaddr_map+0xfb/0x2d0 io_uring/memmap.c:183 io_rings_map io_uring/io_uring.c:2611 [inline] io_allocate_scq_urings+0x1c0/0x650 io_uring/io_uring.c:3470 io_uring_create+0x5b5/0xc00 io_uring/io_uring.c:3692 io_uring_setup io_uring/io_uring.c:3781 [inline] ... </TASK> io_pin_pages()'s uaddr parameter came directly from the user and can be garbage. Don't just add size to it as it can overflow. Cc: stable@vger.kernel.org Reported-by: syzbot+2159cbb522b02847c053@syzkaller.appspotmail.com Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/1b7520ddb168e1d537d64be47414a0629d0d8f8f.1732581026.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: io_uring/rsrc.c io_uring/memmap.c [Conflicts due to io_pin_pages() move to io_uring/memmap.c in mainline] Signed-off-by: Long Li <leo.lilong@huawei.com> | 1 年前 | |
io_uring: drop any code related to SCM_RIGHTS stable inclusion from stable-v6.6.23 commit 6fc19b3d8a45ff0e5d50ec8184cee1d5eac1a8ba bugzilla: https://gitee.com/openeuler/kernel/issues/I9MPZ8 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=6fc19b3d8a45ff0e5d50ec8184cee1d5eac1a8ba -------------------------------- This is dead code after we dropped support for passing io_uring fds over SCM_RIGHTS, get rid of it. Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: ZhangPeng <zhangpeng362@huawei.com> | 2 年前 | |
fs: Initial atomic write support mainline inclusion from mainline-v6.10-rc3 commit c34fc6f26ab86d03a2d47446f42b6cd492dfdc56 category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/8862 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c34fc6f26ab86d03a2d47446f42b6cd492dfdc56 -------------------------------- An atomic write is a write issued with torn-write protection, meaning that for a power failure or any other hardware failure, all or none of the data from the write will be stored, but never a mix of old and new data. Userspace may add flag RWF_ATOMIC to pwritev2() to indicate that the write is to be issued with torn-write prevention, according to special alignment and length rules. For any syscall interface utilizing struct iocb, add IOCB_ATOMIC for iocb->ki_flags field to indicate the same. A call to statx will give the relevant atomic write info for a file: - atomic_write_unit_min - atomic_write_unit_max - atomic_write_segments_max Both min and max values must be a power-of-2. Applications can avail of atomic write feature by ensuring that the total length of a write is a power-of-2 in size and also sized between atomic_write_unit_min and atomic_write_unit_max, inclusive. Applications must ensure that the write is at a naturally-aligned offset in the file wrt the total write length. The value in atomic_write_segments_max indicates the upper limit for IOV_ITER iovcnt. Add file mode flag FMODE_CAN_ATOMIC_WRITE, so files which do not have the flag set will have RWF_ATOMIC rejected and not just ignored. Add a type argument to kiocb_set_rw_flags() to allows reads which have RWF_ATOMIC set to be rejected. Helper function generic_atomic_write_valid() can be used by FSes to verify compliant writes. There we check for iov_iter type is for ubuf, which implies iovcnt==1 for pwritev2(), which is an initial restriction for atomic_write_segments_max. Initially the only user will be bdev file operations write handler. We will rely on the block BIO submission path to ensure write sizes are compliant for the bdev, so we don't need to check atomic writes sizes yet. Signed-off-by: Prasad Singamsetty <prasad.singamsetty@oracle.com> jpg: merge into single patch and much rewrite Acked-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: John Garry <john.g.garry@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Link: https://lore.kernel.org/r/20240620125359.2684798-4-john.g.garry@oracle.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: fs/read_write.c include/linux/fs.h include/uapi/linux/fs.h io_uring/rw.c Signed-off-by: Long Li <leo.lilong@huawei.com> | 2 个月前 | |
io_uring: avoid indirect function calls for the hottest task_work We use task_work for a variety of reasons, but doing completions or triggering rety after poll are by far the hottest two. Use the indirect funtion call wrappers to avoid the indirect function call if CONFIG_RETPOLINE is set. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: silence variable ‘prev’ set but not used warning If io_uring.o is built with W=1, it triggers a warning: io_uring/io_uring.c: In function ‘__io_submit_flush_completions’: io_uring/io_uring.c:1502:40: warning: variable ‘prev’ set but not used [-Wunused-but-set-variable] 1502 | struct io_wq_work_node *node, *prev; | ^~~~ which is due to the wq_list_for_each() iterator always keeping a 'prev' variable. Most users need this to remove an entry from a list, for example, but __io_submit_flush_completions() never does that. Add a basic helper that doesn't track prev instead, and use that in that function. Reported-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> Reviewed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/splice: use fput() directly No point in using io_file_put() here, as we need to check if it's a fixed file in the caller anyway. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 2 年前 | |
io_uring: split out splice related operations This splits out splice and tee support. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: Support forcing sq thread to be idle and woken up by hrtimer euleros inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/8500 --------------------------------------------- This patch adds one option IORING_SETUP_SQ_THREAD_IDLE for io_uring user program to reduce cpu usage of sq thread. When enabled, sq polling thread will try to be idle. a hrtimer will be created to wake up the sq thread periodically, the period can be set by io_uring_params.sq_thread_wakeup_period(unit: us, default 10ms). Signed-off-by: Chen Zhen <chenzhen126@huawei.com> | 4 个月前 | |
io_uring: Support forcing sq thread to be idle and woken up by hrtimer euleros inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/8500 --------------------------------------------- This patch adds one option IORING_SETUP_SQ_THREAD_IDLE for io_uring user program to reduce cpu usage of sq thread. When enabled, sq polling thread will try to be idle. a hrtimer will be created to wake up the sq thread periodically, the period can be set by io_uring_params.sq_thread_wakeup_period(unit: us, default 10ms). Signed-off-by: Chen Zhen <chenzhen126@huawei.com> | 4 个月前 | |
io_uring: for requests that require async, force it Some requests require being run async as they do not support non-blocking. Instead of trying to issue these requests, getting -EAGAIN and then queueing them for async issue, rather just force async upfront. Add WARN_ON_ONCE to make sure surprising code paths do not come up, however in those cases the bug would end up being a blocking io_uring_enter(2) which should not be critical. Signed-off-by: Dylan Yudaken <dylany@meta.com> Link: https://lore.kernel.org/r/20230127135227.3646353-3-dylany@meta.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: move statx handling to its own file Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/sync: validate passed in offset stable inclusion from stable-v6.6.128 commit ca762fa01f64687b10050c038b8b9bcc84cfa872 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/ Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ca762fa01f64687b10050c038b8b9bcc84cfa872 -------------------------------- commit ca762fa01f64687b10050c038b8b9bcc84cfa872 upstream. [ Upstream commit 649dd18f559891bdafc5532d737c7dfb56060a6d ] Check if the passed in offset is negative once cast to sync->off. This ensures that -EINVAL is returned for that case, like it would be for sync_file_range(2). Fixes: c992fe2925d7 ("io_uring: add fsync support") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 个月前 | |
io_uring: split out fs related sync/fallocate functions This splits out sync_file_range, fsync, and fallocate. Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/tctx: work around xa_store() allocation error issue mainline inclusion from mainline-v6.10-rc2 commit 7eb75ce7527129d7f1fee6951566af409a37a1c4 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IBEAP3 CVE: CVE-2024-56584 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7eb75ce7527129d7f1fee6951566af409a37a1c4 -------------------------------- syzbot triggered the following WARN_ON: WARNING: CPU: 0 PID: 16 at io_uring/tctx.c:51 __io_uring_free+0xfa/0x140 io_uring/tctx.c:51 which is the WARN_ON_ONCE(!xa_empty(&tctx->xa)); sanity check in __io_uring_free() when a io_uring_task is going through its final put. The syzbot test case includes injecting memory allocation failures, and it very much looks like xa_store() can fail one of its memory allocations and end up with ->head being non-NULL even though no entries exist in the xarray. Until this issue gets sorted out, work around it by attempting to iterate entries in our xarray, and WARN_ON_ONCE() if one is found. Reported-by: syzbot+cc36d44ec9f368e443d3@syzkaller.appspotmail.com Link: https://lore.kernel.org/io-uring/673c1643.050a0220.87769.0066.GAE@google.com/ Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Long Li <leo.lilong@huawei.com> | 1 年前 | |
io_uring: simplify __io_uring_add_tctx_node Remove submitter parameter from __io_uring_add_tctx_node. It was only called from one place, and we can do that logic in that one place. Signed-off-by: Dylan Yudaken <dylany@fb.com> Fixes: 97bbdc06a444 ("io_uring: add IORING_SETUP_SINGLE_ISSUER") Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/timeout: check unused sqe fields stable inclusion from stable-v6.6.140 commit 44100ed1bdce2f4f983c127495dca681223aa97e category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/ Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=44100ed1bdce2f4f983c127495dca681223aa97e -------------------------------- commit 484ae637a3e3d909718de7c07afd3bb34b6b8504 upstream. Zero check unused SQE fields addr3 and pad2 for timeout and timeout update requests. They're not needed now, but could be used sometime in the future. Cc: stable@vger.kernel.org Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Wang Hai <wanghai38@huawei.com> | 1 个月前 | |
io_uring: remove unused return from io_disarm_next We removed conditional io_commit_cqring_flush() guarding against spurious eventfd and the io_disarm_next()'s return value is not used anymore, just void it. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/9a441c9a32a58bcc586076fa9a7d0dc33f1fb3cb.1662652536.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring/cmd: let cmds to know about dying task mainline inclusion from mainline-v6.10-rc2 commit df3b8ca604f224eb4cd51669416ad4d607682273 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICJPON Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=df3b8ca604f224eb4cd51669416ad4d607682273 -------------------------------- When the taks that submitted a request is dying, a task work for that request might get run by a kernel thread or even worse by a half dismantled task. We can't just cancel the task work without running the callback as the cmd might need to do some clean up, so pass a flag instead. If set, it's not safe to access any task resources and the callback is expected to cancel the cmd ASAP. Reviewed-by: Jens Axboe <axboe@kernel.dk> Reviewed-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: David Sterba <dsterba@suse.com> Conflicts: include/linux/io_uring_types.h [Context differences.] Signed-off-by: Yifan Qiao <qiaoyifan4@huawei.com> Signed-off-by: Long Li <leo.lilong@huawei.com> | 7 个月前 | |
io_uring/cmd: move io_uring_try_cancel_uring_cmd() mainline inclusion from mainline-v6.9-rc4 commit da12d9ab5889b87429d9375748dcd1485b6241f3 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICJPON Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=da12d9ab5889b87429d9375748dcd1485b6241f3 -------------------------------- io_uring_try_cancel_uring_cmd() is a part of the cmd handling so let's move it closer to all cmd bits into uring_cmd.c Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Reviewed-by: Ming Lei <ming.lei@redhat.com> Tested-by: Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/43a3937af4933655f0fd9362c381802f804f43de.1710799188.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: io_uring/io_uring.c [Context differences.] Signed-off-by: Yifan Qiao <qiaoyifan4@huawei.com> Signed-off-by: Long Li <leo.lilong@huawei.com> | 7 个月前 | |
io_uring: for requests that require async, force it Some requests require being run async as they do not support non-blocking. Instead of trying to issue these requests, getting -EAGAIN and then queueing them for async issue, rather just force async upfront. Add WARN_ON_ONCE to make sure surprising code paths do not come up, however in those cases the bug would end up being a blocking io_uring_enter(2) which should not be critical. Signed-off-by: Dylan Yudaken <dylany@meta.com> Link: https://lore.kernel.org/r/20230127135227.3646353-3-dylany@meta.com Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 | |
io_uring: move xattr related opcodes to its own file Signed-off-by: Jens Axboe <axboe@kernel.dk> | 3 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 1 个月前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 5 个月前 | ||
| 3 年前 | ||
| 1 个月前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 1 个月前 | ||
| 2 年前 | ||
| 30 天前 | ||
| 4 个月前 | ||
| 1 个月前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 5 个月前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 5 个月前 | ||
| 3 年前 | ||
| 5 个月前 | ||
| 3 年前 | ||
| 1 个月前 | ||
| 3 年前 | ||
| 7 个月前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 个月前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 4 个月前 | ||
| 4 个月前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 1 个月前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 1 个月前 | ||
| 3 年前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 3 年前 | ||
| 3 年前 |