| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
dma: pswiotlb: Move platform-identification to cpu errata phytium inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICBHX3 CVE: NA ---------------------------------------------------------- The judgment process whether the CPU can use the PSWIOTLB is placed in the initialization phase of cpu features by cpu errata. Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn> Signed-off-by: Jiakun Shuai <shuaijiakun1288@phytium.com.cn> | 9 个月前 | |
dma: phytium: Add PSWIOTLB mechanism to improve DMA performance phytium inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICBHX3 CVE: NA ---------------------------------------------------------- This patch added additional "memory copy" to improve D2H direction DMA performance on Phytium Server SoCs. Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn> Signed-off-by: Jiakun Shuai <shuaijiakun1288@phytium.com.cn> | 9 个月前 | |
dma: phytium: Add PSWIOTLB mechanism to improve DMA performance phytium inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICBHX3 CVE: NA ---------------------------------------------------------- This patch added additional "memory copy" to improve D2H direction DMA performance on Phytium Server SoCs. Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn> Signed-off-by: Jiakun Shuai <shuaijiakun1288@phytium.com.cn> | 9 个月前 | |
dma-mapping: clear dev->dma_mem to NULL after freeing it stable inclusion from stable-v5.10.209 commit 64299791d070fc07e897783d0de75f530d460066 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9U3NW Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=64299791d070fc07e897783d0de75f530d460066 -------------------------------- [ Upstream commit b07bc2347672cc8c7293c64499f1488278c5ca3d ] Reproduced with below sequence: dma_declare_coherent_memory()->dma_release_coherent_memory() ->dma_declare_coherent_memory()->"return -EBUSY" error It will return -EBUSY from the dma_assign_coherent_memory() in dma_declare_coherent_memory(), the reason is that dev->dma_mem pointer has not been set to NULL after it's freed. Fixes: cf65a0f6f6ff ("dma-mapping: move all DMA mapping code to kernel/dma") Signed-off-by: Joakim Zhang <joakim.zhang@cixtech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: sanglipeng1 <sanglipeng1@jd.com> | 2 年前 | |
dma: phytium: Add PSWIOTLB mechanism to improve DMA performance phytium inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICBHX3 CVE: NA ---------------------------------------------------------- This patch added additional "memory copy" to improve D2H direction DMA performance on Phytium Server SoCs. Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn> Signed-off-by: Jiakun Shuai <shuaijiakun1288@phytium.com.cn> | 9 个月前 | |
dma-debug: don't call __dma_entry_alloc_check_leak() under free_entries_lock stable inclusion from stable-v5.10.198 commit c79300599923daaa30f417c75555d5566b3d31ae category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I987V5 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=c79300599923daaa30f417c75555d5566b3d31ae -------------------------------- [ Upstream commit fb5a4315591dae307a65fc246ca80b5159d296e1 ] __dma_entry_alloc_check_leak() calls into printk -> serial console output (qcom geni) and grabs port->lock under free_entries_lock spin lock, which is a reverse locking dependency chain as qcom_geni IRQ handler can call into dma-debug code and grab free_entries_lock under port->lock. Move __dma_entry_alloc_check_leak() call out of free_entries_lock scope so that we don't acquire serial console's port->lock under it. Trimmed-down lockdep splat: The existing dependency chain (in reverse order) is: -> #2 (free_entries_lock){-.-.}-{2:2}: _raw_spin_lock_irqsave+0x60/0x80 dma_entry_alloc+0x38/0x110 debug_dma_map_page+0x60/0xf8 dma_map_page_attrs+0x1e0/0x230 dma_map_single_attrs.constprop.0+0x6c/0xc8 geni_se_rx_dma_prep+0x40/0xcc qcom_geni_serial_isr+0x310/0x510 __handle_irq_event_percpu+0x110/0x244 handle_irq_event_percpu+0x20/0x54 handle_irq_event+0x50/0x88 handle_fasteoi_irq+0xa4/0xcc handle_irq_desc+0x28/0x40 generic_handle_domain_irq+0x24/0x30 gic_handle_irq+0xc4/0x148 do_interrupt_handler+0xa4/0xb0 el1_interrupt+0x34/0x64 el1h_64_irq_handler+0x18/0x24 el1h_64_irq+0x64/0x68 arch_local_irq_enable+0x4/0x8 ____do_softirq+0x18/0x24 ... -> #1 (&port_lock_key){-.-.}-{2:2}: _raw_spin_lock_irqsave+0x60/0x80 qcom_geni_serial_console_write+0x184/0x1dc console_flush_all+0x344/0x454 console_unlock+0x94/0xf0 vprintk_emit+0x238/0x24c vprintk_default+0x3c/0x48 vprintk+0xb4/0xbc _printk+0x68/0x90 register_console+0x230/0x38c uart_add_one_port+0x338/0x494 qcom_geni_serial_probe+0x390/0x424 platform_probe+0x70/0xc0 really_probe+0x148/0x280 __driver_probe_device+0xfc/0x114 driver_probe_device+0x44/0x100 __device_attach_driver+0x64/0xdc bus_for_each_drv+0xb0/0xd8 __device_attach+0xe4/0x140 device_initial_probe+0x1c/0x28 bus_probe_device+0x44/0xb0 device_add+0x538/0x668 of_device_add+0x44/0x50 of_platform_device_create_pdata+0x94/0xc8 of_platform_bus_create+0x270/0x304 of_platform_populate+0xac/0xc4 devm_of_platform_populate+0x60/0xac geni_se_probe+0x154/0x160 platform_probe+0x70/0xc0 ... -> #0 (console_owner){-...}-{0:0}: __lock_acquire+0xdf8/0x109c lock_acquire+0x234/0x284 console_flush_all+0x330/0x454 console_unlock+0x94/0xf0 vprintk_emit+0x238/0x24c vprintk_default+0x3c/0x48 vprintk+0xb4/0xbc _printk+0x68/0x90 dma_entry_alloc+0xb4/0x110 debug_dma_map_sg+0xdc/0x2f8 __dma_map_sg_attrs+0xac/0xe4 dma_map_sgtable+0x30/0x4c get_pages+0x1d4/0x1e4 [msm] msm_gem_pin_pages_locked+0x38/0xac [msm] msm_gem_pin_vma_locked+0x58/0x88 [msm] msm_ioctl_gem_submit+0xde4/0x13ac [msm] drm_ioctl_kernel+0xe0/0x15c drm_ioctl+0x2e8/0x3f4 vfs_ioctl+0x30/0x50 ... Chain exists of: console_owner --> &port_lock_key --> free_entries_lock Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(free_entries_lock); lock(&port_lock_key); lock(free_entries_lock); lock(console_owner); *** DEADLOCK *** Call trace: dump_backtrace+0xb4/0xf0 show_stack+0x20/0x30 dump_stack_lvl+0x60/0x84 dump_stack+0x18/0x24 print_circular_bug+0x1cc/0x234 check_noncircular+0x78/0xac __lock_acquire+0xdf8/0x109c lock_acquire+0x234/0x284 console_flush_all+0x330/0x454 console_unlock+0x94/0xf0 vprintk_emit+0x238/0x24c vprintk_default+0x3c/0x48 vprintk+0xb4/0xbc _printk+0x68/0x90 dma_entry_alloc+0xb4/0x110 debug_dma_map_sg+0xdc/0x2f8 __dma_map_sg_attrs+0xac/0xe4 dma_map_sgtable+0x30/0x4c get_pages+0x1d4/0x1e4 [msm] msm_gem_pin_pages_locked+0x38/0xac [msm] msm_gem_pin_vma_locked+0x58/0x88 [msm] msm_ioctl_gem_submit+0xde4/0x13ac [msm] drm_ioctl_kernel+0xe0/0x15c drm_ioctl+0x2e8/0x3f4 vfs_ioctl+0x30/0x50 ... Reported-by: Rob Clark <robdclark@chromium.org> Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org> Acked-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: sanglipeng <sanglipeng1@jd.com> | 2 年前 | |
dma-mapping: move dma-debug.h to kernel/dma/ Most of dma-debug.h is not required by anything outside of kernel/dma. Move the four declarations needed by dma-mappin.h or dma-ops providers into dma-mapping.h and dma-map-ops.h, and move the remainder of the file to kernel/dma/debug.h. Signed-off-by: Christoph Hellwig <hch@lst.de> | 5 年前 | |
dma-direct: always leak memory that can't be re-encrypted mainline inclusion from mainline-v5.15 commit a90cf30437489343b8386ae87b4827b6d6c3ed50 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ID7GTW CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a90cf30437489343b8386ae87b4827b6d6c3ed50 --------------------------- We must never let unencrypted memory go back into the general page pool. So if we fail to set it back to encrypted when freeing DMA memory, leak the memory instead and warn the user. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Zhiguang Ni <nizhiguang@hygon.cn> | 7 个月前 | |
dma-direct: avoid redundant memory sync for swiotlb stable inclusion from stable-v5.10.112 commit 26f827e095aba4141ffd684276ef54025de27574 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I5HL0X Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=26f827e095aba4141ffd684276ef54025de27574 -------------------------------- commit 9e02977bfad006af328add9434c8bffa40e053bb upstream. When we looked into FIO performance with swiotlb enabled in VM, we found swiotlb_bounce() is always called one more time than expected for each DMA read request. It turns out that the bounce buffer is copied to original DMA buffer twice after the completion of a DMA request (one is done by in dma_direct_sync_single_for_cpu(), the other by swiotlb_tbl_unmap_single()). But the content in bounce buffer actually doesn't change between the two rounds of copy. So, one round of copy is redundant. Pass DMA_ATTR_SKIP_CPU_SYNC flag to swiotlb_tbl_unmap_single() to skip the memory copy in it. This fix increases FIO 64KB sequential read throughput in a guest with swiotlb=force by 5.6%. Fixes: 55897af63091 ("dma-direct: merge swiotlb_dma_ops into the dma_direct code") Reported-by: Wang Zhaoyang1 <zhaoyang1.wang@intel.com> Reported-by: Gao Liang <liang.gao@intel.com> Signed-off-by: Chao Gao <chao.gao@intel.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Acked-by: Xie XiuQi <xiexiuqi@huawei.com> | 3 年前 | |
dma-mapping: split <linux/dma-mapping.h> Split out all the bits that are purely for dma_map_ops implementations and related code into a new <linux/dma-map-ops.h> header so that they don't get pulled into all the drivers. That also means the architecture specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h> any more, which leads to a missing includes that were pulled in by the x86 or arm versions in a few not overly portable drivers. Signed-off-by: Christoph Hellwig <hch@lst.de> | 5 年前 | |
dma-mapping: benchmark: Add support for multi-pages map/unmap mainline inclusion from mainline-v5.13-rc1 commit ca947482b0b30443e6da1f0f5ba7244e34a4f65a category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/ID5X4C Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ca947482b0b30443e6da1f0f5ba7244e34a4f65a ---------------------------------------- Currently it only support one page map/unmap once a time for dma-map benchmark, but there are some other scenaries which need to support for multi-page map/unmap: for those multi-pages interfaces such as dma_alloc_coherent() and dma_map_sg(), the time spent on multi-pages map/unmap is not the time of a single page * npages (not linear) as it may use block description instead of page description when it is satified with the size such as 2M/1G, and also it can send a single TLB invalidation command to invalidate multi-pages instead of multi-times when RIL is enabled (which will short the time of unmap). So it is necessary to add support for multi-pages map/unmap. Add a parameter "-g" to support multi-pages map/unmap. Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com> Acked-by: Barry Song <song.bao.hua@hisilicon.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Conflicts: kernel/dma/map_benchmark.c [fix context conflicts] Signed-off-by: Lin Yujun <linyujun809@h-partners.com> | 7 个月前 | |
dma: pswiotlb: Move pswiotlb dma functions behind dma_map_ops phytium inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICBHX3 CVE: NA ---------------------------------------------------------- To reduce the difficulty of maintenance, pswiotlb_dma* functions are moved behind the struct dma_map_ops. Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn> Signed-off-by: Jiakun Shuai <shuaijiakun1288@phytium.com.cn> | 9 个月前 | |
dma-mapping: handle vmalloc addresses in dma_common_{mmap,get_sgtable} stable inclusion from stable-5.10.54 commit 89837669031005da8644ecb1476936421aaaab40 bugzilla: 175586 https://gitee.com/openeuler/kernel/issues/I4DVDU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=89837669031005da8644ecb1476936421aaaab40 -------------------------------- [ Upstream commit 40ac971eab89330d6153e7721e88acd2d98833f9 ] xen-swiotlb can use vmalloc backed addresses for dma coherent allocations and uses the common helpers. Properly handle them to unbreak Xen on ARM platforms. Fixes: 1b65c4e5a9af ("swiotlb-xen: use xen_alloc/free_coherent_pages") Signed-off-by: Roman Skakun <roman_skakun@epam.com> Reviewed-by: Andrii Anisov <andrii_anisov@epam.com> [hch: split the patch, renamed the helpers] Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Chen Jun <chenjun102@huawei.com> Acked-by: Weilong Chen <chenweilong@huawei.com> Signed-off-by: Chen Jun <chenjun102@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> | 4 年前 | |
dma/pool: create dma atomic pool only if dma zone has managed pages stable inclusion from stable-v5.10.94 commit e04b1dfe15cefe465060e3ab1eb4328ce255e051 bugzilla: https://gitee.com/openeuler/kernel/issues/I531X9 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=e04b1dfe15cefe465060e3ab1eb4328ce255e051 -------------------------------- commit a674e48c5443d12a8a43c3ac42367aa39505d506 upstream. Currently three dma atomic pools are initialized as long as the relevant kernel codes are built in. While in kdump kernel of x86_64, this is not right when trying to create atomic_pool_dma, because there's no managed pages in DMA zone. In the case, DMA zone only has low 1M memory presented and locked down by memblock allocator. So no pages are added into buddy of DMA zone. Please check commit f1d4d47c5851 ("x86/setup: Always reserve the first 1M of RAM"). Then in kdump kernel of x86_64, it always prints below failure message: DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations swapper/0: page allocation failure: order:5, mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.13.0-0.rc5.20210611git929d931f2b40.42.fc35.x86_64 #1 Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.12.0 06/04/2018 Call Trace: dump_stack+0x7f/0xa1 warn_alloc.cold+0x72/0xd6 __alloc_pages_slowpath.constprop.0+0xf29/0xf50 __alloc_pages+0x24d/0x2c0 alloc_page_interleave+0x13/0xb0 atomic_pool_expand+0x118/0x210 __dma_atomic_pool_init+0x45/0x93 dma_atomic_pool_init+0xdb/0x176 do_one_initcall+0x67/0x320 kernel_init_freeable+0x290/0x2dc kernel_init+0xa/0x111 ret_from_fork+0x22/0x30 Mem-Info: ...... DMA: failed to allocate 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocation DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations Here, let's check if DMA zone has managed pages, then create atomic_pool_dma if yes. Otherwise just skip it. Link: https://lkml.kernel.org/r/20211223094435.248523-3-bhe@redhat.com Fixes: 6f599d84231f ("x86/kdump: Always reserve the low 1M when the crashkernel option is specified") Signed-off-by: Baoquan He <bhe@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Acked-by: John Donnelly <john.p.donnelly@oracle.com> Reviewed-by: David Hildenbrand <david@redhat.com> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: Robin Murphy <robin.murphy@arm.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Christoph Lameter <cl@linux.com> Cc: David Laight <David.Laight@ACULAB.COM> Cc: David Rientjes <rientjes@google.com> Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Acked-by: Xie XiuQi <xiexiuqi@huawei.com> | 4 年前 | |
dma-remap: use kvmalloc_array/kvfree for larger dma memory remap stable inclusion from stable-v5.10.192 commit 04e774fb678979a28ff82e860938e4e06829e268 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I933RF Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=04e774fb678979a28ff82e860938e4e06829e268 -------------------------------- [ Upstream commit 51ff97d54f02b4444dfc42e380ac4c058e12d5dd ] If dma_direct_alloc() alloc memory in size of 64MB, the inner function dma_common_contiguous_remap() will allocate 128KB memory by invoking the function kmalloc_array(). and the kmalloc_array seems to fail to try to allocate 128KB mem. Call trace: [14977.928623] qcrosvm: page allocation failure: order:5, mode:0x40cc0 [14977.928638] dump_backtrace.cfi_jt+0x0/0x8 [14977.928647] dump_stack_lvl+0x80/0xb8 [14977.928652] warn_alloc+0x164/0x200 [14977.928657] __alloc_pages_slowpath+0x9f0/0xb4c [14977.928660] __alloc_pages+0x21c/0x39c [14977.928662] kmalloc_order+0x48/0x108 [14977.928666] kmalloc_order_trace+0x34/0x154 [14977.928668] __kmalloc+0x548/0x7e4 [14977.928673] dma_direct_alloc+0x11c/0x4f8 [14977.928678] dma_alloc_attrs+0xf4/0x138 [14977.928680] gh_vm_ioctl_set_fw_name+0x3c4/0x610 [gunyah] [14977.928698] gh_vm_ioctl+0x90/0x14c [gunyah] [14977.928705] __arm64_sys_ioctl+0x184/0x210 work around by doing kvmalloc_array instead. Signed-off-by: Gao Xu <gaoxu2@hihonor.com> Reviewed-by: Suren Baghdasaryan <surenb@google.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: sanglipeng <sanglipeng1@jd.com> | 2 年前 | |
x86,swiotlb: Adjust SWIOTLB bounce buffer size for SEV guests mainline inclusion from mainline-v5.11 commit e998879d4fb7991856916972168cf27c0d86ed12 category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/8295 CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e998879d4fb7991856916972168cf27c0d86ed12 --------------------------- For SEV, all DMA to and from guest has to use shared (un-encrypted) pages. SEV uses SWIOTLB to make this happen without requiring changes to device drivers. However, depending on the workload being run, the default 64MB of it might not be enough and it may run out of buffers to use for DMA, resulting in I/O errors and/or performance degradation for high I/O workloads. Adjust the default size of SWIOTLB for SEV guests using a percentage of the total memory available to guest for the SWIOTLB buffers. Adds a new sev_setup_arch() function which is invoked from setup_arch() and it calls into a new swiotlb generic code function swiotlb_adjust_size() to do the SWIOTLB buffer adjustment. v5 fixed build errors and warnings as Reported-by: kbuild test robot <lkp@intel.com> Hygon-SIG: commit e998879d4fb7 upstream x86,swiotlb: Adjust SWIOTLB bounce buffer size for SEV guests Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> Co-developed-by: Borislav Petkov <bp@suse.de> Signed-off-by: Borislav Petkov <bp@suse.de> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Signed-off-by: Zhiguang Ni <nizhiguang@hygon.cn> | 5 个月前 | |
dma-mapping: split <linux/dma-mapping.h> Split out all the bits that are purely for dma_map_ops implementations and related code into a new <linux/dma-map-ops.h> header so that they don't get pulled into all the drivers. That also means the architecture specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h> any more, which leads to a missing includes that were pulled in by the x86 or arm versions in a few not overly portable drivers. Signed-off-by: Christoph Hellwig <hch@lst.de> | 5 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 9 个月前 | ||
| 9 个月前 | ||
| 9 个月前 | ||
| 2 年前 | ||
| 9 个月前 | ||
| 2 年前 | ||
| 5 年前 | ||
| 7 个月前 | ||
| 3 年前 | ||
| 5 年前 | ||
| 7 个月前 | ||
| 9 个月前 | ||
| 4 年前 | ||
| 4 年前 | ||
| 2 年前 | ||
| 5 个月前 | ||
| 5 年前 |