欢迎您访问 最编程 本站为您分享编程语言代码,编程技术文章!
您现在的位置是: 首页

【verbs】ibv_reg_mr

最编程 2024-07-24 12:10:33
...

描述


ibv_reg_mr() 注册一个与保护域(pd)关联的内存区域 (MR)。通过这样做,允许 RDMA 设备向该内存读取和写入数据。执行此注册需要一些时间,因此当需要快速响应时,不建议在数据路径中执行内存注册。
每次成功注册都会产生一个具有唯一(在特定 RDMA 设备内)lkey 和 rkey 值的 MR。
MR 的起始地址是 addr,它的大小是 length。可以注册的块的最大大小限制为 device_attr.max_mr_size。调用它的进程空间中每个的虚拟内存地址都可以注册,包括但不限于:
本地内存(变量或数组)
全局内存(变量或数组)
动态分配的内存(使用 malloc() 或 mmap())
共享内存

来自text segment的地址

注册的内存缓冲区不必是页面对齐的。没有任何方法可以知道可以为特定设备注册的内存总大小是多少。
参数 access 描述了 RDMA 设备所需的内存访问属性。它是 0 或以下一个或多个标志的按位或:

V_ACCESS_LOCAL_WRITE Enable Local Write Access: Memory Region can be used in Receive Requests or in IBV_WR_ATOMIC_CMP_AND_SWP or
IBV_WR_ATOMIC_FETCH_AND_ADD to write locally the remote content value
IBV_ACCESS_REMOTE_WRITE Enable Remote Write Access: Memory Region can be access from remote context using IBV_WR_RDMA_WRITE or
IBV_WR_RDMA_WRITE_WITH_IMM
IBV_ACCESS_REMOTE_READ Enable Remote Read Access: Memory Region can be access from remote context using IBV_WR_RDMA_READ
IBV_ACCESS_REMOTE_ATOMIC Enable Remote Atomic Operation Access (if supported): Memory Region can be access from remote context using IBV_WR_ATOMIC_CMP_AND_SWP or
IBV_WR_ATOMIC_FETCH_AND_ADD
IBV_ACCESS_MW_BIND Enable Memory Window Binding

如果设置了 IBV_ACCESS_REMOTE_WRITE 或 IBV_ACCESS_REMOTE_ATOMIC,则也必须设置 IBV_ACCESS_LOCAL_WRITE,因为只有在允许本地写入时才允许远程写入。
MR 始终启用本地读取访问,即可以使用 IBV_WR_SEND、IBV_WR_SEND_WITH_IMM、IBV_WR_RDMA_WRITE、IBV_WR_RDMA_WRITE_WITH_IMM 在本地读取内存区域。
内存注册请求的权限可以是该内存块的操作系统权限的全部或子集。例如:只读内存不能注册有写权限(本地或远程)。
一个特定的进程可以注册一个或多个内存区域。

参数

Name Direction Description
pd in Protection Domain that was returned from ibv_alloc_pd()
addr in The start address of the virtual contiguous memory block
length in Size of the memory block to register, in bytes. This value must be at least 1 and less than dev_cap.max_mr_size
access in Requested access permissions for the memory region

返回值

Value Description
MR Pointer to the newly allocated Memory Region.
This pointer also contains the following fields:
lkey The value that will be used to refer to this MR using a local access
rkey The value that will be used to refer to this MR using a remote access

Those values may be equal, but this isn't always guaranteed.

NULL On failure, errno indicates the failure reason:
EINVAL Invalid access value
ENOMEM Not enough resources (either in operating system or in RDMA device) to complete this operation

Examples

Register a MR to allow only local read and write access and deregister it:

struct ibv_pd *pd;
struct ibv_mr *mr;
 
mr = ibv_reg_mr(pd, buf, size, IBV_ACCESS_LOCAL_WRITE);
if (!mr) {
	fprintf(stderr, "Error, ibv_reg_mr() failed\n");
	return -1;
}
 
if (ibv_dereg_mr(mr)) {
	fprintf(stderr, "Error, ibv_dereg_mr() failed\n");
	return -1;
}

Register a MR to allow Remote read and write to it:

mr = ibv_reg_mr(pd, buf, size, IBV_ACCESS_LOCAL_WRITE |
		IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ);
if (!mr) {
	fprintf(stderr, "Error, ibv_reg_mr() failed\n");
	return -1;
}

原文:https://www.rdmamojo.com/2012/09/07/ibv_reg_mr/