1. udaddy
This script covers RDMA_CM UD connections. (It establishes a set of unreliable RDMA datagram communication paths between two nodes using the librdmacm, optionally transfers datagrams between the nodes, then tears down the communication)
Run the following command on one server (act as a server):
#udaddy
Run the following command on the second server (act as a client)
# udaddy -s 12.12.12.1
udaddy: starting client
udaddy: connecting
initiating data transfers
receiving data transfers
data transfers complete
test complete
return status 0
"return status=0" means good exit (RDMA is running).
2. rdma_server, rdma_client commands
Another options is to use rdma_server and rdma_client commands:
Those commands are simple RDMA CM connection and ping-pong test (It uses synchronous librdmam calls to establish an RDMA connections between two nodes).
Run the following command on one server (act as a server):
#rdma_server
Run the following command on the second server (act as a client)
rdma_client -s 12.12.12.1
rdma_client: start
rdma_client: end 0
"rdma_client: end 0" means good exit (RDMA is running).
3. ib_send_bw (performance test)
Run pefformance test such as ib_send_bw, ib_read_bw or similar
For Example:
Run the following command on one server (act as a server):
# ib_send_bw -d mlx4_0 -i 1 -F --report_gbits
Run the following command on the second server (act as a client):
# ib_send_bw -d mlx4_0 -i 1 -F --report_gbits 12.12.12.1
---------------------------------------------------------------------------------------
Send BW Test
Dual-port : OFF Device : mlx4_0
Number of qps : 1 Transport type : IB
Connection type : RC
RX depth : 512
CQ Moderation : 100
Mtu : 1024[B]
Link type : Ethernet
Gid index : 0
Max inline data : 0[B]
rdma_cm QPs : OFF
Data ex. method : Ethernet
---------------------------------------------------------------------------------------
local address: LID 0000 QPN 0x0065 PSN 0xc8f367
GID: 254:128:00:00:00:00:00:00:246:82:20:255:254:23:27:129
remote address: LID 0000 QPN 0x005d PSN 0x884d7d
GID: 254:128:00:00:00:00:00:00:246:82:20:255:254:23:31:225
---------------------------------------------------------------------------------------
#bytes #iterations BW peak[Gb/sec] BW average[Gb/sec] MsgRate[Mpps]
65536 1000 0.00 36.40 0.069428
---------------------------------------------------------------------------------------
4. rping
This script covers RDMA_CM RC connections, but only userspace (It establishes a set of reliable RDMA connections between two nodes using the librdmacm, optionally transfers data between the nodes, then disconnects).
Run the following on one of the servers (act as a rping server)
# rping -s -C 10 -v
Run the following on one of the servers (act as a rping client)
# rping -c -a 12.12.12.1 -C 10 -v
ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu
ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv
ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx
ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy
ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA
client DISCONNECT EVENT...
5. ucmatose
This script covers RDMA_CM RC connections, but only userspace (same as rping) (It establishes a set of reliable RDMA connections between two nodes using the librdmacm, optionally transfers data between the nodes, then disconnects).
Run the following on one of the servers (act as a server)
# ucmatose
Run the following on the other server (act as a client)
#ucmatose -s 12.12.12.1
cmatose: starting client
cmatose: connecting
receiving data transfers
sending replies
data transfers complete
test complete
return status 0