Looks like your client ran out of ephemeral port or there's some problem with your client environment.
Are you using Windows?
You can possibly do at least the following:
- Windows: look into this article for solution for Windows system as host for jmeter.
- Use Linux system instead as host to run you Jmeter load-scenarios.
As well you possibly will find this article useful for your testing activities (I've seen Jboss in tags).
UPDATE:
Once more from linked article above:
When an HTTP request is made, an ephemeral port is allocated for the TCP / IP connection. The ephemeral port range is 32678 – 61000. After the client closes the connection, the connection is placed in the TIME-WAIT state for 60 seconds.
If JMeter (HttpClient) is sending thousands of HTTP requests per second and creating new TCP / IP connections, the system will run out of available ephemeral ports for allocation.
. . .
Otherwise, the following messages may appear in the JMeter JTL files:
Non HTTP response code: java.net.BindException
Non HTTP response message: Address already in use
The solution is to enable fast recycling TIME_WAIT sockets.
echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
Other options include TCP_FIN_TIMEOUT to reduce how long a connection is placed in the TIME_WAIT state and TCP_TW_REUSE to allow the system to reuse connections placed in the TIME_WAIT state.
On the server's side:
-
This enables fast recycling of TIME_WAIT sockets
/sbin/sysctl -w net.ipv4.tcp_tw_recycle=1
-
This allows reusing sockets in TIME_WAIT state for new connections - a safer alternative to tcp_tw_recycle
/sbin/sysctl -w net.ipv4.tcp_tw_reuse=1
The tcp_tw_reuse setting is particularly useful in environments where numerous short connections are open and left in TIME_WAIT state, such as web-servers. Reusing the sockets can be very effective in reducing server load.
-
Maximum number of timewait sockets held by system simultaneously
/sbin/sysctl -w net.ipv4.tcp_max_tw_buckets=30000
or the same but in another way - add the lines below to the /etc/sysctl.conf file so that the change survives reboot:
net.ipv4.tcp_max_tw_buckets = 30000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
As well on the server's side look onto result of ulimit -n.
Default value is 1024 for the limit of max open files, which can explain appearance of BindExceptions at 1000 connections.
As well you can monitor number of connections between the server and jmeter during test-run using e.g.
netstat -an | grep SERVER_PORT | wc -l
to define limit of connections - if any.