HAproxy 技术分享

简介

HAProxy是一款提供高可用性、负载均衡以及基于TCP（第四层）和HTTP（第七层）应用的代理软件

---

Features

1.免费

2.能够做到4层以上代理

3.高性能

4.高稳定性

---

使用案例

淘宝CDN(HTTP反向代理)

---

测试:

HTTP代理 ab -i -c 500 -n 100000

       | --- node 8910 URL = /

HAproxy| --- node 8911 URL = /

       | --- node 8912 URL = /

       | --- node 8913 /test/ (reqisetbe ^[^\ ]*\ /(test|uri)/ server_uri_route) #按照规则转发

####### haproxy :  (单独由haproxy进行均衡负载)

Concurrency Level:      500

Time taken for tests:   32.562 seconds

Complete requests:      100000

Failed requests:        0

Write errors:           0

Total transferred:      36606588 bytes

HTML transferred:       0 bytes

Requests per second:    3071.02 [#/sec] (mean)

Time per request:       162.812 [ms] (mean)

Time per request:       0.326 [ms] (mean, across all concurrent requests)

Transfer rate:          1097.85 [Kbytes/sec] received

####### nginx : (单独由nginx进行均衡负载)

Concurrency Level:      500

Time taken for tests:   36.539 seconds

Complete requests:      100000

Failed requests:        0

Write errors:           0

Total transferred:      38600000 bytes

HTML transferred:       0 bytes

Requests per second:    2736.82 [#/sec] (mean)

Time per request:       182.694 [ms] (mean)

Time per request:       0.365 [ms] (mean, across all concurrent requests)

Transfer rate:          1031.65 [Kbytes/sec] received

---

对比Nginx

Name	Nginx	HAproxy
HTTP代理性能	9	10
TCP 代理性能	0	10
稳定性	10	10
转发规则	10	7
HTTP代理性能	9	10
平滑升级	10	8

原理 epoll 转发

验证 转发方式为如下

通过TCP代理请求 www.baidu.com

client                   HAProxy                     Backend

curl |----------------->|

                         accept(client)

                         recvfrom(client)

                         sendto(Backent)

                                          |-------->|

                                                     dowith(HAProxy)

                                          |<--------|

                         recvfrom(Backend)

                         sendto(Client)

     |<-----------------|

strace 查看系统调用

    zhangbo3@vm-222:/etc/haproxy$ sudo strace -p 7876

    epoll_wait(0, {{EPOLLIN, {u32=5, u64=5}}}, 7, 1000) = 1

    *** (事件循环，监听连接事件)

    accept(5, {

        sa_family=AF_INET,

        sin_port=htons(56479),

        sin_addr=inet_addr("127.0.0.1")

    }, [16]) = 1

    *** (来自客户端[127.0.0.1]的请求, 端口56479[随机端口])

    fcntl(1, F_SETFL, O_RDONLY|O_NONBLOCK)  = 0

    *** (设置连接socket为非阻塞)

    setsockopt(1, SOL_TCP, TCP_NODELAY, [1], 4) = 0

    *** (设置TCP连接为 NODELAY, 禁止Nagle算法)

    accept(5, 0x7fff8da2d7a0, [128]) = -1 EAGAIN

    (Resource temporarily unavailable)

    *** (再次去accept连接失败，因为accept被循环包裹，需要返回 EAGAIN才break)

    recvfrom(1, "GET HTTP://www.baidu.com HTTP/1."..., 8192) = 212

    *** (收到来自客户端的请求 curl -x 127.0.0.1:1180 www.baidu.com)

    recvfrom(1, 0x2560ac4, 7980, 0, 0, 0)   = -1 EAGAIN

    (Resource temporarily unavailable)

    *** (返回EAGAIN后break,表示收到完整的TCP数据, 开始处理请求数据)

    epoll_ctl(0, EPOLL_CTL_ADD, 1, {EPOLLIN, {u32=1, u64=1}}) = 0

    *** (将来自客户端的socket加入epoll监听队列)

    socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 2

    fcntl(2, F_SETFL, O_RDONLY|O_NONBLOCK)  = 0

    setsockopt(2, SOL_TCP, TCP_NODELAY, [1], 4) = 0

    connect(2, {

        sa_family=AF_INET,

        sin_port=htons(8912),

        sin_addr=inet_addr("127.0.0.1")

    }, 16) = -1 EINPROGRESS (Operation now in progress)

    *** (创建到backend的socket连接,这里backend配置为8912)

    *** (server server3 127.0.0.1:8912)

    sendto(2, "GET HTTP://www.baidu.com HTTP/1."...,

    212, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 212

    *** (发送数据到backend)

    epoll_wait(0, {}, 7, 0)                 = 0

    *** (继续进入epoll_wait,等待来自backend的数据返回)

    gettimeofday({1413020002, 322710}, NULL) = 0

    recvfrom(2, 0x255c960, 16384, 0, 0, 0)  = -1 EAGAIN

    (Resource temporarily unavailable)

    *** (Backend数据来了)

    epoll_ctl(0, EPOLL_CTL_ADD, 2, {EPOLLIN, {u32=2, u64=2}}) = 0

    epoll_wait(0, {{EPOLLIN, {u32=2, u64=2}}}, 7, 1000) = 1

    gettimeofday({1413020003, 308930}, NULL) = 0

    recvfrom(2, "HTTP/1.1 200 OK\r\nDate: Sat, 11 O"...,16384) = 2896

    *** (backend 返回 HTTP 200, 这是标准HTTP协议头)

    recvfrom(2, 0x255d4b0, 13488, 0, 0, 0)  = -1 EAGAIN

    (Resource temporarily unavailable)

    sendto(1, "HTTP/1.1 200 OK\r\nDate: Sat, 11 O"...) = 2896

    *** (将backend返回的数据发送到front end)

    epoll_wait(0, {{EPOLLIN, {u32=2, u64=2}}}, 7, 1000) = 1

    *** (继续进入epoll_wait)

    gettimeofday({1413020003, 309695}, NULL) = 0

    *** (客户端连接超时，关闭连接)

    shutdown(2, 1 /* send */)               = 0

    close(2)                                = 0

    shutdown(1, 1 /* send */)               = 0

    close(1)                                = 0

验证单进程模型 -> 查看线程数

cat /proc/7878/status

    Name:   haproxy

    State:  S (sleeping)

    ...

    Threads:        1

    SigQ:   0/15594

    SigPnd: 0000000000000000

    ShdPnd: 0000000000000000

    ...