Load Balancing

HAProxy:

HAProxy vs Nginx description

HAProxy vs Nginx latest graphs

HAProxy was simple to setup and configure for sticky sessions and x-forwarded-for source IP pass-through, see HOWTO

It was in the Ubuntu Synaptic Package Manager, so it was a click to install.

Edited the /etc/haproxy.cfg to configure

Started with:

sudo /etc/init.d/haproxy [start|stop]

Comes with admin stats that by default is on the same balancer front-end address and port, i.e., http://<balancer-address>:<port>/haproxy?stats
The docs indicate default is /admin?stats

Hot reconfig:

${HAPROXY} -p ${PIDFILE} -f ${CONFIG} ${HAPROXY_OPT} -sf $(cat ${PIDFILE})


# haproxy.cfg file example

global
 log 127.0.0.1 local0
 log 127.0.0.1 local1 notice
 #log loghost local0 info
 maxconn 4096
 #debug
 #quiet
 user haproxy
 group haproxy

defaults
 log global
 mode http
 option httplog
 option dontlognull
 retries 3
 redispatch
 maxconn 2000
 contimeout 5000
 clitimeout 50000
 srvtimeout 50000

listen eon 0.0.0.0:8282
 mode http
 stats enable
 stats uri /admin?stats
 stats auth haproxy:haproxy
 balance roundrobin
 cookie STICKY_SESSION_SERVERID insert indirect nocache
 option httpclose
 option forwardfor
 option httpchk HEAD /somepath/someHealthCheckPage.jsp HTTP/1.0
 server www1 10.60.54.76:8282 cookie www1 check inter 2000ms
 server www2 10.60.24.107:8282 cookie www2 check inter 2000ms
 server www3 10.60.25.254:8282 cookie www3 check inter 2000ms
 server www4 10.10.189.69:8282 cookie www4 check inter 2000ms


HAProxy logging requires syslog configuration: enabling syslog for remote logging and adding facility and level to syslog.conf.

To disable logging of external test performed on a known URI, use "monitor-uri" configuration option.


Modify apache log to record x-forwarded-for source-IP:

jboss-4.2.2.GA/server/<config>/deploy/jboss-web.deployer/server.xml



 prefix="access." suffix=".log"
 pattern="%{X-Forwarded-For}i %l %u "%{User-Agent}i" %t "%r" %s %b "%{Referer}i""
 directory="${jboss.server.home.dir}/log" resolveHosts="false" />



Configuring STunnel to provide https for HAProxy. It is installed and configured on the same server as HAProxy. STunnel receives the https connection decrypts it and forwards to HAProxy.

STunnel requires a patch to avoid losing source IP.

HOWTO patch

Comments

Post a Comment

Popular posts from this blog

Sites, Newsletters, and Blogs

BLOGS Java.net Weblogs Martin Fowler's blog Paul Graham Ruminations of a Programmer Form Follows Function Newsletters Core Java Tech Tips Enterprise Java Tech Tips JavaSpecialists.EU TheServerSide.com Sites Java.net Today Hot Java Sites CSS Zen Garden Evernote Hacker News Pragmatic Programmer
Read more

Oracle JDBC ReadTimeout QueryTimeout

If a query exceeds the oracle.jdbc.ReadTimeout without receiving any data, an exception is thrown and the connection is terminated by the Oracle driver on the client. Unfortunately the session will still be queued on the database and continue to wait for locks, hold any current locks, and complete any DML/PL*SQL procedures that are pending on the server-side of the orphaned connection. If an application, on a another connection, due to ReadTimeout exception, retries DML/PL*SQL  which requires locks, those queries will queue behind the initial DML/PL*SQL. This will increase the workload exacerbating the situation. On applications with retries, this can be observed by querying the v$session table  or gv$session on RAC and noting new sessions started periodically based on the ReadTimeout interval. The sessions may often have the same SQL_ID and/or SQL_HASH_VALUE. If stmt.setQueryTimeout(Seconds) is issued and the statement exceeds the timeout, it will atte...
Read more

Locks held on Oracle for hours after sessions abnormally terminated by node failure

If a session holds db locks and is abnormally terminated (no fin/ack), the locks will persist until the db session is closed, typically around 2 hours and 12 minutes with default network tcp_keepalive settings. Abnormally terminated does not include CTRL C or kill -9. The ojdbc6 driver apparently has a shutdown hook thread that closes the connections in graceful shutdowns. The OS apparently closes the connections when processes are killed. Abnormally terminated might include power failure, firewall failures, Out Of Memory, OS/kernel crash, network connection failure, JBoss or other server node failure, etc. This was easily reproduced by creating a  process that connected to the db and updated a record but did not commit. While the first process was waiting, a second process with a contending update was started which blocked on the first update. The first clients network cable was then disconnected. The second clients transaction waited for approximately 2 hours until the databa...
Read more