Posts

Showing posts from May, 2009

HAProxy syslogd logging alternative

HAProxy uses syslogd which was not trivial to configure. I wrote a UDP logger that takes a bind address and port and logs all UDP packets via log4j DailyRollingFileAppender.

startLogger.sh

#!/bin/bash

java -cp log4j-1.2.15.jar:udp_logger-0.0.1-SNAPSHOT.jar UdpLogger localhost 11088 &
echo $! > haproxy.udp_logger.pid


shutdownLogger.sh

#!/bin/bash

if [ -e haproxy.udp_logger.pid ] ; then
vpid=`cat haproxy.udp_logger.pid`
else
echo "no pid file, exiting"
exit
fi

if [ -n "$vpid" ] ; then
exists=`ps -ef | grep $vpid | grep -v "grep"`
echo $exists
else
echo "no pid found, exiting"
fi

if [ -n "$exists" ] ; then
echo "kill $vpid"
kill $vpid
while ps -p $vpid > /dev/null; do sleep 1; done
else
echo "pid process not found to kill, exiting"
fi


restartLogger.sh

#!/bin/bash

./shutdownLogger.sh
./startLogger.sh


On Solaris in /etc/rc2.d install startup / shutdown links


ln -s /usr/local/util/udp_logger/startLogger.sh S104startUdpLogg…

HostMonster / BlueHost and Java

Successfully installed Java, Apache Tomcat, JBoss 4.2.2, 4.2.3, 5, and Glassfish on HostMonster. All ran in minimal configuration. JBoss would not run with default configuration receiving `too many open files` because `ulimit -n` is 100. JBoss needs ~350 file descriptors to run. The admins at HostMonster were adamant that they would not increase the `ulimit -n` value. Told to go VPS or dedicated server. Bummer!

JBoss Tattletale

SVG

Bookmarklets

Interesting images

Apache JMeter load testing

JMeter was easy to setup and use.

It was very easy to launch non-gui remote instances and control and monitor them from a local GUI client. On the remote server, just launch the server from the included server script in the bin directory with no options.

The client ships the test plan to the remote instances.

To create a test plan:
Under Test Plan node:

Add : Thread Group: specify concurrent users, ramp-up, scheduling, and looping

Under Thread Group:
Add : Config Element : HTTP Request Defaults: specify host
Add : Config Element : HTTP Cookie Manager: specify cookie type and clearing every session if desired
Add : Sampler : HTTP Request: specify resource path
Add : Listener : Graph Results: specify results file and what data to display on the graph, dynamically configurable at run time

Click on Run : Start or Remote Start

Hostmonster dedicated IP and ports

Hostmonster documentations says:
"These ports will automatically be opened, by default, after purchasing a dedicated IP address."

After wasting hours, I discovered they changed the policy a week ago and now opening up ports must be requested by a ticket.

After requesting the ports to be opened, I have been told that my account has to be migrated to a Dedicate IP server for no additional cost. The migration may cause my sites to be down from 4 - 24 hours.

It was also requested that I backup my account before the migration.

cpanel full backup command:

/bin/gtar -c -f - -X /home1/<id>/cpbackup-exclude.conf -X /etc/cpbackup-exclude.conf --exclude backup-[!_]*_[!-]*-[!-]*-[!_]*_<id>* .

Initially started a full backup without deleting two versions of Java JDK, two versions of Apache Tomcat and two versions of JBoss. Killing the backup from cpanel Process Manager did not kill the backup since it was running as root. Scheduling another backup after cleaning up and while the pre…

Apache Tomcat 5 book

Subversion

Hostmonster Performance Test Results

Created a very simple Java math execution loop with Thread.yield() calls after every calculation so it wouldn't hog all the CPU and hurt other user's performance. Executed with java1.6.0_13 with defaults. The program was executed only a few times (< 12) and never more than a few seconds (< 5). The program spawned 8 worker threads with no contention so there was one for each CPU.

Execution times were observed as low as ~900 nS on Hostmonster vs. ~1500 nS on my laptop running Ubuntu 8.10, see additional specs and cpu info below.

Monitoring top, and running the test program very briefly (< 5 sec), the CPU% peaked at 589% - WOW!

Of course as a hosting service, and considering Java and GC, the performance vs. memory utilization, I/O performance, and availability are more important.

Hostmonster system:

From set:
x86_64-redhat-linux-gnu

From top:
Mem: 16473136k total, 15851752k used,
621384k free, 653060k buffers
Swap: 8193108k total, 235480k used,
7957628k f…

JBoss

JavaScript Disable Backspace Internet Explorer

Cookies

Create and read cookies with JavaScript

At least:
30 cookies per domain
4 KB per cookie
300 cookies total

DNS cache, TTL, ARP cache

To list TTL on linux:
dig <FQ Hostname>

To list TTL on Windows:
nslookup
set debug
<FQ Hostname>

To display remaining TTL on Windows:
ipconfig /displaydns

To flush dns on windows:
ipconfig /flushdns
Windows also caches failed dns resolution for a brief period. Flushing the DNS clears these also, it can be fixed by following this post.

Linux by default doesn't have DNS cache unless nscd was installed. If nscd is installed restart it with:
sudo /etc/init.d/nscd restart

Firefox does have a DNS cache:
To clear it go to off-line mode and back to on-line mode, select and deselect: File : Work Offline.
To diable it get the DNS cache plugin or enter 'about:config' in the address bar and add a new Integer 'network.dnsCacheExpiration' with value of '0'.

Microsoft Internet Explorer (IE) also has DNS cache:
Older versions cached for 24 hours and newer versions cache for 30 minutes, see the tech article on manipulating.

To display/list ARP cache on windows, linux, and unix (h…

Enterprise Architect free image library

Download here

Import into a project, Tools : Import Reference Data

After importing, select a node, type Ctrl+Shift+W and select alternate image from library.

Solaris VIP behavior - Gratuitous ARP

Tested VIP fail-over between two Solaris 10 x86 servers.

If multiple systems with conflicting VIPs bring up the interfaces, the first interface up responds to the service.

If the first interface fails or is downed, the other interface will not take over unless is is again signaled up and sends a Gratuitous ARP, see blog entry.


How is gratuitous arp used on Windows

Java JVM options

Sun HotSpot: Garbage Collection Tuning
IBM JVM: Download, incredibly-fast GC
Oracle BEA JRocket - claims "industry's highest performing" JVM

Sub-second Full GC on 100GB heap, 50% nursery and 64 cpus

IBM JVM with -Xgcpolicy:gencon, the newest of all their GC policies

IBM JVM downloads

Quote from the Blog post:

"And last I tried gencon (-Xgcpolicy:gencon), the newest of all their policies. Gencon is suppose to be used on "transactional systems", systems that create a lot of short lived objects. Isn't that what most java applications do? When I started it up, it seemed to be faster, and our load test confirmed that, 30% more throughput. But the amazing thing was that the full GCs were fast, really fast, in the 200-400ms range, for a 50G heap! But wait, it wasn't using most of the heap, and it was GCing all the time. Back to the verbose:gc log, AHA! The nursery was too small, it was about 10% of the heap, and because our application is almost all short lived objects I decided to increase the size of the nursery. I gave it 50% of the heap, and the time between GCs slowed down, but the time to perform the GC was still in the 350ms range. Awesome!

I finally s…

Java RMI hole punching

Config VIP on Solaris 10

list existing interfaces
ifconfig -a

select an interface to add a VIP, i.e., hme0
select the next number that isn't used, i hme0:#


Add virtual interface
ifconfig hme0:1 plumb

To remove the interface:
ifconfig hme0:1 unplumb

configure virtual interface ip
you need an ip and mask assigned in advance
ifconfig hme0:1 10.60.54.238 \
netmask 255.255.255.0 \
broadcast 10.60.54.255

check that it is available
ifconfig -a

At this point you can bind services to it, i.e., backups, but it isn't up,

Up the interface when you need it to respond, down other server VIP interfaces if this server is taking over responding to the service.
ifconfig hme0:1 up

To make the service come up automatically, add the ip and hostname to the /etc/hosts file and create a file /etc/hostname.hme0:1 with the hostname in it.

Configure, and up in one command:
ifconfig hme0:1 10.60.54.238 \
netmask 255.255.255.0 \
broadcast 10.60.54.255 up

See this post

Installing Failover VIP on Solaris 10 x86

Since I don't have a working alternative on Solaris:

To avoid losing service access due to the loss of a single load balancer I’m writing a simple program that will run on each of the primary and secondary HAProxy servers. Each will monitor the alternate with a TCP connection. If it sees the alternate VIP disappear it will attempt to reestablish the connection to eliminate transient events. If still down and it can access the gateway it will up the VIP on its own OS. It will continue to monitor the alternate VIP. If it sees the alternate VIP available again, it will enter a small random wait. If the alternate stays up it will down the VIP on its own OS. The later is to avoid duplicate IPs and flapping.

netstart -rn | grep default


Use linux and keepalived, it's simpler and it works.

Tried the following, but make failed:

Download and install tk

gunzip tk-8.5.3-sol10-x86-local.gz
pkgadd -d tk-8.5.3-sol10-x86-local

Download and install tcl

gunzip tcl-8.5.3-sol10-x86-local.gz
pkgadd -d tcl-8…

HTML JavaScript

NetBeans 6.5 - JavaScript Debugger: FireFox and Internet Explorer

JavaScript libraries

Prototype

JavaScript framework comparison

Various ways to play sound

Sound
bgsound, embed

W3Schools

HTML
JavaScript

Shake example

Table CSS

Redirect in Firefox and Internet Explorer:
Works in Firefox and Not IE:
<META HTTP-EQUIV="refresh" content="0 ; /foo/index.html">
Works in both:
<META HTTP-EQUIV="refresh" content="1;URL=/foo/index.html">

Safe fonts on PC and Mac:
arial, arial black, comic sans ms, courier, courier new, georgia,
helvetica, impact, palatino, times new roman, trebuchet ms, verdana
Include backups:
CSS:
{font-family: courier, arial}
HTML Tag:
<font face="helvetica, arial">text</font>
HTML font list

HOWTO build and install HAProxy on Solaris 10 x86

SunOS 5.10 i86pc i386

Download HAProxy source from here

gunzip haproxy-1.3.17.tar.gz
tar -xvf haproxy-1.3.17.tar

The readme file contains instructions on how to build it.

cd haproxy-1.3.17
less README

Download GNU make and gcc from sunfreeware.com - search the list on the right

Install GNU make

Installs to /usr/local/bin

gunzip make-3.81-sol10-x86-local.gz
pkgadd -d make-3.81-sol10-x86-local

Install gcc:

gunzip gcc-3.4.6-sol10-x86-local.gz
pkgadd -d gcc-3.4.6-sol10-x86-local

Make and install HAProxy:

Make sure that GNU make is the first make in the path or specify path
i.e., export /usr/local/bin:$PATH
Make sure gcc can be found in the path
i.e., export $PATH:/usr/sfw/bin

make TARGET=solaris CPU=generic

Create a user and group:

/etc/passwd

haproxy:x:114:126::/home/haproxy:/bin/false

/etc/group

haproxy:x:126:

Create an config file, see docs and examples directory
/etc/haproxy.cfg

Install and link to startup / shutdown script
Start last (relative to range of numbers)
ln -sf /usr/local/util/haproxy/haproxy.init …

Apache commons Virtual File System (VFS)

Doesn't support creating jar or zip files

Capability matrix

JBoss 4.2.x

The default app is located at:

jboss-4.2.2.GA/server//deploy/jboss-web.deployer/ROOT.war

To change the default landing page, change its WEB-INF/web.xml and/or index.html

Photoshop

IRC IM Bots

Sprint now flash

Java Swing Calendar Component

SwingRCP for EclipseRCP

NetBeans RCP

Java InetAddress isReachable ICMP (Ping)

It doesn't do ICMP on Windows, it opens TCP echo on port 7.

On linux/unix you have to set the suid bit on Java executables to allow it to use ICMP.


So that article didn't clearly explain the whole story.

When the executable bit is set if a user has permissions to run the program the executable runs at permissions of the owner. If like mine, your java was owned by account with non-root privileges, it still won't be able to access ICMP. You have to set suid bit after changing ownership as the latter clears the suid bit.

sudo chown -R root:root
sudo chmod 4755 /bin/java

Never did get this to work because once the suid bit was set, encountered the following error: libjli.so file not found. I believe this might have to do with suid bit not being set on LD_LIBRARY_PATH references and/or shared libs because it might present a security issue.

After discussing with associates realized going this route is probably a bad idea. This would cause all Java programs to run as root presenting a …

Java traceroute, tracert