SSL Handshake Failure

[dakaix]

Hello,

I am trying to setup SSL access to our UnrealIRCd instance. The server is running CentOS 5.4 x86_64 and OpenSSL & OpenSSL-devel 0.9.8e-12.el5_4.1 installed. We have UnrealIRCd 3.2.8.1 setup on a different port (6697), with a certificate located with unrealircd.conf and generated by ./Config.

./Config Options:

Code: Select all

Do you want to enable the server anti-spoof protection?
[Yes] -> 

What directory are all the server configuration files in?
[/home/unr3al/UnrealIRCd] -> 

What is the path to the ircd binary including the name of the binary?
[/home/unr3al/UnrealIRCd/src/ircd] -> 

Would you like to compile as a hub or as a leaf?
Type Hub to select hub and Leaf to select leaf.
[Hub] -> 

What is the hostname of the server running your IRCd?
[noflood.irc.{DOMAIN REMOVED}.com] -> 

What should the default permissions for your configuration files be? (Set this to 0 to disable)
It is strongly recommended that you use 0600 to prevent unwanted reading of the file
[0600] -> 

Do you want to support SSL (Secure Sockets Layer) connections?
[Yes] -> 

If you know the path to OpenSSL on your system, enter it here. If not
leave this blank
[] -> 

Do you want to enable IPv6 support?
[No] -> 

Do you want to enable ziplinks support?
[Yes] -> 

If you know the path to zlib on your system, enter it here. If not
leave this blank
[] -> 

Do you want to enable remote includes?
[No] -> 

Do you want to enable prefixes for chanadmin and chanowner?
This will give +a the & prefix and ~ for +q (just like +o is @)
Supported by the major clients (mIRC, xchat, epic, eggdrop, Klient,
PJIRC, irssi, CGI:IRC, etc.)
This feature should be enabled/disabled network-wide.
[Yes] -> 

What listen() backlog value do you wish to use?  Some older servers
have problems with more than 5, others work fine with many more.
[5] -> 

How far back do you want to keep the nickname history?
[2000] -> 

What is the maximum sendq length you wish to have?
[3000000] -> 

How many buffer pools would you like?
This number will be multiplied by MAXSENDQLENGTH.
[18] -> 


How many file descriptors (or sockets) can the IRCd use?
[1024] -> 

Would you like any more parameters to configure?
Write them here:
[]-> 

Config Snippet:

Code: Select all

listen         {IP REMOVED, * makes no difference}:6697
{
	options
	{
		ssl;
		//clientsonly;
	};
};

log "ircd.log" {
	/* Delete the log file and start a new one when it reaches 2MB, leave this out to always use the 
	   same log */
	maxsize 20MB;
	flags {
		oper;
		kline;
		connects;
		server-connects;
		kills;
		errors;
		sadmin-commands;
		chg-commands;
		oper-override;
		spamfilter;
	};
};

With all the log options set, nothing appears in ircd.log when anyone connects (either SSL or non-SSL), only when there's an error such as the IP in use. I have recompiled it with DEBUGMODE on, this doesn't make any difference and I still don't get any debug information on connections to the server.

Ports are definitely open, since if I remove the SSL option from the listen block for that port, connections succeed. A tcpdump from the server, shows that the TCP handshake is completed, one response is sent and then the TCP connection is closed.

Test Results:

Code: Select all

OpenSSL Client:
openssl s_client -showcerts -connect {IP REMOVED}:6697
CONNECTED(00000003)
32221:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:

mIRC Output:
* Connecting to {IP REMOVED} (+6697)
-
* Unable to connect to server (Host disconnected)

I had a search around the FAQs and these forums, but couldn't find any similar problems, so any assistance would be greatly appreciated!

Many Thanks,

Dakaix.

[warg]

I suspect this to be an issue w/ your system's openssl.

Try compiling a local openssl in your $HOME for Unreal:

Code: Select all

wget http://openssl.org/source/openssl-1.0.0a.tar.gz
tar xf openssl-1.0.0a.tar.gz
cd openssl-1.0.0a
./config --prefix=$HOME/openssl no-threads zlib-dynamic
make && make install

Then compile a clean Unreal:

Code: Select all

wget http://www.unrealircd.com/downloads/Unreal3.2.8.1.tar.gz
tar xf Unreal3.2.8.1.tar.gz
cd Unreal3.2
export CFLAGS="-I$HOME/openssl/include"
export LDFLAGS="-L$HOME/openssl/lib -L$HOME/openssl/lib64"
./Config

If you use the unreal script to start unreal, you may need to add...

Code: Select all

export LD_LIBRARY_PATH=$HOME/openssl/lib:$HOME/openssl/lib64

...to the top of it just below the #!/bin/sh

If you do not use the unreal script to start unreal, and instead execute the binary directly, you may need to do...

Code: Select all

LD_LIBRARY_PATH=$HOME/openssl/lib:$HOME/openssl/lib64 ./ircd

...or you can export it in your ~/.bash_profile (or your distributions equiv.)...

Code: Select all

echo "export LD_LIBRARY_PATH=$HOME/openssl/lib:$HOME/openssl/lib64" >> $HOME/.bash_profile
source $HOME/.bash_profile

...to automatically export it when you login.

Give this a try, goodluck! =)