SSL - java.net.SocketException: Connection reset

OpenAM 10.0 EA has recently been released (you are download from here). So, as usual, I'll have a copy running in our labs.

I have Tomcat 7.0.26 installed and enabled SSL. Fairly straightforward to enable SSL on Tomcat with APR (Read here). I have also ensure the CA certificate is imported into the Java keystone which Tomcat was running on.

However, when I run the OpenAM configurator, I kept getting "Connection reset" error.

[openam@IDP config]$ java -jar configurator.jar -f idp.config

java.net.SocketException: Connection reset

        at java.net.SocketInputStream.read(SocketInputStream.java:168)

        at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)

        at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)

Very strange. I would expect a "PKIX path building failed" error, which is very common when an invalid certificate or no certificate was imported into the keystore.

I even went to the extend of firing up SSLPoke to identify what has went wrong. No luck! It threw me the same "Connection reset" error.

What's next is set the following JVM-option:
"-Djavax.net.debug=SSL,handshake,trustmanager"

$ java -Djavax.net.debug=ssl,handshake -cp . SSLPoke idp.azlabs.sg 8080

Bingo!

:
:
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
:
:
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, handling exception: java.net.SocketException: Connection reset
main, SEND TLSv1 ALERT:  fatal, description = unexpected_message
main, WRITE: TLSv1 Alert, length = 2
main, Exception sending alert: java.net.SocketException: Broken pipe
main, called closeSocket()
java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:168)
        at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
        at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:830)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:11                                                                                                                                          70)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:637)
        at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:89)
        at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:103)
        at SSLPoke.main(SSLPoke.java:31)

Now I know what's wrong.

I shouldn't have cut-n-paste from Tomcat 7 documentation without thinking.

port="8443" maxThreads="200"
scheme="https" secure="true" SSLEnabled="true"
SSLCertificateFile="/usr/local/ssl/server.crt"
SSLCertificateKeyFile="/usr/local/ssl/server.pem"
clientAuth="optional" SSLProtocol="TLSv1"/>

TLSv1 !! Remove it resolve the issue.

.

OpenIDM - The Next Generation Identity Management Solution

Gael Allioux from ForgeRock visited us earlier this week during his APAC trip. We had a great OpenIDM workshop conducted here in Singapore.

I must say I'm impressed with the progress of OpenIDM v2.0. Yes, it still not mature and lack some features at the moment. But the Synchronization Engine is fully complete and I'm equally impressed OpenIDM v2.0 has already been deployed to production in some key customers' sites!

From an architecture perspective, OpenIDM v2.0 is clean.

That's it! Clean and lightweight. 

.

Policy Agent 2.2 in CDSSO mode connecting to OpenAM Issue

One of my customers has a legacy Policy Agent 2.2 configured in CDSSO mode. It needs to connect to the newly installed OpenAM 9.5.3 server.

No luck... It was not a breeze porting over... We kept getting the following error:
WARNING: LdapSPValidator.validateAndGetRestriction: Invalid agent ID: http://stqa.as.com.sg:80/

See here.


Finally after much research, I found a link from Oracle. Not exactly the same deployment, but similar sympton.

The Web Proxy Agent 2.2-01 in Cross Domain Single Sign-on mode does not work with Access Manager 7.1 Patch . The agentRootURL requirement was added as a security measure to ensure that CDC is handing off ssotoken cookie to trusted agents running at known URLs.


Workaround

• Go to Access Control > / (Top Level Realm) > Agents > 2.2 Agents > UrlAccessAgent
• Key in agentRootURL=http://stqa.as.com.sg:80/ to Agent Key Value(s).

Jackpot!


.

OpenAM: #403x error

Sometimes, when a Policy Agent is configured and this very not-so-helpful error #403x appears on the browser, one needs to investigate further...

Usually, I systematically scan through the following log files:
1. Agent debug log files (at node where PA is installed)
2. OpenAM debug log files (usually Authentication will reveal what's wrong)

In this particular case, the Policy Agent was not defined properly in OpenAM.

amCDC:01/04/2012 12:07:36:371 PM SGT: Thread[http-2020-4,5,main] WARNING: LdapSPValidator.validateAndGetRestriction: Invalid agent ID: http://stqa.as.com.sg:80/ amCDC:01/04/2012 12:07:36:371 PM SGT: Thread[http-2020-4,5,main] ERROR: Invalid Agent: Could not get agent for the realm java.lang.Exception: Invalid Agent: Not configured in directory at com.iplanet.services.cdc.LdapSPValidator.validateAndGetRestriction(LdapSPValidator.java:160) at com.iplanet.services.cdc.CDCServlet.redirectWithAuthNResponse(CDCServlet.java:394) at com.iplanet.services.cdc.CDCServlet.doGetPost(CDCServlet.java:355) at com.iplanet.services.cdc.CDCServlet.doGet(CDCServlet.java:270) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:91) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:864) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1665) at java.lang.Thread.run(Thread.java:662)

amCDC:01/04/2012 12:07:36:371 PM SGT: Thread[http-2020-4,5,main] ERROR: CDCServlet.doGetPost java.lang.Exception: Invalid Agent: Could not get agent for the realm at com.iplanet.services.cdc.LdapSPValidator.validateAndGetRestriction(LdapSPValidator.java:229) at com.iplanet.services.cdc.CDCServlet.redirectWithAuthNResponse(CDCServlet.java:394) at com.iplanet.services.cdc.CDCServlet.doGetPost(CDCServlet.java:355) at com.iplanet.services.cdc.CDCServlet.doGet(CDCServlet.java:270) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:91) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:864) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1665) at java.lang.Thread.run(Thread.java:662)


.

OpenDJ Replication Server error during handshake phase

I have a pair of OpenDJ servers running in MMR mode. Customer has a sudden requirement to change IP addresses.

Simple request.. So I went ahead to modify the /etc/hosts. As simple as that.

No. The following error is observed in OpenDJ errors logs on both nodes:

[21/Dec/2011:12:46:32 +0800] category=SYNC severity=SEVERE_ERROR msgID=14942387 msg=Replication server 30809 was attempting to connect to replication server a125.az.com/172.8.8.125:8888 but has disconnected in handshake phase


[21/Dec/2011:12:46:32 +0800] category=SYNC severity=SEVERE_ERROR msgID=14942263 msg=In Replication server Replication Server 8888 30809: replication servers 200.2.2.125:8888 and 172.8.8.125:8888 have the same ServerId : 20398

:
:

[21/Dec/2011:12:46:36 +0800] category=SYNC severity=SEVERE_ERROR msgID=14942316 msg=Unable to send monitor data request for domain "cn=admin data" to replication server RS(30809) due to the following error: Socket closed

I resolved this by restarting both nodes.


.

OpenAM Fedlet

A customer asked me what's a OpenAM Fedlet and its usage. 

There isn't a lot of detailed document on OpenAM Fedlet. But this article from Oracle is great!

ForgeRock's documentation only has a section on Using Fedlets in Java Web Applications.

In layman term, this is my interpretation of Fedlet:

Basically, big organizations with budget will be using OpenAM Federation service.

e.g. One organization will install OpenAM to act as IdP (Identity Provider), while the rest of the organizations will enable their applications to be SAMLv2 -ready. These applications will then act as SP (Service Provider).

However, this takes time and effort and money.

Smaller organizations will definitely not be able to overhaul their applications to be SAMLv2-ready, as it is not cost-effective. So the way to go is to just deploy Fedlets (generated from OpenAM servers).

The Fedlet will act like a bridge between the OpenAM server (acting as IdP) and the applications.


It's simple and neat.

.

Overriding OpenAM classes

I was trying to change some behavior in OpenAM core components and I find the easiest way to do it is:

• Modify the OpenAM source code
• Compile the Java class
• Deploy the compiled class in ../WEB-INF/classes
• Restart OpenAM

The application container will let /WEB-INF/classes take priority over the class in the jar file residing in the /WEB-INF/lib directory. Nice!

In fact, it's not recommended to put back the modified class into the original jar file. I find that ugly in practice.

.

APR based Apache Tomcat Native Library

I want better performance running OpenAM on Tomcat application server, thus I spent the effort to configure APR (Apache Portable Runtime) for Tomcat.

As usual (this is not my 1st time), I always encounter this error whenever I start Tomcat after configuration:

Nov 30, 2011 4:01:23 PM org.apache.catalina.core.AprLifecycleListener init



INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /am/bin/jdk1.6.0_27/jre/lib/amd64/server:/am/bin/jdk1.6.0_27/jre/lib/amd64:/am/bin/jdk1.6.0_27/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

 
This is a very silly mistake for not following the instruction carefully.
 
The resolution is to add the following to catalina.sh:

[am@testMachine bin]$ vi catalina.sh


JAVA_OPTS="-server -Xms2048M -Xmx2048M -Djava.library.path=/usr/local/apr/lib"
 
 
.