Head in the Web™

I ran across this today. It took me a long time to find the right solution, so I thought I would post it here.

The error message I was getting was:

Failed to invoke operation load on MBean oracle.as.management.
mbeans.register:type=component,name=ohs1,instance=asinst_1,
Location=AdminServer
Load failed with IOException

The logs further showed the following:

OHS-2079 nzos handshake error, nzos_Handshake returned
28860(server obscured.server.com:9999, client 111.127.66.159)
OHS-2171 NZ Library Error: SSL fatal alert

The fix I found towards the end of a thread at https://forums.oracle.com/forums/thread.jspa?threadID=1054032:

ORACLE_HOME\opmn\bin\upgradenonj2eeapp.bat
-oracleInstance Instance_Home_Location
-adminHost WebLogic_Server_Host_Name
-adminPort administration_server_port_number
-adminUsername administration_server_user

Change it to .sh for Linux/Unix. I found I only had to run this on the first server in the farm rather than all of them as one might expect. Your mileage may vary.

I ran into an issue where I couldn’t figure out why I was getting the following error when running a WebLogic Server 10.3.2 upgrade in silent mode pert the documentation:

C:\temp\ten_two_domain1>java weblogic.Upgrade -mode silent
-type domain Calling Wizard framework for upgrade: args2:
[-mode=silent,-file=wcf/plugin_silent_wizard.xml,
-p:plugin:plugin.silent.response.file=weblog
ic-upgrade-domain-responses.xml,
-log=stdout,-p:plugin:plugin.executionPlan.file=weblogic/
upgrade/domain/execplan.xml,-p:plugin:I18N_PLUGI N=weblogic/
upgrade/i18n_upgrade,
-p:plugin:TARGET_VERSION=10.3.2.0] 2011-05-03 12:12:25,926
ERROR [inputAdapter_silent] com.bea.plateng.wizard.
WizardController - Uncaught Exception java.lang.
ArrayIndexOutOfBoundsException: 0 at weblogic.upgrade.domain.
directoryselection.SelectWebLogicVersionPlugIn.
validateInputAdapter(SelectWebLogicVersionPlugIn.java:50) at
com.bea.plateng.wizard.plugin.silent.tasks.
InputAdapterSilentTask.execute(InputAdapterSilentTask.java:178)
at com.bea.plateng.wizard.silent.tasks.AbstractSilentTask.
run(AbstractSilentTask.java:28) at java.lang.Thread.run(
Thread.java:619) A fatal error has occurred. This application
will terminate.

I thought I had followed the example from http://docs.oracle.com/cd/E15523_01/web.1111/e13754/xml_ref.htm#g1066162 perfectly:

<?xml version="1.0" encoding="UTF-8"?>

<plugin-silent-responses>
</plugin-silent-responses>

<!-- SAMPLE BELOW -->
<!--
<plugin-silent-responses>
  <group name="DomainSelectionGroup">
    <plugin name="SelectWebLogicVersionPlugIn">
      <input-adapter name="ChoiceIA">
        <bind-property name="selectedChoiceIds">
          <value>__WEBLOGIC_VERSION__</value>
        </bind-property>
      </input-adapter>
    </plugin>
    <plugin name="DomainDirectorySelectionPlugIn">
      <input-adapter name="IA">
        <bind-property name="selectedFile">
          <value>__DOMAIN_DIR__</value>
        </bind-property>
      </input-adapter>
    </plugin>
  </group>
  <group name="PostDirSelectionGroup">
    <plugin name="AdminServerSelectionPlugIn">
      <input-adapter name="IA">
        <bind-property name="selectedChoiceIds">
          <value>__ADMIN_SERVER_NAME__</value>
        </bind-property>
      </input-adapter>
    </plugin>
    <plugin name="NodeManagerCredentialsPlugIn">
      <input-adapter name="UsernameIA">
        <bind-property name="value">
          <value>__NODE_MANAGER_USERNAME__</value>
        </bind-property>
      </input-adapter>
      <input-adapter name="PasswordIA">
        <bind-property name="value">
          <value>__NODE_MANAGER_PASSWORD__</value>
        </bind-property>
      </input-adapter>
      <input-adapter name="PasswordConfirmIA">
        <bind-property name="value">
          <value>__NODE_MANAGER_PASSWORD__</value>
        </bind-property>
      </input-adapter>
    </plugin>
    <plugin name="OptionalGroupsSelectionPlugIn">
      <input-adapter name="IA">
        <bind-property name="selectedChoiceIds">
          <value>__OPTIONAL_ACTION_1__</value>
          <value> . . . </value>
        </bind-property>
      </input-adapter>
    </plugin>
  </group>
  <group name="PostDirSelectionPost81Group">
    <plugin name="AdminServerSelectionPlugIn">
      <input-adapter name="IA">
        <bind-property name="selectedChoiceIds">
          <value>__ADMIN_SERVER_NAME__</value>
        </bind-property>
      </input-adapter>
    </plugin>
    <plugin name="OptionalGroupsSelectionPlugIn">
      <input-adapter name="IA">
        <bind-property name="selectedChoiceIds">
          <value>__OPTIONAL_ACTION_1__</value>
          <value> . . . </value>
        </bind-property>
      </input-adapter>
    </plugin>
  </group>
  <group name="DomainBackupGroup">
    <plugin name="DomainDirectoryBackupPlugIn">
      <input-adapter name="FileSelectionIA">
        <bind-property name="selectedFileNames">
          <value>__BACKUP_DIR__</value>
        </bind-property>
      </input-adapter>
      <input-adapter name="TextIA">
        <bind-property name="value">
          <value>___BACKUP_FILE_NAME__</value>
        </bind-property>
      </input-adapter>
    </plugin>
  </group>
</plugin-silent-responses>

-->

But, as it turns out, where I wanted to be literal on the  SelectWebLogicVersionPlugIn value with 10.2, the correct format is to be a bit broader, with:

    <plugin name="SelectWebLogicVersionPlugIn">
      <input-adapter name="ChoiceIA">
        <bind-property name="selectedChoiceIds">
          <value>9.0 or higher</value>
        </bind-property>
      </input-adapter>
    </plugin>

Note the value 9.0 or higher

I hope this saves some folks the time I spent figuring it out

Here’s a bit of fun with WebLogic JDBC configurations.  I ran into this issue after reading that p13nDataSource and  cgDataSource-NonXA should not be configured as multi-source. There were  some issues changing them to use the basic JDBC connection string and  when rolling back to the bad configuration the server went “Boom”.  Since one purpose behind this blog is to share lessons learned, I just had to post this.

If you write your descriptors manually (as opposed to generating them using the WLS console) and put a comma-separated list of JNDI addresses like this:

<jdbc-data-source-params>
  <jndi-name>weblogic.jdbc.jts.commercePool,
	contentDataSource,
	contentVersioningDataSource,
	portalFrameworkPool</jndi-name>
  <algorithm-type>Load-Balancing</algorithm-type>
  <data-source-list>portalDataSource-rac0,
	portalDataSource-rac1</data-source-list>
  <failover-request-if-busy>false</failover-request-if-busy>
</jdbc-data-source-params>

so long as the first address resolves, it will still work. Sort of.  If you call this connection to do an update, only one node of the RAC instance is updated. Other wonderful side-effects include the server refusing to start sometimes.

The proper way to list the JNDI sources is one per node, like this:

<jdbc-data-source-params>
	<jndi-name>weblogic.jdbc.jts.commercePool</jndi-name>
	<jndi-name>contentDataSource</jndi-name>
	<jndi-name>contentVersioningDataSource</jndi-name>
	<jndi-name>portalFrameworkPool</jndi-name>
	<algorithm-type>Load-Balancing</algorithm-type>
	<data-source-list>portalDataSource-rac0,
			portalDataSource-rac1,
			portalDataSource-rac2
	</data-source-list>
	<failover-request-if-busy>false</failover-request-if-busy>
</jdbc-data-source-params>

(Props to Sandeep Seshan for locating the root cause)

I ran across a thread on LinkedIn that pointed me to an excellent article on IT on Guard dealing with a slow WLS start up issue.