If you learned something
today, please
1x

The Art of Stubbing

August 5th, 2014
No Gravatar

(Originally published as Stubbing For Fun, Profit, and Survival )

Anyone reading or watching the news these days know that IT project budgets will be tightening further than you may already be experiencing. While proponents of some languages and platforms claim to provide lower development costs, many others firmly believe that it is the habits of the developers who prefer those environments (more than the environment itself) that leads to rapid results. One of these habits is the proper use of method stubs. By proper, let’s define that as stubs that facilitate the development process first, and checking off “done” on a project plan second.

Stubbing Defined

Half the time someone asks me if I am the person who wrote a particular article it is a manager or analyst rather than a developer. Since the next person who asks you what you mean by “stubbing” may be using this article as their only reference, I thought I would add something that makes it easy to clarify this technique, and some fodder that will allow you to either argue or support my particular view on the topic…

Wikipedia defines a stub as “a piece of code used to stand in for some other programming functionality”. Sadly, the use of stubs in “general software development and testing” is thrown in as an after-thought. They give much more emphasis to its use in distributed computing, which is logical in the sense that the project failure rate is much higher if stubs aren’t used in distributed application development, and some techniques, such as SOAP-based web services require them as part of the API. In my mind, the only reason not to stub is when you can write the final code from end to end as a single developer in a single day. Any task more complex requires some type of stubbing, the specifics of which will vary between teams.

Benefits of Stubbing

The following are just a few of the advantages of developing methods stubs that return a meaningful value. I’ve left out some as obvious (despite the fact that I often say the obvious bears repeating in development) and others that are a bit inflammatory (having worked many long hours fixing the issues of not stubbing properly I find it hard to be objective).

The Ability to Demonstrate Progress on Demand

Many developers will only stub their methods if they are working in a large team. One value of stubbing methods even when working alone is that not only will the code compile, the code will run and return meaningful values. While this may appear as a minimal value to code quality and speed of development, it can be an enormous advantage to demonstrate to non-technical user (i.e., those that determine your budget) exactly how much progress you have made. While the code may be in the same stage of completion with or without meaningful return values, it is far more acceptable to clients and management to explain that the demonstration is not ready for production than to have to constantly excuse test cases that fail because of a null value.

Facilitating a Test First Development Approach

Test first development is a very useful Agile method. By writing test cases first that check for valid values and then implementing code that moves the test cases from “fail” to “pass”, you will know that your methods are written correctly at the earliest point possible in the development process. By running your test cases automatically at each build you can then pinpoint many bugs immediately after they are written rather once your code is integrated into the full project. Integrated code takes longer to debug for many reasons, one of which is that everyone assumes it is someone else’s code, thus costing the time of multiple developers to locate the issue rather than the one developer who wrote it.

Parallel Development

While it can be argued that it is possible to develop in parallel without providing stubs, it is rare that the eventual integration of layers developed in parallel goes smoothly if stubs with meaningful return values are not used to test the contracts between teams early and often.

Weekends Off

Ok, this one won’t apply to everyone, but I bring it up because during the holiday weekend I wrote this article I was also debugging integration issues for the very reason that the points covered in this article were not followed by experienced developers. I understand that it was not their intent to cause more work, which is why the lessons here are important for both the newbies and seasoned professionals.

Misconceptions about Stubbed Methods

Stubbing is Only Necessary at the Model Layer

As mentioned under Benefits of Stubbing, one advantage to stubs is the ability to demonstrate on demand. The HTML of a view object that is blank or displays “null” is far less likely to help your budget case than seeing “John Doe”, which is obviously not real data but is meaningful data.

When working in parallel development it may be tempting to point at a project plan that has dependencies being delivered in perfect synchronization as consider stubbing a controller with null because the model should be ready before the view is integrated. In the event that everyone does not hit their milestone exactly (gee, how often does that happen?) development can fall behind waiting for someone else to catch up. Alternatively, if you get ahead of your own schedule you will not be able to progress until your dependencies are at the same plan stage as you. If all developers stub their code with meaningful value at every step, they are not dependant on others until the actual point of integration, and then only in the event of bugs, which are far fewer if stubbing was occurring at each layer and each stage.

Null is Not a Stub

There are two phrases that will raise the blood pressure of many team leads and development managers. The first is “it works on my machine”, and the second is “it compiled for me, so I checked it into source control”. It isn’t the mere fact that a developer’s machine will not be the one used in production that is the issue here (though, it is one of the obvious things that still need to be pointed out often), it is that these phrases are only used when the code in question has broken the application as a whole. The most common cause of integration builds failing is a null value that should not be null. It may even (and probably will) compile within the integrated application. The failure point is when the application is run, a point that is far more costly and the a compilation failure because it can impact other developers (who have to trace their code to find the cause from someone else’s code), QA engineering teams (who have to reschedule testing because the application isn’t available), and likelihood of future team projects if this disaster occurs during a client demonstration.

The effort involved in adding and later removing:

//FIXME: Remove when data available
if(value==null) {return “John Doe”;}
else return value;

should total about 15 minutes, versus the average 1.5 man hours it takes to detect the issues caused after deployment. And 15 minutes is on the long side, as it took me 25 seconds to write the above example and should take no more than 1 minute to remove it.

“Foo” Is Only Missing an “L”

Our example above returns a meaningful value. Even in the case where full JavaDoc comments are omitted, any developer seeing “John Doe” returned has a pretty good idea how they can use the method.  A return value of “foo” would give no such indication. Even if the method has full JavaDoc notations, the value “foo” is very unlikely to be of use to the client code. It may be helpful to consider that code examples that use “foo” generally use “bar” as the second value. “foo bar” is a homonym for FUBAR, which is an acronym for “F***** Up Beyond All Recovery”.

I recently worked on a project where stubs returned both null and “foo”. These values, coupled with generated JavaDoc comments rather than informative descriptions, led to delays in integration testing and the need to re-write 70% of the consumer code once integration began due to the assumed values not matching the actual values. A valid value returned by stubs for all methods would have prevented this and saved a substantial amount money (and one of my weekends).

Meaningful Stubs May End Up In Production

Some people believe that cleaning up stubbed code takes too much of their time, or that stubs can make their way into production by being missed. Looking at that second argument, it is only valid if comments are not used properly. All modern IDEs will provide a listing of comments that start with FIXME and TODO. Stubs that are commented with FIXME and cultivating the habit of reviewing these lists will always prevent stubs from making it to production and lead to cleaner code. Using these notations also allows for being able to quickly review code where stubs have been left and remove them when they are no longer needed.

Figure 1: Task View in Eclipse

Figure 1: Task View in Eclipse

 

Using a Stub Harness

If you don’t know how long it will be until you have data, consider writing a stub harness. A stub harness is a simple check to see if the class should use stubbed values before returning them. Whether to use the stubs or not can be set in a property file at a class or as a node in your deployment descriptor. This value can be set at a method level:

if(property.thisMethod.useStubs)

at the application level:

if(property.useStubs)

or both:

if(property.useStubs && property.thisMethod.useStubs)

The key to this approach is to make sure your build process will never allow these values to be set in either pre-production or production (or other applicable environments, depending on how your organization operates). Providing your processes include a regression tests in pre-prod, methods that are still stubbed will fail prior to going to production. It also allows you to switch off the stubs at any stage of development to run integration tests.

Conclusion

There are two inspirations for my article topics. Things that are cool, and things that are painful. In both cases my goal is to share the solution in the hopes of saving others the long, tedious discovery process that I and others have gone through. In the case of this particular topic, the tedious process isn’t the discovery but the realization that an approach that may seem at first glance to be time consuming is in fact time saving.

Fix for InvalidRegistrationException with a WLP Producer and WebCenter Portal Consumer

July 15th, 2014
No Gravatar

I have come across a case where I have deployed a WebCenter Portal EAR to a newly created domain that was copied in from another domain. When attempting to update the WLP producer registration with the new environment, I received a com.bea.wsrp.faults.InvalidRegistrationException.

I had done this sort of new domain configuration many times, with the only difference here being that the EAR I was deploying was copied from a running domain. While I don’t know the actual root cause of why it had a problem this time, I suspect that OEM is writing back to the EAR for some reason. Anyways…

After much head-banging, and successfully registering other domains to the same producer in a vain attempt to prove I was not insane, I re-read the error for the umpteenth time and finally realized what was causing the exception. Do you see it?:

<BEA-000000> <<Jun 16, 2014 3:08:28 PM PDT> <Error> <oracle.portlet.client.connection.wsrp.HTTPClientTransport> <WCS-40152> <A request to the producer URL "http://myhost5:8001/myAccount/producer/wsrp-2.0/markup" resulted in a status 500 response with fault string "The given registrationHandle [22001] is invalid.". The fault code given was "{urn:oasis:names:tc:wsrp:v2:types}InvalidRegistration". The producer generated a timestamp of 2014-06-16T15:08:28.584-07:00 and associated the following stack trace with the fault message: com.bea.wsrp.faults.InvalidRegistrationException: The given registrationHandle [22001] is invalid.

So, since it was trying to register for the first time with a handle that had not been created yet, it made sense that it would fail. The WLP schema is a bit too complex to allow just creating the one row, so instead I went back to JDeveloper, updated the registration handle there and built a new EAR. The new EAR had a new value for MDS so that when I tried for the last time to update the producer it decided to request a handle rather than insist it already had one.

Adding Problem 3 for “23.2.1 Portlet Displays a Portlet Consumer Error”

June 20th, 2014
No Gravatar

The Troubleshooting Oracle WebCenter Portlets documentation lists two problems under the Portlet Displays a Remote Portlet Communication Error section. Both seem legitimate causes and solutions, and I have run across both of them in the past. I hit a new one this week. Number three is running out of available threads in the Managed Server that is handling the request. This one was fun to find, because OOM errors that do not cause the server to fall over eventually clear up, and with an adequately sized cluster (the memory errors were caused by a dependent service failing) the issue is very random.

The symptom to look for is the OOM error in the log, which looks like this:

oracle.portlet.client.container.PortletException: java.lang.OutOfMemoryError: 
unable to create new native thread
	at oracle.portlet.client.tech.common.PortletFutureUtils.get(Portlet
FutureUtils.java:72)
	at oracle.portlet.client.tech.common.PortletSessionManager$InitSession
CompletionListener.conditionChange(PortletSessionManager.java:340)
	at oracle.portlet.client.service.pipeline.FutureConditionTask.done
(FutureConditionTask.java:168)
	at java.util.concurrent.FutureTask$Sync.innerSetException(Future
Task.java:273)
	at java.util.concurrent.FutureTask.setException(FutureTask.java:125)

Restarting the servers will give you temporary relief, though for the long-run you need to find the root cause. In this case, it was an undersized WebCenter Content cluster running on the same physical box sucking all the resources down. A larger cluster and load balancing corrected the issue for good (knock wood).

Leveling the Playing Field

May 11th, 2014
No Gravatar

(semi-continuation of A Tale of Two Migrations)

Before I get into the trials and tribulations of dealing with the inevitable custom implementations that are part of every WebLogic Portal application that is older than a year and has more than 5 pages (and there are newer, smaller apps with more customizations, trust me on that), it is best to look at some basics. In the path taken by this blog series we are using WSRP with the retiring WebLogic Portal (WLP) application as the producer and WebCenter Portal (WCP) as the migration target and WSRP consumer. It is definitely of value to review the standard documentation about how to configure this standards-based integration, which can be found for 10.3.6 at http://docs.oracle.com/cd/E41236_01/wlp.1036/e14235/chap_webcenter_interop.htm/ .

With all of the useful points provided in the documentation, the one point that I would like to add is to first start with upgrading your WebLogic Portal application and environment to the latest version. I am a strong proponent of not upgrading unless there is a clear value in doing so, and in this case there are several. One is that a migration from WLP to WCP is almost definitely going to require at least one (and most likely several) Oracle Support cases to complete. You will hear every time you open a case “please upgrade to the latest version”. I’m not a big fan of this approach, but since the goal of the project is to move everything to the WebCenter and all of the functionality will eventually need to be updated to the latest version as part of that process, it is worthwhile to get that out of the way now.

Another value of upgrading your WebLogic Portal before embarking on a staged migration is that with one of the release of WebLogic Portal 10.3.x from .2 or later (I didn’t know the specific version, am too lazy to go look it up, and don’t feel bad about that because it is moot at this point in time) and the corresponding WebCenter Portal release (see previous parenthetical comment) the WSRP engine was moved from the portal framework to the application server (WebLogic Server). This makes a great deal of sense since WebLogic Server has had a WSRP engine for quite some time (again, not doing the historical research for the curious). Since the underlying WSRP support is coming from the application server, it makes good sense to have the same engine on both sides of the tractor-pull known as WSRP. While it may not be entirely necessary between some versions (I strongly suspect –but have no proof- that this is the case between 10.3.5 and 10.3.6), it eliminates one point in the long chain that will eventually be examined in detail for the source of some unexpected result or other, so why not get it out of the way now?

Finally, WebLogic Server is still the gold standard of J2EE application servers and does continuously improve with each release (disclaimer: I have not worked with the 12c versions and am just hoping that they have not proven me wrong). So why not start off with the best foundation possible before dealing with the known risk of every migration which is that there will be unknown and unforeseen issues that arise.

In my case I pushed for the WebLogic Portal and Server upgrade to 10.3.6 because WebCenter 11.1.1.8 had just been released with its capability of including the MAR within the EAR, a major time and hassle saver. WebCenter 11.1.1.8 was release with no support for WebLogic Server 10.3.5, which explains the “coincidence” of WebLogic Portal 10.3.6 coming out two weeks before hand. Interestingly enough, development on WCP 11.1.1.8 is done with JDeveloper 11.1.1.7 which is only configured to work with WebLogic Server 10.3.5. Ah, if Fox Mulder were only a software engineer instead of an FBI agent ;)

A Tale of Two Migrations

May 11th, 2014
No Gravatar

My first portal migration project was not originally expected to be a migration; it was scoped and planned as an upgrade. The expectation of an upgrade was well-founded given the high-level scope of moving from one version to the next. Those involved with WebLogic Portal at the time know the move from 7.x to 8.x was in fact a migration rather than an upgrade. While there were some bridge APIs to allow the use of old code it was poorly suited to anything that was even slightly customized, which is to say every implementation in production. The project had been expected to take 4 – 6 weeks with one developer and actually took 10 weeks and 2 developers. It was the first of two migration/upgrades for the same customer and the lessons learned from the first move allowed the second move to be completed in 3 weeks with a single developer.

Fast-forward a decade and enterprises with WebLogic Portal are once again faced with a migration. This time we know up front that it will be a migration and IT managers along with their business customers are hesitant to move forward without a clear path to follow. While the last ten years have seen all J2EE-based portal products move to standard APIs for their foundation, they still have unique frameworks that serve as (from the vendors point-of-view) market differentiator and (from the customers perspective) a new form of vendor lock-in. In the case of WebLogic Portal, the vendor remains the same (albeit through acquisition) yet there is still no easy way to switch to the newer version, and a risk to staying with the old as it has been feature-frozen since 2010 and is slated for “sustaining-only support”.

The migration path from WebLogic Portal (WLP) to WebCenter Portal (WCP) is unclear for many reasons. Perhaps the most common and least-realized barrier to an “automated” migration is that WLP in his current form has 10 years of layered patches and deprecated-yet-still-in-production APIs supporting features that are a broad amalgam of proprietary solutions and customized integrations with standard APIs fueled by a demanding customer base for a (relatively) small vendor where WCP’s current form is currently a little over 3 years old and managed by a company that believes in setting the standards and letting the market follow.

Another source of great confusion is the mixed messages over time and between messengers as to whether a staged migration is practical or if it must be done in either a parallel phase out or big bang approach. To make that even more confusing, all of those options are equally the “best” approach, though the true determination of the correct approach is specific to a combination of the enterprise time-to-market needs, development skill sets, ability to maintain multiple environments and maturity of the enterprise architecture.

Given all these contributors to the fear, uncertainty and doubt surrounding a move off of a long-lived portal platform to a new, unfamiliar technology landscape I am currently in the process of acting as guide down this winding path for a company where all the various factors pointed mostly in the direction of a staged migration.

Like that first move to WLP 8.x, the approach from the 20,000 foot elevation looked very simple and straight-forward. Build the header, footer and navigation components in WebCenter and consume all of the legacy WebLogic Portal-produced portlets over WSRP. Where the WLP application had a backing file at the desktop-level that assembled user and account information at log-in, the backing file would be modified for use at the portlet level and the rest would be easy-peasy. This original understanding was entirely correct except the last phrase, i.e., no easy-peasy.

To be continued…

References for Historical Release Dates

http://en.wikipedia.org/wiki/Oracle_WebLogic_Server

http://en.wikipedia.org/wiki/Oracle_WebCenter

It’s the Little Tags that Count

May 4th, 2014
No Gravatar

In my on-going (and infrequent) effort to post tidbits about migrating from WebLogic Potal to WebCenter Portal I ran across this little reminder today:

WSRP does not support redirects. I’m not sure if that is per the WSRP 2.0 Specification or just the way Oracle goes about it, but for those of us working with WebCenter it doesn’t matter.

So, when doing a staged migration from WebLogic Portal to WebCenter where you have JSF portlets in WLP, in addition to making sure you update your pages to use UTF-8, check your faces-config.xml and look for navigation cases like this:

<navigation-case>
	<from-outcome>goBack</from-outcome>
	<to-view-id>/pages/SOLandingPage.jsp</to-view-id>
	<redirect/>
</navigation-case>

And remove the

<redirect/>

Failure to do so will cause all of your JSF WSRP page transitions to blow up in annoying ways.

Note that this discovery was by a team member who then passed it on to me.

Staking My Claim to BASE Acronym

May 3rd, 2014
No Gravatar

Basic
Architecture &
Software
Executable

Referring to a first release of an application’s foundation.

The First Step of a Journey that Began Five Years Ago

March 22nd, 2014
No Gravatar

Note: I will update this article with a link to the application once the customer has done their own announcements in accordance with their external communication policies and procedures.

In the Beginning there was BEA WebLogic Portal

In 2008, Oracle acquired BEA Systems. 19 days after the official merger, Oracle announced that premiere support for WebLogic Portal would end in 2014. The current policy document (latest can be found on http://www.oracle.com/us/support/lifetime-support/index.html) has moved this date out to 2018, though they have been sticking to the “no new feature release” policy since the 10.3.2 release in 2010. 10.3.2 was intended to be 11g, except it came out a year later than originally announced at Open World in 2008 and was released as a “dot” release of 10g despite the fact that it had several major enhancements and new features.

I had been hired by BEA in 2006 as a WebLogic Portal consultant due to my extensive experience with the product as a consultant for netNumina Solutions. In 2009, Oracle released WebCenter 11g and I attended the Masters Training two weeks prior to the GA date where I learned just how very different the two portal products are.

Which Way Do We Go?

I have been unable to find any officially published direction for WebLogic Portal customers who wish to migrate to WebCenter Portal, though I have had numerous conversations with engineers, architects, consultants and product managers about how to go about this. These discussions revealed three general approaches.

One approach is to simply re-build the portal in WebCenter. This is quite viable for very small portals and avoids the pitfall of other approaches, which is the need to maintain two architectures. It is not a very practical approach for medium to large portals as it is a great deal of effort and expense over a long period of time to just to provide the same functionality.

Both the second and third approaches are about transitions. On method is to create the new WebCenter portal and build all new features there and link over to the legacy WebLogic Portal for existing features. This is very quick and easy to deliver but difficult to maintain.

The third approach is staged migration. This approach creates the a new WebCenter portal that is the where users log in and interact, with the legacy functionality being exposed using WSRP. This solution allows for the immediate introduction of the WebCenter architecture and minimizes maintenance cost. By following a policy where any legacy portlet that requires modification be first moved over to WebCenter, Business and Technology stake holders can plan the complete retirement of the WebLogic Portal infrastructure as best suits the Enterprise as a whole.

Every Journey Starts with a Sprint

This month marks the deployment of the third solution to production for my current client. It is a medium-sized, high-complexity portal and it was brought from inception to production in six months using a mixed-Agile approach. It consists of 20 portlets produced by the legacy WebLogic Portal application and two legacy JSF portlets that were migrated in two days because they include file download functionality that made them easier to migrate than to dig through the documentation to fix as WSRP. The portal also includes managed content from WebCenter Content and a shared navigation structure with a legacy Struts 1.1 application.

The Enterprise Portal Architecture for the customer is to migrate all legacy functionality from WebLogic Portal to WebCenter Portal over the next year in staged releases that will also include the introduction of new features and functionality.

Stopping Sticky States in WebLogic Portal Beehive Producers in a WebCenter Portal Consumer

March 19th, 2014
No Gravatar

Recently had an issue where Beehive pageflows would retain their state when the user navigated to another page and returned. The business owners would not accept either the use of a button to start over or the ever-popular “Well, then don’t do that”. Understandable given it is a public site.

Without putting you through the tedium I went through to obtain the solution from Oracle Support (https://support.oracle.com/epmos/faces/DocumentDisplay?id=1419774.1 for those with an account, and thanks to Ming Kwok for providing it), turns out there is a setting accessible from the WebCenter Portal Page Definition that will cure this ill:

portletStateLifetime="ViewScope"

The above is an attribute of a portlet element. Note that in this project, all portlets are placed at design time (not fun with WebCenter, BTW). We did a bulk update of the entire application by doing a find and replace of

retainPortletHeader="false"

with

retainPortletHeader="false" portletStateLifetime="ViewScope"

Additional effort is required for JSF portlets, which a colleague is working on will be posted here when he completes his solution.

Stopping _afrLoop Looping Refresh

February 13th, 2014
No Gravatar

While building out a new environment I found that my WebCenter portal application URL would go into a long cycle of refreshing the page at login when more than one managed server in the cluster was running.

To save many of you the long process of research I went through, the key piece to my puzzle was that SiteMinder is used for perimeter authentication. The fix for this combination (WebCenter and SiteMinder) is to change the WebLogic cookie name from JSESSION to anything else and ALL_CAPS. In my case, there was already a custom cookie name, but it was lower case.

While I did not test each part of the fix, I’ll include the rest and let someone else do so. That is, setting the cookie name as the same in both the Apache plug in and the WebCenter application. For the Apache plug-in, the cookie name is managed by WLCookieName. In the application, is done by setting the cookie name in weblogic.xml:

<session-descriptor>
 <cookie-name>MYWEBCENTERCOOKIE</cookie-name> 
</session-descriptor>

Also, there were many posts suggesting to remove the Trinidad filter to address this. That is the wrong solution in most cases and will lead to new and annoying issues.

Official solution can be found at Oracle Support Document 1491984.1 (ADF Application Redirects Browser Indefinitely) can be found at: https://support.oracle.com/epmos/faces/DocumentDisplay?id=1491984.1. I found the all-capse naming convention on some post, and don’t know if it is necessary but do know that it is working now.