IT-eye

At the moment we have a SOA/ BPEL project in production for quite some time. One of the lessons we have learned will be the topic of this post: automaticly restarting BPEL processes.

In this case a process is divided into one root process and a number of subprocesses. The root process is started when a custormer applies for a specific test in a laboratorium. After the application a number of tests (subprocesses) are carried out. After 30 to 50 days the results are returned to te customer. At the moment their are 180.000 process instances, of which approximately 25-30.000 are still active.

We (and the customer) are very pleased with the results Oracle BPEL is bringing. But the implementation of a new product can give you some headache (Out of memory ) As a result of this bug BPEL instances went on abort. The tasks disappair in the worklist and users start calling. They want their tasks back….

At first we kicked off the root BPEL process from the BPEL Console and acquired the tasks and entered the recovered data. We restarted the BPEL proces manualy. This is nice break for 1 or 2 tasks, but becomes expencive when 100 tasks have to be restarted within a few hours. Here is where our creativity came in.

We have developed a webservice to collect all data to kickoff the root BPEL process and data the users entered. The payload for kicking off the root BPEL process is assembled by looking at the xml structure in the BPEL Console.

 SELECT '<bpaanvraagcanalyseprocessrequest xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">
       <aanvraagid xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">0</aanvraagid><oogstjaar xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">2006</oogstjaar><aanvraagnummer xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">'||
	   decode(p_action,'RESET',a.nummer_aanvraag,0)||'</aanvraagnummer> '||chr(10)||
      '<relatienummer xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">' ||rle.nummer||'</relatienummer> '||chr(10)||
      '<keuringsobjectid xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">'|| kot.ID||'</keuringsobjectid> '||chr(10)||
      '<aantalverpakkingen xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">'|| avrg.aantal_verpakkingen ||'</aantalverpakkingen>'||chr(10)||
      '<verpakkingsvorm xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">'|| lower(avrg.verpakkingsvorm) ||'</verpakkingsvorm> '||chr(10)||
      '<totaalgewicht xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">'|| avrg.gewicht_totaal ||'</totaalgewicht> '||chr(10)||
      '<start xmlns="http://xmlns.oracle.com/bpAanvraagCanalyse">'||decode(p_action,'RESET','HERSTEL;'||substr(bmr.nummer,2),'MONSTERONTVANGST')||'</start></bpaanvraagcanalyseprocessrequest>'
      INTO  l_payload
      FROM  avrg_aanvragen  avrg
      ,	   kot_keuringsobjecten kot
      ,	   kot_basismonsters bmr
      ,    rle.rle_relaties rle
      WHERE a.kot_id=b.id
      AND c.avrg_id=a.id
      AND c.nummer=upper(p_werkmonster)
      AND  a.rle_id=e.id
      AND a.id=r_avrg.avrg_id;

The payloads of the task is found in the BPEL dehydration store. We are storing the process instance id of the root process in our data store. With this id you can find all the child processes in cube_instance in the dehydration database. For every child we fetch the taskid form pc_taskand collect the last payload in the table pc_taskpayload. This payload consists the data the user entered while updating the task. Before returning all payloads to the GUI the old process and subprocesses are deleted.

The GUI will first kick off the BPEL process. Then get the first task and give the collected payload too the taskhandler. This is repeated until the BPEL process is restarted.

Unlike CruiseControl, Continuum does not make the build number available in your build scripts. Having the build number is usefull, so you can include it in an about screen, or for including it in the filename of the file you generate. I usually include a version information page with every web application which states the version and the build number of the application.

There’s an easy work around: Continuum stores the build number for your project in a the table PROJECT. You can query it like this:

select "BUILD_NUMBER" + 1 from "PROJECT" where "NAME" = 'my_project_name';

In an Ant build file you can get it like this (if you use PostgreSQL):

    <sql driver="org.postgresql.Driver"
         url="jdbc:postgresql://bserver1:5432/continuum"
         classpathref="postgresql.jdbc.lib" userid="continuum"
         password="continuum" onerror="continue" print="yes" showheaders="no"
         output="${java.io.tmpdir}/build.properties">
         select 'build=' || "BUILD_NUMBER" + 1 from "PROJECT" where "NAME" = 'project_name'
    </sql>
    <property file="${java.io.tmpdir}/build.properties"/>
    <echo file="${java.io.tmpdir}/version.txt">
Build         : ${build}
    </echo>

Nick Rudnik has a post on doing continuous integration using CruiseControl.NET for Java Projects.

We recently switched from cruisecontrol (java version) to Continuum (also a java application). I like continuum’s user interface better (not sure how it compares to CruiseControl.NET). Instead of using an xml file to configure your projects, you can do everything via a web ui. No support for project dependencies yet though. The installation is pretty easy, unzip an archive, and start the application server. Done.

I also like the fact that it uses a database to store all build information. I’ve configured it to use PostgreSQL, which enables me to easily create some extra reports on the data. We’ve also got Trac (bug database, wiki and subversion ui) on the same machine using PostgreSQL. I have no specific needs at this moment, but i think that being able to create reports combining data from both tools may be usefull. Especially if you can combine this with data from unit tests or QA tools like PMD and Findbugs. This would allow me to create a report displaying quality indications per build, like how many open bugs, have many failed units tests, etc.

Running this on linux instead of windows makes a big difference. It allows you to create a vmware virtual machine containing these tools and deploy it within 5 minutes for new projects without bothering with licensing issues. Similar to Buildix, but using Continuum instead of CruiseControl.

In a comment to BPEL Designer 10.1.3 Gaurav asks if there are any free BPEL designer tools. I’m aware of the following free bpel designers:

• Oracle JDeveloper, which includes a BPEL designer. The BPEL designer was originally developed for Eclipse by collaxa, but after Oracle bought it they ported it to JDeveloper. JDeveloper is free to use.
• ActiveBPEL Designer. Used to be a 30 day free evaluation, but now a free tool. Not open source, as far as i can tell. Plugin for Eclipse.

And some that aren’t really finished:

• Netbeans Enterprise pack. Free and Open Source, but it seems to be beta software right now.
• Apache ODE. Still in incubator, so probably not production ready. Based on Fivesight PXE BPEL engine. I’m not sure it also includes a BPEL designer.
• Eclipse BPEL Project. Still seems to be in development. But the Eclipse BPEL screenshot looks promissing.

While searching for free BPEL design tools i found this article on eWeek: Building an Open-Source SOA. It offers an overview of all the open source components you can use to implement the different parts of an open source SOA, like components for messaging, SOAP, ESB, and orchestration.

What free BPEL designers did i miss?

Just noticed on Wilfred’s blog that Oracle’s SOA Suite 10.1.3 Developer preview is available for download. It includes JDeveloper 10.1.3 Developer Preview containing the new BPEL designer. (More info: a list of fixes, and a list of new features).

We’ve been testing BPEL 10.1.3 Beta internally for some months now, expect a paper describing the improvements soon.

I’ll be glad when 10.1.3 goes production, working with 10.1.2 is a bit of a pain. I’m currently using 10.1.2 for a project, but i’m missing all the improvements in 10.1.3. No support for Ant 1.6, no subversion support, no clicking on tabs to have full screen display and many more.

Update: 10.1.3 isn’t just about BPEL as you can read on Shay Shmeltzer’s blog: One thing that I really don’t like about Oracle. This new release of JDeveloper also includes support for the final spec of EJB3 and JPA. This is also a very welcome update. I’ve been teaching EJB3 at a University recently using JDeveloper, and it was a bit of a pain to explain that the JDeveloper annotations where different from the final EJB 3 annotations. This will make teaching about EJB3 en JPA easier.

OnJava.com is running an article on JDBC 4.0 Enhancements in Java SE 6. Like with everything else in java these days, you can use annotations to simplify your code. But i’m not sure this will be the most important new feature of jdbc 4. It doesn’t seem to reduce your work a lot. Some of the other new features, like support for National Character Set Conversion (isn’t this already handled by Oracle jdbc implementation?) and improved SQL Exception handling seem more useful. Anyway, a good read to stay up to date on the new JDBC features.

In my current project we are working with Oracle BPEL as the service orchestrator in a SOA architecture. Oracle BPEL includes a well-featured human workflow mechanism to let end-users interact with the business processes in terms of BPEL usertasks. For example, an insurance company has a business process to handle and approve insurance requests. A possible user task in this process could be the explicit approval of the request by an empoyee of the insurance company.

In my project, we have now over 40 unique usertasks that are integrated in many different business processes. On top of each usertask we have build an Java UI to let the employees fulfill their task in a intuitive way. A usertask in Oracle BPEL consists of an XML message that contains the general task data and the task payload. The task payload structure varies per usertask and is defined by an XSD. To access the payload in Java in an OO way, you have to use ‘’schemac” to generate facade classes for the elements in the payload. Oracle BPEL is provided with a schemac command-line tool and an ant task. However they only can handle one XSD at once.So, when you have (as in our case) a large amount of usertasks you have to do a lot of work by yourself to generate the facade classes and wrap them in a Jar file. You can make live easier by using the schemac ant task, but then you still need to hard-code all the xsd files in your build.xml..

In this posting, I will describe how you can generate all the facade classes for all the different usertask payload structures and bundle them in a Jar by using a single Ant task and a custom batch file.

• Step 1: Put all the xsd files in a single directory. It preferrable to put this directory under version control
• Step 2: Make a dos-batch file and name it schemac_loop.bat and fill it the with the code below. Put the batch file in the same directory as where schemac.bat is located
  
  echo ------------------------------------------------------
  echo Schemac loop process
  echo Facade generator
  echo ------------------------------------------------------
  @echo off

  @REM Start the loop
  FOR %%A IN (%2\*.xsd) DO CALL SCHEMAC -d %1 "%%A"
  @REM End of the loop

  @REM Facades classes successfull generated
echo ----------------- Facades generated ---------------------


• Step 3: Make sure that Ant can locate the batch files on your local pc. Put the following in your local build properties file. Note that all the paths are full paths. If you omit this Ant will not find the batch files. At least this is what I have experienced
  
  #Schemac location
  schemac.dir = C:/OraBPEL/integration/orabpel/bin

  #XSD location
  facades.xsd.dir = C:/workspaces/libs/repository/com/facades/1.0/xsds

  #Facade classes location
facades.classes.dir = C:/workspaces/gui-tasks/view/facades/


• Step 4: Put the following targets in your build.xml file.

    <property name="facades.outdir" value="facades"/>     
  
    <!-- Generate the facade classes -->
    <target name="genFacades">
      <delete dir="${facades.outdir}"/>
      <exec dir="${schemac.dir}" executable="cmd">
        <arg line="/c schemac_loop.bat ${facades.classes.dir} ${facades.xsd.dir}"/>
      </exec>
    </target>
  
    <!-- Bundle the facade classes in a jar -->
    <target name="facadesJar" depends="genFacades">
      <jar destfile="${facades.outdir}/facades.jar"
           basedir="${facades.outdir}"
      />
    </target>
    

Ok..this is all you have to do to

Probably the question “Why didn’t you use the bpel schemac ant task instead of the batch file?” will arise now after reading till this point. The answer to this question is simple, we have to do a lot of configuration work to get the schemac task to work. BPEL itself is also running the bpel ant tasks by using the a special ant executable, called ”obant” that sets all the specific configuration settings. The solution provided in this posting was much more faster to realize. Also, I couldn’t find a build-in ant task to loop over a directory of files and execute a command on each file.

Finally, I need to mention my solution requires a Windows environment to work. However, when you are in a unix environment you can follow the steps as well with the remark that you have to use the .sh files where I use dos-batch files. Integration with a continuous integration tool like Continuum or Cruise Control running on a unix server could be fixed by using an specific build-all target that uses the unix variant to generate the facade classes.