Using Oracle BI Baseline Validation Tool

Experimenting with the new Oracle BI Baseline Validation Tool

The Oracle BI Baseline Validation Tool is a long overdue new tool introduced in 12c that allows BI administrators to perform regression testing after a BI migration or a BI upgrade. The goal of this post is to document my experimentation with this new tool in Windows 7. As an example I’ve got an environment with OBIEE installed and another equivalent environment with version The BVT is run from the command with a configuration file and set of arguments. Following are the high-level steps for using the Oracle BI BVT:

  1. Establish a baseline (set of dashboards, reports), in this example we compare all items
  2. Configure the configuration file to specify the location of the target catalog, Oracle BI Presentation Services login information, and the plug-ins to activate. A sample configuration testconfig.xml is provided out the box, one can create a new using the BVT utility
  3. Run the Oracle BVT tool three times:
    • First run to generate the baseline
    • Second run to generate the same data from the patched environment
    • Third run to compare results between the before and after web catalog

The BVT allows a fine grain level comparison allowing even the ability to set a maximum tolerance for floating point fluctuations between reports.

Oracle BI Validation Tool Installation
  • Java 1.6 and higher
  • Windows: Windows 7 and higher, Windows Server 2012 and higher
  • Linux: Oracle Linux U5 or Oracle Linux 7
  • Mozilla Firefox or IE

It  is shipped with the Oracle Business Intelligence Developer Client Tool, after installation, the tool is located in

[Installation Folder]\bi\components\

BVT tool

Once the zip file is uncompressed

BVT tool

Configuration File

BVT tool

Looking at the sample configuration file, there are 2 sections:


Description of the target environment that the tests will run against, the deployment name can be updated in order to describe your specific environment, in our case we will define the following deployment names; “OBIEE11117base” to designate our baseline OBIEE environment and ““OBIEE11119upgrade” to designate the upgraded environment. The deployment names will be fed to the BVT as command line arguments

Test Plug-ins
BVT tool

This section describes the different types of comparisons (5) that are available for the web catalog; each type of comparison is developed as a plug-in. Following are the plug-ns available out of the box:

  • Catalog Plug-in Compares the metadata of objects in catalog folders,
  • UI Plug-in [] : Compares screen captures of individual analyses and dashboards.
  • Reports Plug-in []: Exports the results of the analyses to CSV and generates a comparison report.
  • Logical Query Plug-in [] : Downloads and saves the logical SQL generated for analyses
  • Dashboard Plug-in []: Exports the results of the analyses to XML and generates a comparison

The plug-ins can be enabled by setting the “enabled” tag to true.

Let’s update the sample configuration file and capture the baseline for our OBIEE web catalog. We enter the URL for the BI environment and the username/password. Note that you can chose to enter the password as a command line argument with the “-password” flag followed by the actual password; if no value is passed with the flag, you will be prompted to enter it.

BVT tool

We run the following command in order to capture our baseline, the syntax to capture the configuration for our BI is:

bin\obibvt.bat -config TestConfig.xml -deployment BI11117base

BVT tool

Where testconfig.xml is the name of the configuration file and BI111117base is the deployment name to run. As an example, we’ve enabled all the tests and run the BVT to capture our OBIEE baseline. A folder named after the deployment name is created under the results folder, a progress status is displayed on the screen while the processing is underway.

BVT tool

Note that you can set timeout for loading each report, by default, the timeout to download a single report is 15000 milliseconds (15 seconds). Once the script is done executing, sub folders are created in the main results folder

BVT tool

We run the script again in order to capture the environment, this will enable us to run comparisons between environments

bin\obibvt.bat -config TestConfig.xml -deployment BI11119

Once the script is done running, a second folder is created in the main Results folder

BVT tool

We can now run the same script with the –compareresults flag in order to perform comparisons between the and the environments.

BVT tool

Now navigate to the Comparisons folders and examine the differences between and; for each test there is an html file with the color coded test results, relevant details and hyperlinks to items of concern.

Web Catalog comparisons

Details are provided such as :

  • number of of BI objects in the base environment
  • number of BI objects in the target environment
  • items only present in the base environment
  • items only present in the target environment
  • items that are different

Drill down links provide more details where there are exceptions

BVT tool
Dashboard comparisons

Differences between dashboard details are provided.

BVT tool
Logical Query comparisons

Differences in logical SQLs between both environments are documented.

BVT tool
Reports Comparisons

An export to CSV of the results for each of the analyses is performed for each environment and a comparison report is performed.

BVT tool
UI comparisons

UI plugin compares screen captures pixel by pixel of individual analyses and dashboards and displays a comparison report side by side. A score is assigned to each comparison based on the match: a score of 1 indicates a perfect match while 0 indicates a complete mismatch. The threshold for a pass of fail score can be set in the configuration file, it is 0.95 by default. The side by side comparisons is very handy and help you see the discrepancies in look and feel.

BVT tool

p.s: Note that we were only to have all tests run successfully using Firefox version 34.0; tests will fail while running with Internet 11.0 and Firefox 42.0 we got the following error in the log:

NS_ERROR_XPC_CI_RETURNED_FAILURE: Component returned failure code: 0x80570015 (NS_ERROR_XPC_CI_RETURNED_FAILURE)

Apparently this error is caused by the Selenium web driver in later version of Firefox, you can read more about it here.

It is worth noting that there is a flag in the configuration file that allows you (1) to set the default browser for running the tests and (2) another flag where you can specify the number of browser instances to run concurrently while performing the tests.

BVT tool

The goal of this post was to document a preliminary experiment with the new regression testing tool that shipped with obiee 12c; setting up the configuration and the running comparisons is fairly straightforward; the results are detailed and handy. There is no doubt that this tool will be useful for OBIEE regression testing. In conclusion, the Oracle BI development team is making big strides in addressing pain points that have made working with OBIEE and migrating and maintaining different Oracle BI environments a challenge. One shortcoming of the tool though is that BI-Publisher reports are not included in the comparisons, hopefully future releases of the BVT will include this type of comparisons.

Leave a Comments

Your browser doesn't support the canvas element - please visit in a modern browser.

(new shape)