Regression Testing Using Baseline Validation Tool
Oracle has introduced a Baseline Validation Tool (BVT) with OBIEE 12c. This tool helps to validate our source configurations, such as whether the Catalogs, Reports, Dashboards, User Interface (UI), BI Publishers, Visual Analyser (or) Data Visualization and Logical Queries have been transferred to target system successfully or not. The BVT gives us a consolidated report for most of our concerns and clarify our queries regarding Business Intelligence upgradation or between two environments. This can be called as Regression Testing using BVT.
Unlike Normal Testing, it will be challenging in Business Intelligence (BI) testing to validate the data from source system to target. This tool will test your environment in six ways, where you can run these tests either one at a time or all of it at the same time. The validation time depends on the number of reports and their performance.
Prerequisites for Installing BVT
- Java 1.8 or higher version
- Oracle Linux 6 or 7 or higher
- Windows 7 or Server 2012 or Higher version
- Mozilla Firefox or IE
Basically BVT is a command line utility that functions like regression testing on Oracle BI presentation catalog on two different compatible systems. This utility can be available in .zip format and downloaded in Oracle website. After downloading, place them into separate folder.
Edit the configurations in XML page, providing some details such as Analytics URL, Xmlp URL (BI Publisher), Visual Analyzer URL, Username, and Password. The need to specify the SSL Certificate errors can be ignored. The process of configuring baseline validation tool has been given below.
Sample testconfiguraton.xml file has been given below for reference.
Executing BVT in Various Environments
This BVT has to run 3 times – first for the lower system (source system), second for the higher version system (target system), and third for the comparison of both the above systems. Running scripts for each of the systems gives you results from source system, target system, and comparison respectively.
In the above figure, PreUpgrade consists of Source system, PostUpgrade consists of Target system, and Comparison has the results of both the systems.
Example: PreUpgrade – OBIEE 11g and PostUpgrade – OBIEE12c or OAC.
Viewing the Results
BVT creates a folder called COMPARISON, which has sub folders consisting of respective object details and information about presentation catalog. After reviewing results from the source system to target system with respect to the comparison results, carry out the necessary changes before you migrate or test it from source system (OBIEE 11g) to the new environment (OBIEE 12c or Oracle Analytics cloud).
Catalog Test: This catalog plug-in test will give an export of metadata between two environments. It also gives us permission to objects such as target path, type, caption and signature of catalog.
UI Test: This test will capture the screenshots of comparing reports and will present to you in the range of 0 to 1. If it is 1, it means that it is perfectly matched pixel to pixel. We can also mention some metric values ranging from 0 to 1, as 0.90 range is the passing value for matching reports.
Report Test: After running these tests, the data can be compared between reports by exporting it in csv, excel or PDF. It is recommended to export in csv type, if the reports are long running.
Dashboard Test: This result will give you export of XML files from respective dashboards. It consists of data, charts, and objects on each dashboard.
BIP Test: This test exports your result in PDF format and compares both the source and target side. We need to provide if any of our BI publisher reports has parameters.
VA Test: This test is similar to UI Test, capturing screens of data visualization or visual analyzer. As the score nears to 1, more accurate matches are made pixel by pixel. We can also specify threshold values here.
Logical Query: This test gives us the logical query between two reports, which can be seen in Advance tab of your editing analysis.
[Note: This plug-in test was obsolete in recent BVT versions].
All plug-ins consists HTML page as output. To view them, we need to open respective folder and open HTML page in any browser. This is how we can compare results from source system to target system environment. After getting a match, all results can be migrated (or) tested to production environment, making it available for users.
Contact for further details
Technology Specialist – Analytics Oracle