Support OzOriginal

Home Cases BI Reporting
BI Reporting case
Print E-mail

BI Reporting case

 

1. Background

As a part of its services company “X” delivers a large quantities different type of complex financial reports to their clients in forms of HTML accessible through Web site (under authenticated login), disseminating PDFs via emails and downloadables through their Website and hard copy printed documents in different formats via post.

 

2. Specifics and possible complexities

Existing reporting system written in-house in Java is very complex, has large number of old not used business cases, the design doesn’t fully fit newly arisen business cases making very hard to maintain the application. As a part of existing application there is a big deal of sending specific printer codes for large industrial printers the company uses for mass report printing.

The current reporting solution consists of JMS queues, where hundreds report jobs sent through, several report J2SE applications running reading jobs from queue, files travelling in filesystem with NFS mounts complexities (file handles “run out” sometimes), database load issues. All of it makes the solution fragile and hard to maintain.

Every new report or business case is developed first by BA via analysis and detail documenting the case and expected solution, then discussed with a developer. Then a developer gets back to BA with potential alternative solution as often BA has no idea what is possible and easy to implement within the current system and what might become a nightmare or risky for deadline and maintenance. The negotiated solution gets implemented. Often BA comes with amendments as the solution comes to live.

A new requirement just came to add dashboard for Website and a series of modern style reports. The question comes if it is time to rewrite old reporting system or redesign existing reusing complex parts like already written system for specific printer codes.

3. Architectural review

A request for review of architecture of the application came to GA group after a first internal inception review with a set of materials incomplete. Existing architecture documents were not provided either due to non existence or they were very outdated. Depending on the outcome it was proposed to fast track the solution. Reaction - it is normal for management expectation to get something from nothing.

The review included analysis of source code, an executable application from within a development environment, the partitioning scheme for the application, a running, fully deployed version of the application. We assessed teams involved in the development and deployment process - developers, architects, testers, BAs, prod support, management.

We discussed overview, structure of the application, relationships of the packages, possible impacts this structure has on maintainability, performance and deployment, roles, responsibilities and relationships to discover issues of performance, reliability and deployment, partitioning, review of runtime properties performance review.

A report comprised of an analysis and a set of recommendations was delivered within one week.

 

Main points:

Issue

Architecture focus

Current design doesn’t fit last business cases, very hard to adopt to new ones

Redesign existing solution and reuse some of existing code base for a new solution or fully new architecture with off the shelf solution

Old complex code requires high maintenance

Analyse if the relation of the issue to a core business of the company. As reports creation determined as not a core business, the focus is moved to off the shelf solution

Printer code specifics - the in-house solution for large industrial printers recognised as a source of endless problems

Focus on standardisation

The current J2SE solution is fragile

Enterprise solution


4. Solution

  1. Use Business Intelligence Reporting Tools available on a market for reporting and data visualization. Such tools and technologies are used to build rich information applications.

  2. Two stage approach: a) starting with Open Source approach, b) then moving to commercial enterprise version.

  3. No printer low level specifics like printer codes to manage. Modern printers are capable of printing PDF files in any paper format PDF file is present. For hard copy mass printing use a deployed BI Reporting System to produce PDFs to be printed. Also on a long term a suggestion was to outsource printing to professional printer companies.

  4. Cross team collaboration:

    1. developers to set up data cubes, data sets and other related tasks

    2. BA teams to create actual reports

    3. Pair works will be used between programmers and BAs

  5. Use an Enterprise Server solution for scalability and reliability

5. Results

  1. The average new business case time to market was shortened from 2 months to 4 days.

  2. Painful maintenance reduced dramatically, mostly managed by BAs with fast turn without application rebuilds, no long QA cycle.

  3. Reliability of the system increased a lot due to use of existing clustered J2EE environment.

  4. Overall solution took 3 month to prod release, which was shorter than expected new set of reports to deliver within old system.

6. Guerrilla tactics employed

  1. Team coordination, cross team collaboration. Originally BA and developers were considered “opponent teams”, with BAs “pushing” developers and developers “pushing back”. Middle management negative team dynamics were recorded. New approach with pair work between developers and BA exceeded any expectations. Now BAs themselves create report they wish to have without long and multi staged discussions with developers help if required.

  2. Low-cost unconventional approaches include:

    1. Starting with Open Source approach for no cost “foot in a door” as a new concept accepting exercise. Then moving to commercial enterprise version once the concept got  accepted.

    2. Using standard PDF printing ability of any printer rather than fiddling around with low level printer codes (through negative team dynamics due to pride of current solution)

    3. No reporting coding at all. A small coding for integration is simple to achieve following ready examples provided by BI Reporting tools vendor. A developer was positively challenged and delivered fully working solution in 3 days.

    4. J2EE based solution using existed clustered J2EE server with existing licensing, hardware, support etc.

    5. Quick mockup with demo BI Reporting tools in 3 days and a good will from BA team allowed to make a decision.

  3. While most of stakeholders agreed for the solution, some guerrilla warfare took place due to a resistance to abandon well known old solution, low level printer codes approach and BI concept acceptance. Interviewing different teams at different company hierarchical levels and investigating the real situation helped to facilitate the solution. People from different teams were positively challenged, a new vision was given to management, new BI trends were brought to awareness.

     

7. Team dynamics

 

Issues found


  1. The hardest part of reporting business cases were in BA/Developer communication. Lots of detailed documents were to be produced by BA, then communicated to developer, understood by developer and implemented, often changed to fit their way of doing things. Some of worst written code fragments were reflecting hardest issues of such communications.

  2. Developer’s decision often is based on their personal technology attachment and pre-existing application design. In this instance the decision making for developers was hand crafted Jasper Reports without using any RAD tools (a “real man” approach). It made it impossible to go any closer in imagination to BI Reporting solutions ideas for years.

  3. Another developers attachment for J2SE solutions made impossible to look at J2EE solutions to solve scalability and reliability issues.

  4. Having no independent (from Dev team) in-house R&D, BAs have never been asked for their opinion, although some were aware of existence and trends in BI Reporting. An off-the-shelf solution was missed

     

Correction


  1. Changed decision making control and ongoing processes. Technology choice advised to be allocated to a full time IT architect position.

  2. Implement pair works (pair programming analogue) between developers and BAs (and others) as a normal process.

8. Business strategy

To define our positioning in the market we have to choose how we define the core of our business. Only core business elements should not be be outsourced, paid to get developed externally, bought off the shelf. And the opposite often is true - what is not a core is a great candidate to be outsourced. The core is defined by our most valuable products, most important channels and distinctive capabilities. The art and science of business strategy is in defining oneself as unique in its kind.
The crucial point in this case was to determine if the reporting code, application, precious old project is the core of the business. Once it was determined that it is not, the solution direction was much easier to take.
 

Add comment


Security code
Refresh



Powered by Investor Services. Valid XHTML and CSS.