1. Introduction
Getting a software application development completed on time, within budget with all functional and non-functional requirements originally specified has been a challenge and sometimes impossible. This issue often arises because requirements are not completely understood early enough in the development process. Non-functional requirements are often inadequately specified. As a result, many software architecture designs do not efficiently support the business needs.
As systems become larger and more complicated, the need for a structured development process becomes crucial. The right software architecture design can help improve timing of system updates, modifications and cost. To help developers determine the best architecture design for their business needs, especially the non-functional requirements such as performance, availability, security, etc., the Software Engineering Institute (SEI) at Carnegie Mellon University developed Architecture Trade-Off Analysis Method (ATAM) [1] and Cost Benefit Analysis Method (CBAM)) [1,2]. CBAM generally follows ATAM that aids in defining scenarios for nonfunctional requirements specification and analyzing architectural strategies. The result of ATAM is a qualitative trade-off analysis of the various design strategies. CBAM further refines the scenarios and architectural design strategies from ATAM. CBAM aids in quantitative analysis for cost, utility and importance ratings to determine the ROI of each architectural design strategy. These architecture centric methods can improve the quality of software systems. However, using these methods requires having access to an expert of the methods. Manually practicing these processes can be time consuming. Thus, it is desirable to have tool support for these methods.
This paper presents CBAM Assistant, a web-based semi-automation of the CBAM process. As a quantitative method, CBAM has its great value for software architecture design analysis. The development of CBAM Assistant aims at facilitating the application of CBAM so it can be widely used. CBAM Assistant walks a user through the CBAM process which can be started by using scenarios and architectural strategies created from ATAM. This tool is intended to be used by a facilitator who will provide input from stakeholders. The primary output of the tool is the ROIs of each architectural design strategy for comparison and selection.
2. Background and Related Work
2.1. ATAM
ATAM is a nine-step, four-phase process in which the stakeholders define scenarios to specify the non-functional requirements for software architecture design. Quality attribute scenario specifications provide a quantifiable requirement for quality attributes (performance, availability, modifiability, usability, security, etc.). The stakeholders prioritize the scenarios, identify and evaluate architectural approaches.
ATAM also identifies trade-offs and risks in software architecture designs. For example as a system becomes more secure, the availability of the system may have to decrease. The trade-off points are identified for scenarios that affect other quality attributes as a side effect.
The output of ATAM includes: scenarios, mapping of architectural decisions to the scenarios, list of sensitivity and trade-off points and a list of risks and non-risks. While ATAM provides the developer with guidance to which architectural strategies would best suit the business needs, it does not incorporate the economic impact of these strategies. This is where CBAM can be effective.
2.2. CBAM
CBAM uses the output from ATAM as a starting point. It can be used independently from ATAM. CBAM helps to ensure that non-functional requirements or quality attributes are fully stated. CBAM elicits and documents the benefit (or utility) and cost associated with each architectural strategy defined. The cost of implementing each architectural strategy is also gathered from the stakeholders and then analyzed. The output from CBAM summarizes the ROI (Return On Investment) for each architectural strategy in consideration. ROI is calculated to show which strategy yields the most benefit. CBAM helps the developers choose the architecture design that will yield the best return on investment for the software system. There are few techniques found that incorporate the economic impact into software architecture analysis.
CBAM is an incremental nine-step process. First, stakeholders define, refine and further prioritize scenarios. The top 1/3 scenarios are selected based on priorities established. In Step 2 stakeholders define the response goals for the best, worst, current and desired cases for the selected scenarios. In Step 3 the stakeholders vote on each of the selected scenarios considering the expected response goals. 100 votes in total are allocated by the stakeholders. The votes are used to rank the priority of each scenario. The scenario with the most votes has the highest priority. Step 4 uses the top 50% of the scenarios from Step 3 based on votes (i.e. top 1/6 of the total). In Step 4, stakeholders assign a utility rating to each response goal (best, worst, current and desired) for the selected scenarios.
In Step 5 stakeholders develop or review architectural strategies for the top 1/6 scenarios. A cost is assigned to each architectural strategy. The scenarios impacted by the strategy must also be defined. In Step 6, the stakeholders determine the expected response goals and utility rates for each scenario and architectural strategy.
The benefit is calculated in Step 7. The benefit for each architectural strategy is based on the current utility rating and expected utility rating for each scenario impacted by the strategy. The benefit (b) for a given scenario (i) based on strategy (j) is calculated bij = Uexpected – Ucurrent. Uexpected is the utility expected if the scenario is implemented and Ucurrent is the current utility.
The total benefit for the strategy is the sum of the weighted benefit for each scenario impacted by that strategy. The weight (wi) of a scenario is based on votes. The total benefit (bj) for an architectural strategy is the sum of the weighted benefit for each scenario impacted by the strategy or 
where wi is the weighting for scenario i. The ROI is the ratio of total benefit for an architectural strategy divided by the estimated cost of implementing the strategy.
The output of CBAM is the ROI for each strategy being compared. The output also includes a list of architectural strategies with their expected utility and total benefit. In Step 8, the developer uses CBAM output to determine the architectural strategies that best suits the business needs. In Step 9, the developer must confirm results based on experience and intuition. For example if a costly strategy that the developer expected to have lower benefit, is returned with the best ROI, the developer may need to review prior steps for accuracy and revisions may be required. The user can run an updated report and review again.
2.3. CBAM and ATAM Integrated
SEI also developed an integrated method for ATAM and CBAM [3]. This method optimized the steps if both ATAM and CBAM are going to be performed. The integrated process further differentiates between new architecture and changes to existing architecture. The process is intended to save stakeholders time and to be more precise than doing ATAM and CBAM separately.
2.4. ATAM Assistant
ATAM Assistant [4] is a tool developed to semi-automate the ATAM process. ATAM Assistant is a desktop tool that walks a user through the various phases of the ATAM process. The ATAM Assistant tool provides output including architectural approaches discovered, scenarios defined, votes and priorities for scenarios. This output can be used as inputs for CBAM Assistant. The tools are not fully integrated at this time and a user must manually enter inputs into CBAM Assistant, but this can be automated. CBAM Assistant also requires additional information to calculate the ROI of each architectural strategy.
2.5. Other Methods
Other methods for evaluating software architecture include: Software Architecture Analysis Method (SAAM) [5], Software Architecture Analysis Method Founded on Complex Scenarios (SAAMCS) [6], Software Architecture Analysis Method for Evolution and Reusability (SAAMER) [7], Scenario-Based Architecture Reengineering (SBAR) [8], Extending SAAM by Integration in the Domain (ESAAMI) [9], Architecture Level Prediction of Software Maintenance (ALPSM) [10] and Software Evaluation Model (SAEM) [11].
SAAM is a five-step method to analyze software architecture. SAAM evaluates impact of architecture on each scenario. The result of SAAM is an evaluation of how well each architectural design supports various tasks. SAAMCS is a risk assessment. The output of SAAMCS shows various factors that indicate the number of components affected, how complicated maintenance will be and a conflict rating. SAAMER builds on SAAM. SAMMER evaluates the evolution and reusability of architecture. SAAMER determines the cost of an architecture, where the cost in terms of effort required for change. SBAR determines the most efficient use of an architecture design to achieve quality attribute goals. ESAAMI expands SAAM to focus on reuse within the specified domain. ALPSM analyzes maintainability and SAEM predicts system quality.
Some methods evaluate specific quality attributes. SAAM, ESAAMI and ALSPSM evaluate modifiability [12]. SAAMCS evaluates flexibility. SAAMER evaluates evolution and reusability and SAEM evaluates overall quality. CBAM varies from these methods by evaluating multiple architectural strategies and quality attributes. ATAM & SBAR evaluate multiple quality attributes and architectural designs but provide qualitative results where CBAM is quantitative. CBAM also includes an economic impact which is not considered in the other methods.
3. Design and Implementation
3.1. Architecture
CBAM Assistant walks a user through the CBAM process which can be started by using non-functional requirement scenarios and architectural strategies created from ATAM. The tool takes inputs from stakeholders during the process, performs quantitative architectural analysis, and generates reports primarily showing the ROIs of each architectural design strategy for comparison and selection.
Model-View-Controller (MVC) architecture pattern was chosen for the CBAM Assistant application. MVC separates concerns so that one can change the design of a web page without changing the data content of the page. This is done within the views. The models get the data content to be displayed. Queries in the models can be modified without affecting the web page, as long as the components do not change. The business logic is contained within the controller. The project is intended to be available for future enhancements. The architecture allows other programmers to work on a single component without having knowledge of the entire system.
CBAM Assistant is built in Microsoft’s ASP.NET 4.0 using MVC 3 (Model-View-Controller). ASP.NET is part of Microsoft Visual Studio (2010) which is integrated with SQL Server Express (10.0.1600). Data classes and relationships are managed by Linq to SQL which is part of the Visual Studio package. The framework provides a lot of built in functionality for routing, user authentication and validation. The project can be easily modified and is available to developers for future enhancements.
In MVC, the model defines the data components (view models), data repositories and validation needed for each view. The controller contains the business logic. The controller uses the view models and repositories to send information to the view. The view simply displays the data. The view expects information in the form of some view model object.
As shown in Figure 1, the CBAM Assistant architecture consists of the CBAM database, CBAM data classes, models (view models and repositories), the controllers and views. NPOI is an open source Apache project that facilitates generating Microsoft Excel reports for the user [13]. The CBAM data classes are abstractions of the database. Users do not directly access the database. The CBAM models contain data repositories and data views which describe data components available for use in each view. For example, if a user clicks on “Create Scenario” in the scenario view, the request goes to the scenario controller. The controller then gets information such as project name and project ID from the model repository then returns the scenario view model to be displayed. The view describes
the format and layout of the information sent by the controller and displays the information to the user.
The CBAM Assistant models contain repositories and view models. The model repositories contain the functions needed to get or modify data. The view models define objects and the relevant components for a specific view. The validation rules for the components are defined as well. Recall, CBAM is an incremental process. The prior step must be completed before one can move on. The validation for the same object can vary depending on the step. For example, a vote for each scenario is not required in CBAM Step 1, yet it is required for scenarios in CBAM Step 3. ASP.NET MVC makes it easy to define the validation rules for each model separate from the view.
The CBAM Assistant views display information provided by the controller. Each web page displays at least one view and may include partial views. When a user submits information or clicks on the page, the request is sent to the controller.
The CBAM Assistant controllers process requests from the view. Updates are made through the model repositories and new information for the next display is requested from the model views. The controller then responds to the view and sends the new output.
The reports are generated in excel using NPOI. NPOI is an open source apache project. It allows one to read and write to excel without having excel installed on the server. NPOI currently supports Excel 2003. An XML template is used to define the report format. There are multiple sheets defined within the template. The Report controller handles rendering the final report. The report controller calls a stored procedure to calculate some of the final numbers for the output. The controller then gets the data for each sheet and writes the data to the template using NPOI. Then the controller sends the excel report as an HTTP response. The end result is a downloaded excel report for the user.
Data classes show how the models, views and controllers interact in CBAM Assistant. CBAM Assistant uses JQuery tabs to display information for each step. Validation varies at each step in the project. For example in Step 4 users assign utility to the top 1/6 scenarios. Utility is not required prior to this step. The following section explains the role of the models, views and controllers. Step 4, assigning utility, will be used in the examples.
3.2. Data Classes
The structure and relationships of the data objects are defined in the data classes. The data classes have corresponding repositories and view models. The repositories define actions and queries to be performed and the view model defines the data components needed for a specific view.
Figure 2 shows the data classes. Steps are updated by the system after a user makes a change. Each project has a list of scenarios and a list of architectural strategies. The utility class contains the current, best, worst and desired utility and response goals for each scenario. The relationship between scenarios and architectural strategies are defined in the expected utility class. The expected utility class identifies the response goal and expected utility for each scenario that is impacted by an architectural strategy. The Importance and Quality Attribute classes are lookup classes that hold descriptions only. The measures needed to calculate the ROIs are defined in the benefit class. A stored procedure, spGetBenefit does the calculations to get the measures needed for the ROI.