Methodology & Analytic Upgrade

The upcoming version of CAST Highlight will introduce significant changes and improvements on indicators such as computation and methodology. The Risk aggregation system and categories, as well as estimates such as Software Maintenance and Technical Debt have all been refined to provide you with the most accurate analytics about the health of your application portfolio. We’ll deliver these changes in the June version, along with a totally revisited benchmark feature. Here’s a detailed recap of these improvements.

 
 

Highlight’s General Principles for Indicators and Methodology

Principle #1 – Continuous Refinement

One of the main purposes of CAST Highlight is to provide you with the most accurate analytics and estimates about application risks, technical debt, maintenance cost, in order to help you make more accurate decisions built on facts, trends and tangible data. We’re constantly refining Highlight’s capabilities, and implement them yearly.

Principle #2 – Onboarding Speed

In the meantime, we’re making it as easy as possible for you to onboard new applications. We know that the speed of deployment and maximum flexibility of configuring code analysis are both vital, and we’re working to make both steps within CAST Highlight respond to your needs.

Principle #3 – Platform Data Consistency

We make sure that all of your data and our trend analyses are consistent. If a calculation changes within any risks or estimates, we recompute all of the data in your portfolio, using the same algorithm throughout, including the results of past campaigns. The outcome is a consistent picture of your portfolio’s risks and opportunities.

Principle #4 – Comprehensive Insight prior to Indicator Refinement

Adjustments are made to CAST Highlight using a global benchmark of all of the applications that have been onboarded. We also regularly check with our customers who are using Highlight to make sure that the indicators we’re using are the most relevant ones possible, from both their business and technical contexts.

 

What’s better in the new version

Risk Aggregation

Until now, application risk was measured by computing a simple average of risk values of each technology of an application, regardless of the number of lines of code. One advantage was that small portions of code with high risk were clearly visible, even if 80% of other technologies were at low risk. But that had potential downsides as well: if 99% of a COBOL application was at high risk, the pretty good and very small portion of SQL didn’t make that fact obvious.

In the coming version of CAST Highlight, we’ve listened to you, and changed the risk aggregation mechanism by using technology lines of code as weights. In order to make sure that we’re not underestimating any potential risk, small but risky technologies will be still spotted, but on the risk distribution page as shown below.

Highlight-Methodology-Post-1

As a result, the risk values for your applications will change. However, as mentioned above, trend lines will be computed using the same algorithm, ensuring consistency from all of your portal’s analytics.


Risk Category Thresholds

Until now, our three risks (production, adaptability and software complexity) have used a common set of thresholds to define the level of risk (i.e.: below 25 for green/low risk, 50 for orange/medium risk and 51 and above for high risks).

Highlight-Methodology-Post-3

Now, each risk will have its own specific threshold, defining the risk level. We’ve defined these thresholds by watching and benchmarking all of the applications in the CAST Highlight database.


Technical Debt Estimates

As you probably know, Technical Debt estimates in CAST Highlight rely on statistics we get from Appmarq, one of the world’s biggest data repositories on software quality. We’ve fixed some quartile positioning issues and updated the average $/LOC for each technology, since the previous update, two years ago.

As a result, we’ve improved Highlight to deliver more pragmatic technical debt estimates, when compared with previous versions. Now, your estimates will be more reliable, delivering actionable analytics to discuss with the people using your applications.


Software Maintenance Estimates

Based on interactions we’ve had with our customers since the previous release of CAST Highlight, we’ve reviewed the way we use Highlight metrics for running the COCOMO model that is used to compute Software Maintenance estimates. As a result, we’ve extended COCOMO coverage by adding several new parameters that feed the computational model, providing better FTE estimates on maintenance effort.

Highlight-Methodology-Post-2

To make these indicators easy to read, so that you can compare them with actual maintenance estimates, we’ve added a tooltip into the chart that recaps the key drivers (i.e., lines of code, percentage of efforts allocated to maintenance, percentage of change of the code base). The result is a figure that quantifies the number of lines of code being maintained per FTE per year.