Product Security

“The more precisely the position is determined, the less precisely the momentum is known in this instant, and vice versa.”
–Heisenberg, uncertainty paper, 1927

The Uncertainty principle is related to the observer effect. In physics, the term observer effect refers to changes that the act of observation will make on the phenomenon being observed.

Ok, now to get to the point. Leaders are often asked to produce several performance metrics or revenue metrics. Some of these metrics are simple and straightforward  Key Performance Indicators (KPIs). KPIs can include net revenue, profit, # of new customers or in our case customer satisfaction numbers.

The problem with metrics crops up when we need to measure a property and no mechanism exists to measure it quickly or the metric is not representative of the property being measured. In general this happens when the following scenarios arise:

1. Metric is not available: No mechanism is in place to measure the property at that time.
2. Property is not measurable: No metrics are available to capture the property.
3. Deliver unplanned metrics quickly: Metrics that the system was not designed to capture need to be measured quickly.
4. CSF masquerading as KPI: Critical Success Factors are vital elements for a strategy to be successful and should not be confused with KPIs which quantify strategic performance.  The metric being asked for is a CSF not a KPI.

In simple words, the amount of effort required to measure the metric changes the amount of effort we can dedicate to create the metric. The act of measuring the metric changes it.  For example, in the economic downturn several teams have had to reduce headcount. If this barebones team is now asked to capture  information on how a recently released tool is being used by customers without that mechanism already in place, then they cannot deliver that metric without additional effort that will impact the overall KPIs.The problem that arises is what I’ve dubbed the Akshay’s Uncertainty Principle:

In a resource constrained environment, a new or modified metric cannot be measured without impacting the metric itself.

– Akshay Aggarwal

Finally, an explanation the kxcd way

Several enterprises are increasingly investing time and money in building application security tasks into their existing SDLCs. Some of them have also reached the conclusion that proactive approaches , like threat modeling, have more ROI than reactive approaches. As a result, some enterprises with nascent appsec programs have turned to threat modeling as a panacea for their security problems. However, threat modeling may not be the solution to their immediate problems. Now I recognize that this may be a controversial statement.

Recently, I have been involved in several situations where organizations with their heart in the right place have made threat modeling mandatory as part of the development process, with limited success. My point is that threat modeling as part of a mature SDLC is a desired end state though not necessarily the initial step. Let’s examine this argument.

Firstly, threat modeling depends on several elements of a SDLC to be fairly mature. Most importantly it depends on requirement and specification gathering process to be rigorous. Also, an enterprise must have well defined standards and policies in place to act as input into the threat modeling process. Without these elements of the SDLC in place, the threat modeling process will be isolated and have a reduced impact.

Secondly, a threat model is a security plan only and is useless without any committed follow-up action as part of development and testing. Most enterprises do not allocate sufficient time and resources to implement the findings of the threat model. A large portion of organizations don’t even have a security assessment team in place. These teams are consumers of the threat modeling process that actual carry out the most crucial task of reducing risk by implementing countermeasures.

Thirdly, it is practically feasible to create threat models only for new projects or those undergoing incremental changes. As a result, legacy applications do not benefit from threat modeling. This leaves a huge gap in the enterprises’ risk profile.

Finally, most nascent application security programs need quick and demonstrable ROI. The threat modeling process ROI can take several months or even years to be quantifiable because it is an incremental process that is dependant on several other SDLC processes to be effective. There are other areas where investment can bring in more immediate ROI. These areas include security assessment team, security training for developers and definition of countermeasures for common vulnerabilities.

For organizations with nascent application security processes, I recommend that they us the following framework to evaluate if they are ready to adopt threat modeling:

• Does a security baseline exist?
• Is the SDLC process fairly well defined and followed during development?
• Has the organization agreed upon countermeasures for common vulnerabilities?
• Are developers trained to avoid common vulnerabilities?
• Do developers do a self review of code for security vulnerabilities?
• Does a security assessment team exist?

If the answer to more than two of the questions above is no then the organization is probably not ready for adopting threat modeling.