ii. Measurement

II. Measurement

Risk Standard 10: Valuation procedures

All readily priced instruments should be valued daily, less-readily priced instruments at least weekly and non-readily priced instruments as often as feasible and whenever a material event occurs. The pricing mechanism and methodologies must be known, understood, follow written policies and be applied consistently by the Primary and Manager Fiduciaries, Managers, custodian and other subcontractors

Accurate and frequent valuation is a fundamental aspect of measuring risk and monitoring compliance within the stated strategy and established risk limits. As a practical matter, the frequency of valuation depends on the instrument.

Readily priced instruments such as publicly traded securities, exchange-listed futures and options, and many over-the-counter securities and derivatives can be priced daily. These instruments are often tracked and priced by exchanges, data vendors, brokers and dealers. Actual market price information may be obtained either electronically or in published form. Positions or portfolios may be valued or "marked-to-market" on the basis of such quotes.

The Primary and Manager Fiduciaries should each set up procedures for pricing these instruments daily. Often this task is delegated to an external portfolio management service, custodian or pricing service after appropriate procedures, quality controls and checks and balances are established. Pricing may be performed internally, as long as there are appropriate checks and balances and independent verification (Risk Standard 3). The Primary Fiduciary should document and understand the pricing procedures and these should be consistent across all Managers, the custodian and other subcontractors

Less-readily priced instruments, such as complex CMOs, exotic derivatives, many private placement notes and other custom instruments should be priced as often as possible and at least weekly. Often the values of less-readily priced instruments provided by dealers, custodians and third-party pricing services are based on theoretical models. Because these valuations are not based on market sale prices, these instruments are sometimes said to be "marked-to-model."

For such instruments, the model and pricing mechanism must be made explicit so that they may be verified independently (Risk Standard 19). The Primary and Manager Fiduciaries should document the procedures, including key data assumptions and type of model (such as matrix pricing or option-adjusted spread models). They should also analyze the degree of model risk that exists for such instruments (Risk Standard 16). The Primary Fiduciary should ensure the procedures for similar instruments are consistent across all Managers, the custodian and other subcontractors.

The Primary and Manager Fiduciaries should document the basis of the valuations; for example, firm bid/offers, broker/dealer indications or estimates based on internal models. In all cases, the Primary and Manager Fiduciary should document what methodology will be used, specify the minimum number of brokers or dealers to be polled and establish a policy for selecting a single valuation if there are disparate quotes.

Non-readily priced assets such as real estate and private equity stakes are difficult to price frequently. The Primary and Manager Fiduciaries should make explicit what valuation method (such as theoretical model, appraisal, committee estimate or single-dealer quote) should be used to facilitate independent verification (Risk Standard 19).

For such instruments, valuations should be performed as frequently as feasible and whenever a material event occurs. Note that the Primary and Manager Fiduciaries should define "material" (Risk Standard 2). The key drivers of value of a non-readily priced instrument should be determined (Risk Standard 12). Changes in key drivers and key events should trigger a valuation update.

The Primary and Manager Fiduciaries should document the accuracy and reliability of pricing data and ensure that all valuations and risk/return measurements are performed consistently across Managers, the custodian and other subcontractors as well as independently of them (Risk Standards 3 and 5). The ultimate authority on valuation for each instrument type should be determined by the Primary Fiduciary and stated in writing. Exceptions to any valuation procedures should be known and reported under established policies (Risk Standard 9). The Primary and Manager Fiduciaries should approve explicitly any valuation process that relies upon the Manager who holds the asset. Investors should also take care that multiple valuations do not ultimately depend on a single pricing source.

The Primary and Manager Fiduciaries should allocate the necessary resources to value non-readily priced instruments. To the degree this increases the cost of holding certain investments, the increased cost should be considered when measuring performance.
For all instrument types, the Primary and Manager Fiduciaries should determine whether the pricing agent has incentives to inflate or deflate valuations. Finally, valuations should be reported and all trades recorded and documented on a timely basis (generally daily).

Risk Standard 11: Valuation reconciliation, bid/offer adjustments and overrides

Material discrepancies in valuations from different sources should be reconciled following established procedures. A procedure for bid/offer adjustments and overrides to valuations should be established in writing and monitored independently.

Differences in valuations between different Managers or between Managers and custodians or other subcontractors should be reconciled under established procedures at least monthly, or more frequently if material differences occur. If consistent valuation procedures are applied by the Managers, custodian and other subcontractors, any price differences can usually be explained by error, bid/offer spread adjustments or valuation overrides.

For all instruments, the Primary and Manager Fiduciaries should approve and document the mechanism for any bid/offer adjustments. For example, complex engineered securities tailored to meet the needs of a specific Manager may contain a substantial bid/offer spread that reflects illiquidity, operational burdens on the dealer, the cost of innovation and a dealer markup. The bid/offer adjustment may be proposed by the Manager in order to amortize the bid/offer spread over the life of the asset. Bid/offer adjustments may be required for simple instruments as well. Large positions in readily priced stocks or bonds may be valued by custodians and others based on closing prices for small round lots and thus reflect an inadequate bid/offer spread. The degree to which the bid/offer spread is (or is not) included in initial and ongoing valuations should be established under a written policy and monitored.

Overrides are adjustments made by a Manager to valuations provided by independent parties under established valuation procedures (Risk Standard 10). Typically, overrides occur during periods of market dislocation when a Manager believes the independent valuation is incorrect. Although their belief at times may be well-founded, all overrides should be reported and investigated if the differences in valuation are material.

The Primary and Manager Fiduciaries should establish written policies and procedures that set forth the circumstances, notification and approval process for all overrides. These policies and procedures should be communicated to all relevant parties such as the Primary and Manager Fiduciaries, relevant oversight staff and custodians. In addition, the number and magnitude of overrides should be tracked and reviewed by the Primary Fiduciary on an ongoing basis to confirm that all material adjustments have been investigated and that practices are consistent with the override policy.

Risk Standard 12: Risk measurement and risk/return attribution analysis

The Primary and Manager Fiduciaries should regularly measure relevant risks and quantify the key drivers of risk and return.

The Primary and Manager Fiduciaries should measure the total risk in the overall portfolio, individual portfolio and each instrument. Then, they should perform attribution analyses to determine the various risks and returns posed by each instrument or portfolio.

Value-at-risk (VaR) is one widely used method for creating a common unit of measurement for risk. It is the maximum dollar (or other currency) amount that a position or portfolio is expected to lose within a specified period of time given a specified probability. There are a number of approaches to computing VaR. The results are quite sensitive to the assumptions made and model used, and both should be understood (Risk Standard 16). One common VaR method for which data is widely available is a parametric approach using historical volatility and correlations between assets.

Other risk measures commonly used include duration, beta, standard deviation, semi-variance, tracking error and drawdown size. It is important to verify that the risk measure chosen captures the relevant risks (Risk Standard 7).

Institutional investors and institutional investment managers also should perform both risk and return attribution analyses of all portfolios along all relevant dimensions. A return attribution analysis looks at historical performance of a portfolio to determine the key drivers of returns. A risk attribution analysis looks at the key sources of risk and the volatility of returns in the current or anticipated portfolio to determine the key drivers of risk. For example, a risk attribution analysis of a U.S. bond portfolio might quantify duration, yield curve, convexity and sector risk in absolute terms or relative to a benchmark. A risk attribution analysis of a U.S. equity portfolio might use a risk factor model to quantify the various sources of absolute and benchmark-relative risk.

The Primary and Manager Fiduciaries also can use risk attribution analysis to monitor whether a Manager is adhering to its stated strategy (Risk Standard 18) and to measure aggregate factor risks from multiple Managers or portfolios. Individual portfolio managers can use risk attribution analysis to make sure they are not taking more of a given risk than their limits allow and to ensure broad diversification of risk.

Risk Standard 13: Risk-adjusted return measures

Risk-adjusted returns should be measured at the aggregate and individual portfolio level to gain a true measure of relative performance.

Investors should compare all Managers on a risk-adjusted basis. By taking into account both the risk and return sides of the equation, risk-adjusted return measures enable investors to evaluate better the relative performance of two managers. Risk-adjusted measures also highlight instances in which a manager's outperformance is due to incurring misunderstood, mispriced, unintended or undisclosed risks.

Consider, for example, two portfolios that each returned 200 basis points over the same bond index for a given year. The first portfolio manager invested in U.S. Treasury securities; the second purchased distressed debt in emerging markets. A risk-adjusted measure applied to the 200 basis points of outperformance of each one would adjust for the differing market volatility, credit risk and foreign exchange risk between the two portfolios and show that the first portfolio may be more desirable on a risk-adjusted return basis.

Risk-adjusted return measures also permit a more meaningful comparison of a Manager's performance relative to its benchmark. If a Manager exceeds the benchmark return by 20%, but takes 30% more risk, his performance is less impressive than if he achieved the same excess return with risk equal to or less than the benchmark.

Common measures of risk-adjusted returns include the information ratio, Sharpe Ratio, Treynor Measure and Sortino Ratio. It is important to verify that the risk-adjusted measure chosen captures the relevant risks (Risk Standard 7).

Risk Standard 14: Stress testing

All investors, internal managers and external managers should test the likely impact of various market conditions or other events on the value of an instrument, portfolio or strategy by performing stress tests. These are typically simulations, which may be performed on a scenario, historical, simulation or random sampling basis (Monte Carlo analysis).

Strategies that pose little risk under normal market conditions may fall apart when the abnormal occurs. Alternately, strategies that hold up under large market moves (stock market crashes, huge interest rate swings) may fall apart under more subtle changes. Investors and managers should test both the impacts of large market moves and of combinations of small market moves to identify those that are likely to affect the portfolio. For example, a relative value trade that involves selling the 10-year Treasury and buying the 5-year is much more sensitive to the spread between 10-year and 5-year rates than to equal increases or decreases in both rates (parallel shifts of the yield curve). Other relevant stress tests include how risk and return change under the use of different assumptions or models (Risk Standard 16). Emphasis should be placed on stress testing the key risk factors identified in Risk Standard 7. Events that would breach risk limits, strategy goals, liability targets or asset allocation ranges should be monitored and acted upon.

Stress tests should be performed at least quarterly and whenever material events occur at the aggregate fund and manager portfolio level, incorporating asset/liability issues as relevant. "Material events" include significant changes in the market as well as in the strategy or composition of a Manager's portfolio or a change in Managers. The Primary Fiduciary should ensure that a consistent, well-defined process is used by all Managers as well as at the portfolio level. Assumptions embedded in the stress testing process should also be stress tested. The stress testing process should also be back tested to see whether the process would have forecasted accurately past outperformance, past underperformance and performance under past market shocks (Risk Standard 15).

Note that the same stress test on a portfolio with stable composition may reveal little impact at one time and great impact at another without any change in market conditions. This is most often true of options-based and mortgage portfolios, whose risk characteristics change over time.

Finally, stress tests should take into account all types of leverage and related cash flows, including such items as:

Loans (reverse repurchase agreements)
Instruments that control leveraged market positions (options)
Instruments with internal leverage (structured notes with embedded leverage or high-Beta stocks)
Initial and variation margin requirements (exchange-traded or over-the-counter futures, forwards or options)

Risk Standard 15: Back testing

Risk and return forecasts and models should be back tested at least quarterly and whenever material events occur to assess their reliability.

Institutional investors and Managers should back test all models and forecasts of expected risk, return and correlations for instruments, asset classes and strategies. Back tests involve evaluating how a strategy, instrument or model actually performed for a given period versus what was predicted by a risk or return forecast. Comparing back testing results to actual experience also helps to evaluate the robustness of an estimate for a strategy, instrument or model. This provides a useful framework for assessing the strategy's, instrument's or model's value as an investment or risk management tool.

Typical questions a back test helps to answer include:

How would the strategy, instrument, risk forecast or model have performed under certain past stressful market conditions (such as the Mexican peso devaluation, the stock market crash, the European currency crisis, the Federal Reserve rate hikes during 1994 or the 1973 oil shock)?
Was past performance or risk due to the presence of a sustained bull or bear market condition or other market condition (steep yield curve, strong technology sector, large cap performance)?
What economic conditions provide the best environment for the strategy, instrument or model?
When has the strategy, instrument or model not performed well?

Back tests should also be performed in the overall portfolio or fund context. This may help the investor to assess the risk of multiple strategies failing under similar market or economic conditions.

Risk Standard 16: Assessing model risk

Dependence on models and assumptions for valuation, risk measurement and risk management should be evaluated and monitored.

Virtually all market participants are exposed to mark-to-model risk. The degree of exposure depends upon the type and concentration of less-readily priced and non-readily priced instruments (Risk Standard 10) in individual and aggregate portfolios and on the type of valuation, risk and strategy models used. The Primary and Manager Fiduciaries should evaluate and monitor both the extent to which their management of risk and return changes under different models and assumptions that are widely used in the market place, as well as the degree to which the percentage of assets changes from readily priced to other valuation categories (see Risk Standard 10) from period to period. Dependence on assumptions within models should be stress tested (see Risk Standard 14). Important dimensions of model risk to analyze include:

Data integrity (e.g. curve construction, differing sources of data, representativeness and statistical significance of samples, time of day data is extracted, data availability and errors)
Definition and certainty of future cash flows (formula-driven cash flows or flows that depend on an option)
Formula, algorithm or other mathematical engine (Black-Scholes versus Hull & White for options valuation)
Liquidity assumptions (length of time to liquidate and bid/ask spreads)
Model parameter selection (selection of spreads, discount rates, scenario and stress test parameters, probability intervals, time horizon, correlation assumptions)