Introduction to Model Analysis

Here you will find different ways that can help you analyze your models and portfolios. Navigate to the bottom for a list of techniques that can assist you in understanding and improving your models.

Stock Selection:

The most important thing in creating a performant model is the Quantile Spread. Generally speaking, if you can take Q1 minus Q5 and it is above 10% - you have created a good model:

If you have achieved that spread, then the core of the model (the stock rankings) is good.  Most problems you have with the return profile of the model can be explained by portfolio construction. 

If you don't have a good quantile spread, then you can go to the Explainability tab of the model and look at the features to see if the importance derived by the machine makes sense to you.  If you have a lot of features with low importance (< 1%) you might try duplicating this model, removing those features and adding different features.  If some features are too important (i.e. they are >20% importance but don't make sense to you that they rank so highly) you can also try removing them to see if that helps.

Portfolio Construction:

If you have gotten to the point that the quantile spread is good, but you are unsatisfied with the performance of the portfolio - then it is likely a problem with Portfolio Settings.  These can be adjusted without creating an entirely new model through the Portfolio Settings (left hand side or the gear beside the portfolio name) options. 

There are several things to check in there:

• Click the Allocation button (left hand side) and see if your Long / Short allocation matches your preferences.  If it does not - this is likely a problem in the interaction of some of your Portfolio Settings.  The most common ones are:
  • Too few maximum number of stocks
  • Too low a maximum short / long position
  • Having trade by % of rankings set too low
  • All of which can be found on the Securities tab of the Portfolio Settings. 

This portfolio cannot reach 100% allocation, because maximum 20 stocks * 4% maximum = 80% maximum weighting.

• If the Allocation is correct and you still are not getting the alpha d/ return that you expect, check the Benchmark tab of your Portfolio Settings and ensure you are using the correct benchmark for your stock universe. This will not affect the return of your model, but an incorrect benchmark can result in a chart that shows poor performance despite good performance for your stock universe (i.e. if you were to use QQQ as a benchmark for a financials universe).

• If the results still do not match your expectations, try some of the basic Portfolio Settings and it can help to diagnose the problem. By looking at different Portfolio Settings you can start to see if the portfolio does well long / short vs long only vs market cap constrained. If it does well in only certain situations then there’s a chance that there is bias in the universe or benchmark. You can create new portfolios on the same model by clicking the + icon beside portfolios (the gear icon will update the existing portfolio):

This allows you to run multiple different Portfolio Settings at the same time.  Below are some common and successful portfolio construction options.

The reason to leave portfolio optimizers off for these tests is to try to isolate the problem before complicating it with portfolio optimizers.  If you have success with any of the above Portfolio Settings, feel free to then try them with optimizers turned on. 

For more detailed information on specific features or metrics, please refer to the list below.

   Model Optimization Techniques
   - Variables
   - Investment horizon

  Portfolio Evaluation
  - Annualized Alpha
  - Sharpe Ratio
  - Drawdowns
  - Allocation
  - Portfolio settings

  Portfolio Optimization Techniques
  - Optimization
  - Signal Optimization
  - Securities settings
  - Neutralization

If you are still having problems with your model please reach out to your customer success rep at Boosted.ai.  We will work with you to understand the problem.