Model Analysis Tips

Stock Selection:

The most important thing in creating a performant model is the Quantile Spread. Generally speaking, if you can take Q1 minus Q5 and it is above 10% - you have created a good model:

return-quantiles

If you have achieved that spread, then the core of the model (the stock rankings) is good.  Most problems you have with the return profile of the model can be explained by portfolio construction. 

If you don't have a good quantile spread, then you can go to the Explainability tab of the model and look at the features to see if the importance derived by the machine makes sense to you.  If you have a lot of features with low importance (< 1%) you might try duplicating this model, removing those features and adding different features.  If some features are too important (i.e. they are >20% importance but don't make sense to you that they rank so highly) you can also try removing them to see if that helps.

 

Portfolio Construction:

If you have gotten to the point that the quantile spread is good, but you are unsatisfied with the performance of the portfolio - then it is likely a problem with Portfolio Settings.  These can be adjusted without creating an entirely new model through the Portfolio Settings (left hand side or the gear beside the portfolio name) options. 

portfolio-settings-gear.png

There are several things to check in there:

  • Click the Allocation button (left hand side) and see if your Long / Short allocation matches your preferences.  If it does not - this is likely a problem in the interaction of some of your Portfolio Settings.  The most common ones are:
    • Too few maximum number of stocks
    • Too low a maximum short / long position
    • Having trade by % of rankings set too low
    • All of which can be found on the Securities tab of the Portfolio Settings. 

100-percent-portfolio

This portfolio cannot reach 100% allocation, because maximum 20 stocks * 4% maximum = 80% maximum weighting.

  • If the Allocation is correct and you still are not getting the alpha / return that you expect, check the Benchmark tab of your Portfolio Settings and ensure you are using the correct benchmark for your stock universe. This will not affect the return of your model, but an incorrect benchmark can result in a chart that shows poor performance despite good performance for your stock universe (i.e. if you were to use QQQ as a benchmark for a financials universe).

benchmark-spy

  • If the results still do not match your expectations, try some of the basic Portfolio Settings and it can help to diagnose the problem. By looking at different Portfolio Settings you can start to see if the portfolio does well long / short vs long only vs market cap constrained. If it does well in only certain situations then there’s a chance that there is bias in the universe or benchmark. You can create new portfolios on the same model by clicking the icon beside portfolios (the gear icon will update the existing portfolio):

multiple-portfolios

This allows you to run multiple different Portfolio Settings at the same time.  Below are some common and successful portfolio construction options.

 

Long Only

Long Only MCAP Constrained

Long / Short

SECURITIES TAB

 

 

 

Trade by % of Rankings

Off

On

Off

Top / Bottom % to Trade

N/A

100%

N/A

Min Number of Stocks

5

5

5

 

 

 

 

Max Number of Stocks

100

# of stocks in benchmark

100 (this means 100 stocks long and 100 stocks short)

Max Short Position

Any

Any

5%

Max Long Position

10%

10%

10%

Percent of Portfolio Long

100%

100%

100%

TRADING TAB

 

 

 

Initial Weighting

Alpha Weight

Alpha Weight

Alpha Weight

Sector Neutral

Off

Off

Off

Constrain Market Cap

Off

On

Off

Maximum Market Cap Variation

N/A

1%

N/A

OPTIMIZATION TAB

 

 

 

Optimizer Type

No Optimization

No Optimization

No Optimization

Dividend Yield

Off

Off

Off

Tracking Error

Off

Off

Off

COMBINATION TAB

 

 

 

Combine Models

N/A

N/A

N/A

BENCHMARK

 

 

 

Benchmark

Correct Universe Benchmark

Correct Universe Benchmark

Correct Universe Benchmark

Weight

1

1

1

Adjust Benchmark for Net Exposure

On

On

On

The reason to leave portfolio optimizers off for these tests is to try to isolate the problem before complicating it with portfolio optimizers.  If you have success with any of the above Portfolio Settings, feel free to then try them with optimizers turned on. 

 If you are still having problems with your model please reach out to your customer success rep at Boosted.ai.  We will work with you to understand the problem.