15. Evaluating Model Performance

Disclaimer

Before reading this tutorial you, the reader, agree to the following. This tutorial and the code it contains are designed to be informational and educational tools only. They do not constitute investment advice. The author, Dave Topper, strongly recommends that you seek the advice of a financial services professional before making any type of investment. This model is provided as a rough approximation of future financial performance. The results presented are hypothetical and will likely not reflect the actual growth of your own investments. The author, Dave Topper, is not responsible for any human or mechanical errors or omissions. The author, Dave Topper, is not responsible for the consequences of any decisions or actions taken in reliance upon or as a direct or indirect result of the information provided by these tools.

Now we’re ready to see how our model has performed over time. The calculations necessary have already been built up in the previous section. Now we’re ready to print them out.

Modify the printStats() method in tradingModel.cpp as follows.


// Print out various stats related to the most recent run
void tradingModel::printStats()
{
    std::cout << std::endl;

    // Portfolio start / end and percent
    std::cout << "Portfolio start = " << modelPortfolio_->getStartValue() << std::endl;
    std::cout << "Portfolio end = " << modelPortfolio_->getTotalValue() << std::endl;
    double portfolioTotalPercentChange =
            tradingUtils::fix2D((modelPortfolio_->getTotalValue()/modelPortfolio_->getStartValue())-1);
    std::cout << "  pct change = " << portfolioTotalPercentChange << std::endl;

    // Portfolio max value and percent
    std::cout << "Portfolio max = " << modelPortfolio_->getMaxValue() << std::endl;
    double portfolioMaxPercentChange =
            tradingUtils::fix2D((modelPortfolio_->getMaxValue()/modelPortfolio_->getStartValue())-1);
    std::cout << "  pct change = " << portfolioMaxPercentChange << std::endl;

    // Portfolio min value and percent
    std::cout << "Portfolio min = " << modelPortfolio_->getMinValue() << std::endl;
    double portfolioMinPercentChange =
            tradingUtils::fix2D((modelPortfolio_->getMinValue()/modelPortfolio_->getStartValue())-1);
    std::cout << "  pct change = " << portfolioMinPercentChange << std::endl;

    // Trades both positive and negative
    std::cout << "Num sell trades = " << modelPortfolio_->getNumSellTrades() << std::endl;
    std::cout << "  Num positive trades = " << modelPortfolio_->getNumTradesPos();
    double posTradePercent =
            tradingUtils::fix2D(modelPortfolio_->getNumTradesPos()/static_cast<double>(modelPortfolio_->getNumSellTrades()));
    std::cout << " (" << posTradePercent << " pct)" << std::endl;
    std::cout << "  Num negative trades = " << modelPortfolio_->getNumTradesNeg();
    double negTradePercent =
            tradingUtils::fix2D(modelPortfolio_->getNumTradesNeg()/static_cast<double>(modelPortfolio_->getNumSellTrades()));
    std::cout << " (" << negTradePercent << " pct)" << std::endl;
}

That’s it. Congratulations. You have finished creating a simple crossover strategy in Qt.

Build and run the program.

You should see the following output (last 10 lines).


Portfolio start = 1000
Portfolio end = 921.36
  pct change = -0.08
Portfolio max = 1200.9
  pct change = 0.2
Portfolio min = 846.35
  pct change = -0.15
Num sell trades = 24
  Num positive trades = 9 (0.38 pct)
  Num negative trades = 15 (0.62 pct)

Using 20 day and 50 day moving averages the model lost 8% over the last 10 years. By comparison, IBM lost 14.2% over the same time period. So we did ok but not necessarily great. We gained as much as 20% at one point. Although we lost as much as 15%. Only about 38% of our trades were profitable.

Writing successful models is not easy and beating the market consistently is known to be quite difficult.

The simplest thing to try next is to run the model with different moving average lengths. Setting the shorter moving average to 10 days instead of 20 already changes things significantly.

Here are the performance statistics generated by using 10 day and 50 day moving averages: ./algoTradingModel -df IBM.csv -mAvgL1 10 -mAvgL2 50.


Portfolio start = 1000
Portfolio end = 1112.07
  pct change = 0.11
Portfolio max = 1251.93
  pct change = 0.25
Portfolio min = 885.83
  pct change = -0.11
Num sell trades = 31
  Num positive trades = 15 (0.48 pct)
  Num negative trades = 16 (0.52 pct)

Trying to find the optimal parameters for a given model is much more complicated. I hope to write future tutorials that explain how to create a distributed system to iterate through various model parameters to find optimal ones. The code we’ve just built can serve as a decent starting point for creating just such a system.

As mentioned previously, there are also some additional conditions we might consider adding to this model.

  1. Only BUY when one or both of the moving averages are moving up. This would be done easily by adding an isMovingUp() method to the moving average class which simply checks if the current value is greater than the previous value.
  2. Only SELL when one or both of the moving averages are moving down. This would be done easily by adding an isMovingDown() method to the moving average class which simply checks if the current value is less than the previous value.
  3. Use more than two moving averages.
  4. Only BUY or SELL when the respective moving averages are a given percentage higher or lower than each other.
  5. Only BUY or SELL when the underlying price is also above or below a given moving average.
  6. SELL once you have made a certain percentage profit.

There are many additional conditions you might consider adding. With the code you’ve just built, it should be relatively easy to do so and I encourage you to try.

In the next tutorial, I hope to show how to view a model’s performance graphically using some of the various Qt chart classes. You can take a look at some of these yourself right now by selecting the “Welcome” icon on the left hand side of qtcreator.

Then select Examples.

Then type “chart” into the search query. You should see several different Qt chart classes.

In order to chart a model’s performance it will be necessary to create a data object that stores metrics alongside dates. Many languages refer to this as a time series class.

I will conclude with that as something to consider and hopefully to look forward too in the next tutorial.

Click here to download the complete source code used in this tutorial.

< previous