This entry was posted on Wednesday, October 3rd, 2007 at 9:28 pm and is filed under Education, system trading. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
What are the parameters and qualifications that I look for when evaluating a system? Having been a serious system tester and designer for some time I thought I might pass along some of the things I am looking for. Just some very basic concepts for now. The Collective2.com site is a great place to see real trading in action. Contrary to all the BS system sellers out there touting 150% returns with no risk the markets are quite efficient. Efficient in that any one set of rules applied consistently will have periods of out performance and periods of under performance. Holy Grails do not exist in trading nor on the Collective2 site.
No system is infallible in all market conditions. Having tested virtual every sort of entry and exit imaginable I believe I can say that fairly certainly. Can anyone point to a trading system with verifiable results that was perfect? That called every top and bottom perfectly? Of course not. So when evaluating a system having unrealistic expectations is going to leave you disappointed. Realism and reality are going to have to be hallmarks in your journey. I am going to try and give you some basics to use when evaluating trading results and systems.
In order to group systems in a way that makes comparision easier I always want to compare systems that have similiar holding periods. I dont want to see a trend system based on daily bars that holds an average of 100 days compared to a swing trade system that holds 5 days on average. These holding periods also tend to have some general characteristics that are similiar throught that group like drawdowns from tops and volatility.
Let us go over to the Collective site and do some system sorting on their grid. It’s this grid which helps that sorting process. One drawback with this tool is its inabilty to sort amoung multiple columns but we will have to make due for now. I am only going to look at stock systems so I only check that off. In the filter area I only want profitable systems. If I check off ’smooth equity curves” I am esentially only focusing on Sharpe Ratios above 1. Sharpe ratio is a critical component to understanding and evaluating systems. It tells us essentially the reward to variabilty ratio. How big are the wiggles compared to the returns. Two systems may have doubled your money over the course of the past year but if the variabilty/volatility is higher in one compared to the other the less volatile system will have a higher Sharpe ratio. A ratio above 1.5 in systems is rare and impressive over the long haul and above 2.0 rarer still. Anyway lets sort some systems.
I have sorted some stock systems by the average trade length - you can see they range from 4 weeks to 6.8 weeks. Now lets look at systems that have a reasonable time traded and a Sharpe ratio over 1.3. This leaves me with 3 systems that I might look a bit further into. One of the things I do not like right off is that the realism factor in each of these systems is relatively low. I like these above 75 generally. The lower rated systems tend to trade in less liquid and wider bid/ask environments. Meaning the ability to match system results could be impacted.
Of these 3 systems lets look at annualized return (underlined in red). How do I quantify these returns? I have 68.7%, 132.6% and 42.4%. Is the one thats highest best? Actually we don’t know because that system could be trading a 2 times leverage and the 42.4% on at 1 times. It would require further investigation. Also the highest in terms of Sharpe and Annual returns is a somewhat newer system that only has 22 trades so far. But one to keep an eye on.
When designing systems I look for these same sorts of parameters. Ultimately I want high returns, smooth equity curve, realism, and one that does not require a case of antacid to trade through. On my old blog I gave out picks to a swing system for one year before the market opened and did 40%, but tracking the trades was a pain so now John and I use C2 to prove ourselves over time. John, who also writes for this blog, being a former Microsoft programmer and system designer and myself being a trader and system designer made a good team for not only this blog but for our endevours in system design. We wanted to create systems the average working guy could use to enhance his portfolio performance or highlight interesting setups here on the blog.
Ok, so you now have some basic tools to sort and scan. Some of these systems are run by some very very good system designers and traders. Not all of them, but there are some really good ones I can attest to. In the coming weeks I am going to do some interviews with traders I think are very knowledgable from this site. Many of these traders I have interacted with at some level over at the Wealth-Lab site. This is the desktop software I use to do portfolio level testing of my systems.I am of the opinion that unless you have some sort of verifiable track record it is difficult for me to assign any value to your opinion. This is why I always encourage other bloggers to start a C2 system. Let us see what you can do. Or how about the systems on TV where they tell us how good they are, or the typical pitches you hear from the unscrupulous. Prove it in real time. Real position sizing. Real Emotions. That is what Collective 2 does. It allows us to sort through the BS and find quality systems. You job is to determine which won’t blow up in the end.
Have a Great Night!
Dave Johnson