Effects of not keeping a Stop Loss

[uasish]

I dont have to go as far as CV. The Buddha (not old as in buddha, but the enlightened one ) would ask you a simple question - are you simulating to forecast (as in patriot missiles hitting) or doing backtest? Backtest involves data and events that have already occurred. If you ise the same parameters 100s of times, you'll get the same results. Anyone can now say how many patriots had hit the targets...and the answers will be the same. So if you had obtained 78% using correct parameters-algorithm combination, it *will remain* 78% unless you change one of them.

If you are talking of forecasting, noone will do it accurately as conditions keep changing. You can oly talk about scenarios.

Anyways, going back to the bodhi( forgot, the banyan tree, will be named bodhi tree after my enlightenment!!) tree now.

Yes you are absolutely right & dot on the point.My communication was wrong ,it should have been ,"i cant rely on these back-test for Forecast".
Once again Abhijeet thanks for pointing out the mistake & true heartfull wishes to succed in your quest.

 

[hmp]

Ratan Sir,
Will U Pl. Give Us Little Idea About Ur Complex Martingales System?
Regards
Hmp

 

[CreditViolet]

Jesse,

In EoD data we can test even on 10yrs data ,on RT data (if it is 1/5 mins) what is the Max input data you use.
Yes you are right that Forward test ( or actual mkt test) may vary in performance with back-test.

Asish

Well it all depends

If the edge you found is on daily data, I would like to validate the hypothesis on a variety of markets and I mean different asset classes and not just stocks or indices that are highly correlated.

If its on intraday data, it gets much easier because of the availability of huge amounts of data, here OOS testing should be done preferably on a different calendar period to see if the edge persists.

Ofcourse all this assume that you have partitioned your data appropriately.

Basically, the idea is to get as much trade sample as possible in different conditions.For any system, No. of Trades - No. of Rules or Parameters should be greater than atleast 30. So it again boils down to degrees of freedom measure.

And probably the most important thing among all these is to have an approach to measure the deviation of actual performance from simulated performance which I think most people gloss over.

Also the idea to test on recent market leads to the multiple hypothesis problem if the hypothesis was based on the recent data itself. One is already biased towards the acceptance of test results.

 

[CreditViolet]

Will U Pl. Give Us Little Idea About Ur Complex Martingales System?

Dont hold your breath

 

[uasish]

Jesse,

Thks,correlation of different asset class ,OSS data,right mix in partitioning with min. representation > 30,finally the actual vis-a-vis performance deviation,a module to work with.

Asish

 

[CreditViolet]

Jesse,

Thks,correlation of different asset class ,OSS data,right mix in partitioning with min. representation > 30,finally the actual vis-a-vis performance deviation,a module to work with.

Asish

Glad to hear that you found the post useful.Chk out my recent blog post where I updated the Strategy Development Process.Its Version 2 of the map I posted in the Experiments in TA Thread. I am currently working on Version 3 that would probably outline the complete process including everything mentioned/published on Trading & Trading Systems including stuff from the recent Aronson book.

Cheers

 

[uasish]

As usual very difficult to catch up with you in conceptualisation,always ahead ,a benchmark for me.

 

[oxusmorouz]

Basically, the idea is to get as much trade sample as possible in different conditions.For any system, No. of Trades - No. of Rules or Parameters should be greater than atleast 30. So it again boils down to degrees of freedom measure.

Hi CV,
Rather than going on a per trade basis, wouldn't it be a better idea to go on detrended "monthly/weekly/daily returns" for EoD data (detrended by logging the quotient of return on system and return on benchmark, rather than logging the quotient of return on system with previous period return, to get a sense of out performance)? I ask this because not all trades are of equal duration, some may test 100 bars and some may get stopped in 2bars, making the systems which stay for a long period of time vulnerable to small amount of data, and thus resulting in large standard errors. Also, 10 years of data would mean 540 weeks of data, or a sample size of 540, which does not depend on how long the system stays in the market.

And probably the most important thing among all these is to have an approach to measure the deviation of actual performance from simulated performance which I think most people gloss over.

How CV? Except using an out of the box sample (which by itself is prone to n number of biases) or by testing it over a reasonable period of time in real life (which takes ages on daily data), is there any other way this can be done?

 

[CreditViolet]

Hi CV,
Rather than going on a per trade basis, wouldn't it be a better idea to go on detrended "monthly/weekly/daily returns" for EoD data (detrended by logging the quotient of return on system and return on benchmark, rather than logging the quotient of return on system with previous period return, to get a sense of out performance)? I ask this because not all trades are of equal duration, some may test 100 bars and some may get stopped in 2bars, making the systems which stay for a long period of time vulnerable to small amount of data, and thus resulting in large standard errors. Also, 10 years of data would mean 540 weeks of data, or a sample size of 540, which does not depend on how long the system stays in the market.

Oxy, I am a bit confused with the detrending system returns, benchmark part.One problem with the mean you describe is the compounding issue, correct?
Also isnt sample size the no. of trades you generate and not the testing data?

Am jst a bit confused, maybe too much work today, will try to get back to your post later.

How CV? Except using an out of the box sample (which by itself is prone to n number of biases) or by testing it over a reasonable period of time in real life (which takes ages on daily data), is there any other way this can be done?

Using basic Quality Control procedures, a system trader friend of mine advocated using them and its been really helpful.Send me a PM, I will send you some resources.

 

[uasish]

Ajay,

Plz correct me if i have any wrong understanding.
To Abhijeet's & my query of 'Period for back testing' Jesse has given an input & how he felt it is to be Moduled.
You have suggested a method ;when 't' (trade duration) of Benchmark performance vis-a-vis our system is inequal how to bring down the anamoly in the 'Scale of Comparision' & also expressed a doubt that as Out of sample data may also contain 'n' no of biases hence you wanted to know from Jesse how to tackle this issue; as Live test of EoD systems will be Time consuming.
Here Jesse has suggested QC of Input for the actual trades taken by system.

Asish