Creating a trading system from scratch

[ncube]

I don't go with targets. It creates too many complications and difficult to replicate. Increasing the stop loss would invariably increase my profits but gets me a major drawdown for doomsday scenarios. In fact, keeping a target reduces the profitability and optimizes too much (so I stuck with generalization for better performance)
I only place limit orders at open price less a small percentage (to make sure that price has fallen)
This gets me the exact price, although I might lose some orders.
Today is a break even day; but my new live strategy resulted in a loss (I believe this is due to the extreme L&T FH, IDBI case)

I am also updating the performance on my website; may take a day or two so that you can check it.

Yes, keeping target is not beneficial. Also its not good to optimize the parameters to backtest. A limit order with a slightly lower open price will definitely add alfa, but if one need to ensure all the trades are executed its beneficial if all trades are executed close to open price just to replicate it with backtest.

So our results match that having targets does not add much value. Just for fun let me run the backtest with SL varying between 1 to 5% and see how it performs..

 

[UberMachine]

Yes, keeping target is not beneficial. Also its not good to optimize the parameters to backtest. A limit order with a slightly lower open price will definitely add alfa, but if one need to ensure all the trades are executed its beneficial if all trades are executed close to open price just to replicate it with backtest.

So our results match that having targets does not add much value. Just for fun let me run the backtest with SL varying between 1 to 5% and see how it performs..

Sure. The greater the stop loss, the greater the overall profit (upto 7%).
All trades are not executed. In fact, there is a case open=high which doesn't get executed 90% of the time and it takes away 40% of my total profits.
Removing it from results also gives me decent returns, provided I am brutally disciplined with execution.
One of my objectives is to create an extremly simple library based on the above framework.
Its under development here and I believe it would add a statistical edge to the existing frameworks.

 

[ncube]

Sure. The greater the stop loss, the greater the overall profit (upto 7%).
All trades are not executed. In fact, there is a case open=high which doesn't get executed 90% of the time and it takes away 40% of my total profits.
Removing it from results also gives me decent returns, provided I am brutally disciplined with execution.
One of my objectives is to create an extremly simple library based on the above framework.
Its under development here and I believe it would add a statistical edge to the existing frameworks.

Just ran the backtest with varying SL values, and the key observation is that 3% SL is ideal for NIFTY50 stocks. Total returns at various SL values are as follows:
1% --> 598.77 (This would have resulted today in a small profit of about 0.25% but it is not significant)
2% --> 690.42
3% --> 734.72
4% --> 695.39
5% --> 687.91

Yes, avoiding open=high is a good factor. 40% of total profits is a big chunk. I will try adding this filter in the backtest when I get some time.
Also your library fastbt looks interesting, will surely explore it.

 

[vinit_7801]

Sure. The greater the stop loss, the greater the overall profit (upto 7%).
All trades are not executed. In fact, there is a case open=high which doesn't get executed 90% of the time and it takes away 40% of my total profits.
Removing it from results also gives me decent returns, provided I am brutally disciplined with execution.
One of my objectives is to create an extremly simple library based on the above framework.
Its under development here and I believe it would add a statistical edge to the existing frameworks.

Try with num of stocks as well for me 2 gives the best mean/std return

 

[UberMachine]

Try with num of stocks as well for me 2 gives the best mean/std return

Once I release the open source library, you can run these sort of simulations within this framework easily.
We would soon be moving this notebook to script for better effectiveness so that each can build their own strategies.
For non-programmers, I would provide an excel template so that you can fill in the necessary details and the backtest does the rest.

 

[vinit_7801]

Damn i m in the middle of creation of mature rule based automated trading system.....any trader's dream....bit emotional...lol

 

[vinit_7801]

Once I release the open source library, you can run these sort of simulations within this framework easily.
We would soon be moving this notebook to script for better effectiveness so that each can build their own strategies.
For non-programmers, I would provide an excel template so that you can fill in the necessary details and the backtest does the rest.

can you add monte carlo simulation to your backtest your notebook if possible it will make system more mature and add another layer of systematic aproach.....
here is the code

#Set the number of iterations to 10000 and define an array to hold the simulation results; initially set to all zeros
num_iterations = 1000000
simulation_res = np.zeros((4+len(stock)-1,num_iterations))



for i in range(num_iterations):
#Select random weights and normalize to set the sum to 1
weights = np.array(np.random.random(6))
weights /= np.sum(weights)
#Calculate the return and standard deviation for every step
portfolio_return = np.sum(mean_returns * weights)
portfolio_std_dev = np.sqrt(np.dot(weights.T,np.dot(cov_matrix, weights)))
#Store all the results in a defined array
simulation_res[0,i] = portfolio_return
simulation_res[1,i] = portfolio_std_dev
#Calculate Sharpe ratio and store it in the array
simulation_res[2,i] = simulation_res[0,i] / simulation_res[1,i]
#Save the weights in the array
for j in range(len(weights)):
simulation_res[j+3,i] = weights[j]



sim_frame = pd.DataFrame(simulation_res.T,columns=['ret','stdev','sharpe',stock[0],stock[1],stock[2],stock[3],stock[4],
stock[5]])
print (sim_frame.head (5))
print (sim_frame.tail (5))


#Spot the position of the portfolio with highest Sharpe Ratio
max_sharpe = sim_frame.iloc[sim_frame['sharpe'].idxmax()]
#Spot the position of the portfolio with minimum Standard Deviation
min_std = sim_frame.iloc[sim_frame['stdev'].idxmin()]
print ("The portfolio for max Sharpe Ratio:\n", max_sharpe.round(3)*100)
print ("The portfolio for min risk:\n", min_std.round(3)*100)

 

[UberMachine]

can you add monte carlo simulation to your backtest your notebook if possible it will make system more mature and add another layer of systematic aproach.....
here is the code

#Set the number of iterations to 10000 and define an array to hold the simulation results; initially set to all zeros
num_iterations = 1000000
simulation_res = np.zeros((4+len(stock)-1,num_iterations))



for i in range(num_iterations):
#Select random weights and normalize to set the sum to 1
weights = np.array(np.random.random(6))
weights /= np.sum(weights)
#Calculate the return and standard deviation for every step
portfolio_return = np.sum(mean_returns * weights)
portfolio_std_dev = np.sqrt(np.dot(weights.T,np.dot(cov_matrix, weights)))
#Store all the results in a defined array
simulation_res[0,i] = portfolio_return
simulation_res[1,i] = portfolio_std_dev
#Calculate Sharpe ratio and store it in the array
simulation_res[2,i] = simulation_res[0,i] / simulation_res[1,i]
#Save the weights in the array
for j in range(len(weights)):
simulation_res[j+3,i] = weights[j]



sim_frame = pd.DataFrame(simulation_res.T,columns=['ret','stdev','sharpe',stock[0],stock[1],stock[2],stock[3],stock[4],
stock[5]])
print (sim_frame.head (5))
print (sim_frame.tail (5))


#Spot the position of the portfolio with highest Sharpe Ratio
max_sharpe = sim_frame.iloc[sim_frame['sharpe'].idxmax()]
#Spot the position of the portfolio with minimum Standard Deviation
min_std = sim_frame.iloc[sim_frame['stdev'].idxmin()]
print ("The portfolio for max Sharpe Ratio:\n", max_sharpe.round(3)*100)
print ("The portfolio for min risk:\n", min_std.round(3)*100)

A couple of things

1. Your monte carlo simulation looks a bit more for our simple model. Any way, hold it so that I can cross check with my own
2. Simulation, performance evaluation and other things should go in a separate file (something similar to utils.py). Including everything in our notebook would make it messy when we go forward

Regarding monte carlo simulation, this is how I would do (it would look very naive) and I would like to have comments on this

# Assume trades is the dataframe that holds all my data

simulations = []
for i in range(1000):
    n = int(len(trades)/10) #Take 10 percent of the samples
    sim = trades.sample(n)
    simulations.append(n)
    
profit_array = []
for s in simulation:
    profit_array.append(s['profit'].sum())
    
import matplotlib.pyplot as plt
plt.plot(profit_array)

I just take across the trades (instead of a particular day) to measure my effective returns.
In fact I think, adding bootstrap=true would generate this automatically for you in pyfolio. I haven't gone through the entire documentation so far

 

[vinit_7801]

A couple of things

1. Your monte carlo simulation looks a bit more for our simple model. Any way, hold it so that I can cross check with my own
2. Simulation, performance evaluation and other things should go in a separate file (something similar to utils.py). Including everything in our notebook would make it messy when we go forward

Regarding monte carlo simulation, this is how I would do (it would look very naive) and I would like to have comments on this

# Assume trades is the dataframe that holds all my data

simulations = []
for i in range(1000):
    n = int(len(trades)/10) #Take 10 percent of the samples
    sim = trades.sample(n)
    simulations.append(n)
   
profit_array = []
for s in simulation:
    profit_array.append(s['profit'].sum())
   
import matplotlib.pyplot as plt
plt.plot(profit_array)

I just take across the trades (instead of a particular day) to measure my effective returns.
In fact I think, adding bootstrap=true would generate this automatically for you in pyfolio. I haven't gone through the entire documentation so far

Just one question why care about profits main thing is decreasing drawdowns and increasing sharpe ratio na profits will come....n second point is about weights....why give equal weights to all...by doing 1000s of simulation we can optimize the weights and sharpe ratio there by increasing the chances of profitability....so shouldn't monte carlo simulation be concentrated toward maximizing sharpe ratio then maximizing profits...???

 

[UberMachine]

Just one question why care about profits main thing is decreasing drawdowns and increasing sharpe ratio na profits will come....n second point is about weights....why give equal weights to all...by doing 1000s of simulation we can optimize the weights and sharpe ratio there by increasing the chances of profitability....so shouldn't monte carlo simulation be concentrated toward maximizing sharpe ratio then maximizing profits...???

Nice questions.

We are optimizing weights based on historical data; this would invariably need to overfitting. It won't generalize well. Simply speaking, we expect history to repeat itself exactly, which would backfire.
Then why would hedge/mutual funds optimize. Because they can hold on to their assets and rebalance them. Generally in financial time series data, the lower the time frequency, the more random the data is and vice versa. Since we are dealing with daily data, its better to not optimize too much.

But I agree with improvements in Sharpe ratio. In case of adjusting weights, we can either adjust it with a Bayesian perspective or fit some linear line to intraday return. Now, this is a big topic that requires some machine learning which I haven't covered in depth so far (I just don't know how the system would react).

You can try this out but do proper cross validation and time invariant testing (testing 2018 data on 2017) to get at the right parameters. You should also test for beta and correlation to make your system even robust. But that's too much code to test and replicate in practice.