EOD Download/ Convert from NSE and BSE

[pkjha30]

Pankaj,
I have been enquiring with my friends as whether they have any idea about the time of uploading MCX EOD Bhavcopy. Unfortunately none of them any idea with regard to this. However, one of them told me that he has somewhere read that MCX is providing the EOD Bhavcopy on the next working day in between 8.00 A. M. and 8.30 A. M. He said he is not sure. He stressed the point next working day. That means 22/01/2008 EOD Bhavcopy will be available on 23/01/2008. He also told me that EOD file of Saturday will be available on Monday morning. We will have check this out.

My suggestion is can the programme be so modified that it downloads previous day's data? For example if today is 22/01/2008, can the programme be modified to download the data of 21/01/2008? That will not make huge difference because, trading time is up to 11.00 P. M. So after one hour of closure of trade for the day, it will be the next day. So even if MCX provides EOD bhavcopy before 11.59 P. M., one can download it from 00.00 hours.

I sincerely hope this helps.
Thanks and regards
R. S. Iyer

P. S.: If any member has got any information on this, kindly post it here. It is urgent.

Mr Iyer

Yes it saves some time, as I am now clear that data for 22nd Jan will be available only on 23rd January.I will check by what time it becomes available.
No need to worry about date logic I will work it out .
Here is the sample data downloaded from MCX. Now I notice that Symbols are repeated with different Expiry dates mentioned. How do you want to format the output file.
Date,Commodity Symbol,Contract/Expiry Month,Open(Rs.),High(Rs.),Low(Rs.),Close(Rs.),PCP(Rs.),Volume(In 000's),Value(In Lakhs),OI(In Lots)
21/Jan/2008,ALUMINIUM,31JAN2008,95.10,95.40,94.60,94.70,95.10,362.00 KGS ,343.42,759
21/Jan/2008,ALUMINIUM,29FEB2008,96.90,97.00,95.85,96.15,96.95,1040.00 KGS ,1004.17,245
21/Jan/2008,ALUMINIUM,31MAR2008,97.15,98.40,97.00,97.15,98.00,1040.00 KGS ,1011.35,200
21/Jan/2008,ARECAJHAJI,15FEB2008,0,0,0,9764.00,9764.00,0.00 KGS ,0,0
21/Jan/2008,ARECAJHAJI,15MAR2008,0,0,0,9788.00,9788.00,0.00 KGS ,0,0
21/Jan/2008,ARECAJHAJI,15APR2008,0,0,0,9800.00,9800.00,0.00 KGS ,0,0
21/Jan/2008,ARECAJHAJI,15MAY2008,0,0,0,9000.00,9000.00,0.00 KGS ,0,0
21/Jan/2008,BRCRUDEOIL,08FEB2008,0,0,0,3678.00,3678.00,0.00 BBL ,0,0
21/Jan/2008,BRCRUDEOIL,10MAR2008,0,0,0,3617.00,3617.00,0.00 BBL ,0,0
21/Jan/2008,BRCRUDEOIL,10APR2008,0,0,0,3640.00,3640.00,0.00 BBL ,0,0
21/Jan/2008,CARDAMOM,15FEB2008,715.00,716.00,704.50,706.00,713.50,287.00 KGS ,2038.66,406
21/Jan/2008,CARDAMOM,15MAR2008,713.00,717.00,708.00,711.00,717.00,45.00 KGS ,320.58,123
21/Jan/2008,CARDAMOM,15APR2008,711.00,716.00,708.00,710.50,716.00,10.00 KGS ,71.13,52
21/Jan/2008,CARDAMOM,15MAY2008,715.00,735.00,715.00,725.00,725.00,1.00 KGS ,7.25,2
21/Jan/2008,CASHEWKERN,15FEB2008,0,0,0,240.00,240.00,0.00 KGS ,0,0
21/Jan/2008,CASHEWKERN,15MAR2008,0,0,0,245.00,245.00,0.00 KGS ,0,0
21/Jan/2008,CASTOROIL,15FEB2008,0,0,0,470.90,470.90,0.00 KGS ,0,0
21/Jan/2008,CASTOROIL,15APR2008,0,0,0,474.40,474.40,0.00 KGS ,0,0
21/Jan/2008,CFI,15DEC2008,1280.00,1333.00,1280.00,1330.00,1281.50,29.80 MT ,392.12,60
21/Jan/2008,CFI,15DEC2009,1361.50,1361.50,1256.50,1309.00,1309.00,0.40 MT ,5.24,0
21/Jan/2008,CFI,15DEC2010,0,0,0,1340.50,1340.50,0.00 MT ,0,0
21/Jan/2008,CFI,15DEC2011,0,0,0,1377.00,1377.00,0.00 MT ,0,0
21/Jan/2008,CFI,14DEC2012,0,0,0,1429.50,1429.50,0.00 MT ,0,0
21/Jan/2008,CHANA,20FEB2008,2205.00,2224.00,2204.00,2214.00,2210.00,3050.00 KGS ,675.60,349
21/Jan/2008,CHANA,20MAR2008,2279.00,2298.00,2269.00,2291.00,2278.00,8850.00 KGS ,2019.56,423
21/Jan/2008,CHANA,18APR2008,2318.00,2333.00,2309.00,2328.00,2311.00,4270.00 KGS ,990.18,210
21/Jan/2008,CHANA,20MAY2008,2377.00,2377.00,2365.00,2370.00,2365.00,100.00 KGS ,23.70,2
21/Jan/2008,CHANA,20JUN2008,2415.00,2422.00,2415.00,2419.00,2410.00,20.00 KGS ,4.84,0


some feedback on output format is needed.

Pankaj

 

[RSI]

Mr Iyer

Here is the sample data downloaded from MCX. Now I notice that Symbols are repeated with different Expiry dates mentioned. How do you want to format the output file.
-------------------------------------------------------------------
some feedback on output format is needed.

Pankaj

Pankaj,
Output format is simple. Key point to note is expiry date. Assuming that this is month of January, if we have one Aluminium Contract expiring on 31/01/2008, its ticker symbol can be ALUMINIUM-I. If another Aluminium contract expires on 29/02/2008, its ticker symbol can be ALUMINIUM-II so on and so forth. Always, near month contract can have symbol <name of commodity either in full or in short>-I, next month contract can have symbol <name of commodity either in full or in short>-II etc. This will provide for continuous charting.

So out put line will be

<ticker> <name> <date> <open> <high> <low> <close> <volume> <open interest>

As OI data is given in lots, volume figure is preferred over PCP and value figures.
Hope this helps
Thanks and regards
R. S. Iyer

 

[pkjha30]

Pankaj,
Output format is simple. Key point to note is expiry date. Assuming that this is month of January, if we have one Aluminium Contract expiring on 31/01/2008, its ticker symbol can be ALUMINIUM-I. If another Aluminium contract expires on 29/02/2008, its ticker symbol can be ALUMINIUM-II so on and so forth. Always, near month contract can have symbol <name of commodity either in full or in short>-I, next month contract can have symbol <name of commodity either in full or in short>-II etc. This will provide for continuous charting.

So out put line will be

<ticker> <name> <date> <open> <high> <low> <close> <volume> <open interest>

As OI data is given in lots, volume figure is preferred over PCP and value figures.
Hope this helps
Thanks and regards
R. S. Iyer

Mr Iyer

Its like FNO symbols. No problem
This one may be finished.

Thanks
pankaj

 

[RSI]

Pankaj,
Please download one sample Bhavcopy EOD file and study. For example, contract expiry date is quite messy. In case of commodities near month contracts are expiring on 31/01/2008, in case of others expiry is on 15/02/2008 and in case of newly incorporated carbon credit contacts near month expiry date is as farther as 15/12/2008.

In my opinion, while converting, the software should search the name of the commodity from the downloaded Bhavcopy csv file and after finding all contracts for that name, it should find out which contract is expiring at the earliest date and then it should assign ticker symbol to it such as <name of commodity long or short>-I etc.

 

[pkjha30]

Pankaj,
Please download one sample Bhavcopy EOD file and study. For example, contract expiry date is quite messy. In case of commodities near month contracts are expiring on 31/01/2008, in case of others expiry is on 15/02/2008 and in case of newly incorporated carbon credit contacts near month expiry date is as farther as 15/12/2008.

In my opinion, while converting, the software should search the name of the commodity from the downloaded Bhavcopy csv file and after finding all contracts for that name, it should find out which contract is expiring at the earliest date and then it should assign ticker symbol to it such as <name of commodity long or short>-I etc.

Mr Iyer

here is the original downloaded file and converted file. Attached(both zipped)

sample of converted file is shown below.

in original file 740 records were downloaded. After eliminating those records for which volume /value of trade was zero only 339 remained.
I hope this is fine.

original commodity name is retained and a new name is generated as Symbol for each commodity with different expiry.
May be we need some more refinement here.

*************************
sample file with proposed meta headers
*************************
<scripname>,<symbol>,<trdDate>,<expDate>,<open>,<high>,<low>,<close>,<prevClose>,<vol>,<oi>
ALUMINIUM,ALUMINIUM-1,1/25/2008,1/31/2008,95.75,97.5,95.3,97,95.2,1496,667
ALUMINIUM,ALUMINIUM-2,1/25/2008,2/29/2008,96.35,98.5,96.35,98.2,96.2,500,117
ALUMINIUM,ALUMINIUM-3,1/25/2008,3/31/2008,97.1,98.9,97.1,98.7,97.15,475,27
CARDAMOM,CARDAMOM-1,1/25/2008,2/15/2008,705.5,708,702,703,702,115.5,398
CARDAMOM,CARDAMOM-2,1/25/2008,3/15/2008,706,712.5,705,707,704.5,16.5,132
CARDAMOM,CARDAMOM-3,1/25/2008,4/15/2008,710.5,714.5,707,713,703.5,7,59
CFI,CFI-1,1/25/2008,12/15/2008,1151,1210,1151,1190,1187.5,8.6,6
CFI,CFI-2,1/25/2008,12/15/2009,1206.5,1220,1194,1205.5,1257,2.2,0
CHANA,CHANA-1,1/25/2008,2/20/2008,2250,2263,2249,2251,2254,15590,329
CHANA,CHANA-2,1/25/2008,3/20/2008,2354,2360,2345,2348,2346,11690,421
CHANA,CHANA-3,1/25/2008,4/18/2008,2390,2402,2389,2391,2389,6610,318
CHANA,CHANA-4,1/25/2008,5/20/2008,2430,2445,2430,2439,2430,8750,101
CHANA,CHANA-5,1/25/2008,6/20/2008,2470,2482,2470,2480,2466,630,29
COPPER,COPPER-1,1/25/2008,2/29/2008,277.8,283.2,276.5,278.65,277.35,44052,12975
COPPER,COPPER-2,1/25/2008,4/30/2008,280.55,285.3,279.25,281.2,279.6,3046,1887
COPPER,COPPER-3,1/25/2008,6/30/2008,284,287,273.3,281.55,281.95,65,103
CRUDEOIL,CRUDEOIL-1,1/25/2008,2/15/2008,3502,3590,3502,3549,3491,3590.8,7351
CRUDEOIL,CRUDEOIL-2,1/25/2008,3/14/2008,3499,3585,3499,3550,3483,226.4,979
CRUDEOIL,CRUDEOIL-3,1/25/2008,4/15/2008,3533,3570,3530,3559,3433,5.7,526
CRUDEOIL,CRUDEOIL-4,1/25/2008,5/15/2008,3514,3570,3510,3543,3457,21.5,385
CRUDEOIL,CRUDEOIL-5,1/25/2008,6/13/2008,3555,3581,3549,3562,3443,4.7,968
GOLD,GOLD-1,1/25/2008,2/5/2008,11510,11654,11491,11532,11484,64692,9396
GOLD,GOLD-2,1/25/2008,4/5/2008,11581,11719,11554,11609,11552,6644,4079
GOLD,GOLD-3,1/25/2008,6/5/2008,11616,11780,11616,11688,11589,47,106
GOLD,GOLD-4,1/25/2008,8/5/2008,11799,11819,11771,11801,11630,5,32
GOLD,GOLD-5,1/25/2008,10/4/2008,11701,11875,11701,11810,11739,6,35
GOLD,GOLD-6,1/25/2008,12/5/2008,11880,11967,11851,11909,11733,18,38
GOLDM,GOLDM-1,1/25/2008,2/5/2008,11485,11645,11481,11529,11474,2366.1,6215
GOLDM,GOLDM-2,1/25/2008,3/5/2008,11525,11662,11507,11548,11497,577.8,3778
GOLDM,GOLDM-3,1/25/2008,4/5/2008,11567,11701,11552,11606,11526,107.9,545
GUARSEED,GUARSEED-1,1/25/2008,3/20/2008,1850,1850,1790,1815,1779,2000,10
JUTE,JUTE-1,1/25/2008,1/31/2008,1310,1311.5,1306.5,1307,1308,520,674
JUTE,JUTE-2,1/25/2008,2/29/2008,1337,1337.5,1327,1329.5,1331,690,605
JUTE,JUTE-3,1/25/2008,3/31/2008,1358.5,1358.5,1348,1350,1353,340,476
KAPAS,KAPAS-1,1/25/2008,4/30/2008,478.2,480.6,477.2,479.2,477.8,1596,2348
KAPASKHALI,KAPASKHALI-1,1/25/2008,1/31/2008,380.1,392,380.1,385.5,381.9,615,596
KAPASKHALI,KAPASKHALI-2,1/25/2008,2/29/2008,392,396.5,392,394.2,390,2655,1143
KAPASKHALI,KAPASKHALI-3,1/25/2008,3/31/2008,404.6,407,402,404.2,400,1000,1130
KAPASKHALI,KAPASKHALI-4,1/25/2008,4/30/2008,413,418,413,415.2,410,245,497

*************************

Comments are requested.

Before release this will be included as beta till further specification is firmed up and frozen.
other members who may be using commodities eod may also comment. for wider acceptance and appropriate formats.


pankaj

ps:- Looks like I am in a village talking of Potato prices etc though hope someday villagers use eoddlc to download latest prices on their $100 laptop. India will grow with their growth.

 

[RSI]

Pankaj,
I am extremely happy to see the converted file. It is just perfect. A few suggestions
1. Kindly modify the software in such a menner that it remove the colums <expDate> and <prevClose> at the time of convertion of file. These are unnecessary colums while importing it in to database. If these columns are present in the converted file, it will create problems in importing in to charting software database.
2. Can the software be modified in such a manner that it gives the following colums of data in the following order of headers in the row?
<symbol> <name> <date> <open> <high> <low> <close> <volume> <open interest>
Why I am saying is many of us may be using Amibroker or Metastock or Fcharts or Fibotrader. In Ami you can suitably modify the column header at the time of importing. But I am not sure about Metastock. If the column headers are given in the order specified by me, there will not be any problem in importing the price file in to Meta Database. I am not an expert on Metastock. But better check out the importing procedure.

Thanks once again and regards
R. S. Iyer

 

[RSI]

Pankaj,

As far as zero volume or zero value of traded prices are concerned, please note that there is no trading interest among traders in respect of some commodities. Some other commodities are banned from future markets by the government whenever some political parties raise hue and cry about inflation or price rise in that particular commodity.
I sincerely hope this feedback helps
Thanks once again and regards
R. S. Iyer

 

[pkjha30]

Pankaj,
I am extremely happy to see the converted file. It is just perfect. A few suggestions
1. Kindly modify the software in such a menner that it remove the colums <expDate> and <prevClose> at the time of convertion of file. These are unnecessary colums while importing it in to database. If these columns are present in the converted file, it will create problems in importing in to charting software database.
2. Can the software be modified in such a manner that it gives the following colums of data in the following order of headers in the row?
<symbol> <name> <date> <open> <high> <low> <close> <volume> <open interest>
Why I am saying is many of us may be using Amibroker or Metastock or Fcharts or Fibotrader. In Ami you can suitably modify the column header at the time of importing. But I am not sure about Metastock. If the column headers are given in the order specified by me, there will not be any problem in importing the price file in to Meta Database. I am not an expert on Metastock. But better check out the importing procedure.

Thanks once again and regards
R. S. Iyer

Mt iyer
Ok , looks like this is what you want.

<symbol>,<date>,<open>,<high>,<low>,<close>,<vol>,<oi>
ALUMINIUM-1,1/25/2008,95.75,97.5,95.3,97,1496,667
ALUMINIUM-2,1/25/2008,96.35,98.5,96.35,98.2,500,117
ALUMINIUM-3,1/25/2008,97.1,98.9,97.1,98.7,475,27
CARDAMOM-1,1/25/2008,705.5,708,702,703,115.5,398
CARDAMOM-2,1/25/2008,706,712.5,705,707,16.5,132
CARDAMOM-3,1/25/2008,710.5,714.5,707,713,7,59
CFI-1,1/25/2008,1151,1210,1151,1190,8.6,6
CFI-2,1/25/2008,1206.5,1220,1194,1205.5,2.2,0
CHANA-1,1/25/2008,2250,2263,2249,2251,15590,329


or as your suggestion

<symbol>,<name>,<date>,<open>,<high>,<low>,<close>,<volume>,<open interest>
ALUMINIUM-1,ALUMINIUM,1/25/2008,95.75,97.5,95.3,97,1496,667
ALUMINIUM-2,ALUMINIUM,1/25/2008,96.35,98.5,96.35,98.2,500,117
ALUMINIUM-3,ALUMINIUM,1/25/2008,97.1,98.9,97.1,98.7,475,27
CARDAMOM-1,CARDAMOM,1/25/2008,705.5,708,702,703,115.5,398
CARDAMOM-2,CARDAMOM,1/25/2008,706,712.5,705,707,16.5,132
CARDAMOM-3,CARDAMOM,1/25/2008,710.5,714.5,707,713,7,59
CFI-1,CFI,1/25/2008,1151,1210,1151,1190,8.6,6

is it ok for others??

No comma in between headers??

Name is also needed in addition to symbol??

Any comments on date format?? Present one is in m/d/yyyy

Any comments on eliminating zero volume data.
Do you need commodities wise EOD data as well?? Is it really needed?

Pankaj

 

[chintan786]

ya date formate is fine .. no probs. 0 volume data shd be eliminated.. and yes I think commodity wise date will be more useful.... so dear friend pankaj when I can see the latest version.

chintan

 

[kuriako]

ps:- Looks like I am in a village talking of Potato prices etc though hope someday villagers use eoddlc to download latest prices on their $100 laptop. India will grow with their growth.

One Indian in a remote village with a $875 laptop was looking at RSS4 Rubber price every day to update his villagers about current market price of rubber...

And now EODDLC will make tech analysis for the planters here on when to sell their stock!!!


BTW Will need to do TA on when to buy Rice, Onion, Potato and vegetables
Guess I dont have to rely any more on "vodafone alerts" to buy onion

Thanks a ton Pankaj for keeping the spirit of FREE software!!!