EOD Download/ Convert from NSE and BSE

[chintan786]

Chintan,
I presume you are intending to say you need commoditywise data. Not datewise (I mean expiry datewise) commodity data. If you are intending to mean that let me inform you that if you import data in that format you will not get continuous chart. every chart will be broken at the end of contract period and new period will bring in new chart. No meaningful technical analysis can be done on that.
I hope this clarifies the request.
R. S. Iyer

thanks RSI, I was not aware of tht.

chintan

 

[pkjha30]

Pankaj,
Do not think that I am too greedy. Let us make your software a one stop for all need of data import software.

What I mean to say that you have almost finished (from what I have seen, it is 100% success) MCX Bhavcopy download and convert utility.

Now the only thing remains is NCDEX. NCDEX also provides EOD bhavcopy in csv format. Kindly download a sample bhavcopy. Except for addition of a few more columns, which we have to remove from the file at the time of conversion, it is almost similar to MCX Bhavcopy. Slight tweaking of one or two software codelines will be sufficient to handle downloading and converting NCDEX bhavcopies as well.

Well

looks like I was able to download NCDEX data as well

But this will push release date a little further as next week is quite tight for me and may not be able to spare enough time for few things but they are crucial for next stage of development as database has to be frozen especially Master records.

If someone has full list of BSE scrips with group scripcode, scripsymbol indices they form part of , company name, Face value, Industry sector classification then job may be little easier else I have to work my way through and I could not get groups and Industry sector classifications for all scrips . Indices constituents I have taken from BSE.

Pankaj

 

[pkjha30]

Mr Iyer

Below is sample NCDEX data

Tell me what the converted file should look like

Symbol,Expiry Date,Commodity,Ex-basisdeliverycentre,Price unit,Previous Closing Price ,Opening Price ,High Price, Low Price, Closing Price, Quantity Traded Today, Measure, No of Trades, Traded Value in lacs, Open Interest
FUTEXAGRI,1/25/2008,NCDEX FUTURE INDEX,NA,NA,1572.51,1577.73,1592.92,1576.82,1586.62,,NA,,,,,
NCDEXAGRI,1/25/2008,NCDEX AGRI INDEX,NA,YR 2001=1000,1543.1267,1549.89,1555.48,1549.89,1555.48,,NA,,,,,
ALUMINIUM,1/31/2008,ALUMINIUM_NEW ,Bhiwandi,RS./KG. ,94,0,0,0,94,0,MT ,0,0,0,,
ALUMINIUM,2/29/2008,ALUMINIUM_NEW ,Bhiwandi,RS./KG. ,94.6,0,0,0,94.6,0,MT ,0,0,0,,
ALUMINIUM,3/31/2008,ALUMINIUM_NEW ,Bhiwandi,RS./KG. ,95.2,0,0,0,95.2,0,MT ,0,0,0,,
BARLEYJPR,4/18/2008,BARLEY ,Jaipur,RS/QUINTAL,991.6,997,997,995.2,995.2,30,MT ,3,2.99,2780,,
BARLEYJPR,5/20/2008,BARLEY ,Jaipur,RS/QUINTAL,1005.6,1009,1012,1009,1012,20,MT ,2,2.02,2150,,
BARLEYJPR,6/20/2008,BARLEY ,Jaipur,RS/QUINTAL,1030,1022.2,1022.2,1022.2,1022.2,10,MT ,1,1.02,330,,
BARLEYJPR,7/18/2008,BARLEY ,Jaipur,RS/QUINTAL,1027.6,0,0,0,1027.6,0,MT ,0,0,10,,
BARLEYJPR,8/20/2008,BARLEY ,Jaipur,RS/QUINTAL,937.6,0,0,0,937.6,0,MT ,0,0,20,,
BARLEYJPR,9/19/2008,BARLEY ,Jaipur,RS/QUINTAL,1271.4,0,0,0,1271.4,0,MT ,0,0,0,,
BRENTCRUDE,2/14/2008,BRENT_CRUDE_OIL ,SullomVoeUnitedKingdom,Rs/Barrel ,3475,3503,3580,3503,3556.5,3100,barrel ,23,110.24,2300,,
BRENTCRUDE,3/14/2008,BRENT_CRUDE_OIL ,SullomVoeUnitedKingdom,Rs/Barrel ,3800,0,0,0,3800,0,barrel ,0,0,100,,
BRENTCRUDE,4/15/2008,BRENT_CRUDE_OIL ,SullomVoeUnitedKingdom,Rs/Barrel ,3511,0,0,0,3511,0,barrel ,0,0,0,,
CASTORDSA,2/20/2008,CASTOR_SEED ,DISA,RS/20 KG ,453.4,453.9,458,453.9,455.5,1130,MT ,104,257.65,4140,,
CASTORDSA,3/20/2008,CASTOR_SEED ,DISA,RS/20 KG ,450.8,454.3,456.5,453,455.4,760,MT ,61,173.03,3090,,
CASTORDSA,4/18/2008,CASTOR_SEED ,DISA,RS/20 KG ,450.8,455.9,457,454,455.6,450,MT ,45,102.49,1160,,
CASTORDSA,5/20/2008,CASTOR_SEED ,DISA,RS/20 KG ,455,461,462,456.2,460.2,170,MT ,16,39.11,290,,
CHARJDDEL,2/20/2008,CHANA ,DELHI,RS/QUINTAL,2295,2296,2308,2285,2289,11720,MT ,968,2692.44,23060,,
CHARJDDEL,3/20/2008,CHANA ,DELHI,RS/QUINTAL,2348,2350,2361,2339,2343,33520,MT ,2304,7875.75,34630,,
CHARJDDEL,4/18/2008,CHANA ,DELHI,RS/QUINTAL,2399,2403,2411,2394,2397,13530,MT ,1050,3250.86,23970,,
CHARJDDEL,5/20/2008,CHANA ,DELHI,RS/QUINTAL,2441,2445,2453,2436,2439,6620,MT ,519,1618.69,26350,,
CHARJDDEL,6/20/2008,CHANA ,DELHI,RS/QUINTAL,2470,2489,2492,2476,2480,580,MT ,49,144.17,2110,,
CHLL334GTR,2/20/2008,RED_CHILLI ,GUNTUR,RS/QUINTAL,3692,3687,3742,3665,3681,615,MT ,102,228.14,2320,,


looks like place and measurement is also important.

Pankaj

 

[RSI]

My Iyer

thanks for feedback. I will see how NCDEX can be incorporated.
MCX,NCDEX and BSE are more restrictive in their efforts to share information
They use asp with some more advanced tech so that automatic download is prevented. Sometimes it is referred to as data mining.

I am using what is easily available for general public use freely as accessed by internet and website. But writing html parser for data mining is something that needs continuous monitoring and support which is beyond my scope.

However, if I am able to download NCDEX then rest is easy part as codes to breakdown files is now within eoddlc so there will be no problem.

Pankaj,
I am extremely sorry. What is stated by you goes above my head. I am a total illiterate on software development or coding language.

 

[RSI]

as for symbols, I think if I attach exchange name would it be ok??

like
GOLDMCX-1
GOLDNCDEX-1

GOLD-1 for MCX and GOLD_1 for NCDEX is also fine.

Pankaj,
If you add exchange name as well, then some symbols will become very long. Note if you add NCDEX it adds five letters. Some charting softwares might have restriction on symbol length. So it is better to have smaller symbols. Any way, NCDEX itself is providing ticker symbol. Follow it and just add <NCDEX ticker symbol>_1 or <NCDEX ticker symbol>-i or <NCDEX ticker symbol>-I. As the time goes it becomes easier to remember ticker symbols as well.

 

[RSI]

Mr Iyer

Below is sample NCDEX data

Tell me what the converted file should look like
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Pankaj

Converted file should have the following headers
<symbol> <name> <date> <open> <high> <low> <close> <volume> <open interest>

I donot know whether any comma is to be added in between headers. If needed add that also.

Date format I prefer mm/dd/yyyy. Simply because, most charting softwares follow this date system by default and exchanges also supply data in this format. Better to stick with majority.

As with case of MCX file, software will scan the expiry date first of respective commodity contracts and thereafter assign ticker symbols i, ii etc.

While converting let the software delete the contract expiry date column. Commodities having zero values can be deleted.

 

[RSI]

But this will push release date a little further as next week is quite tight for me and may not be able to spare enough time for few things but they are crucial for next stage of development as database has to be frozen especially Master records.


Pankaj

Never mind. If the release date is postponed by one or two weeks, it does not matter. What is important is that the programme should be bug free and easy to operate. Last but not least finish your urgent needs first and thereafter devote your time on this. You are doing a great job. All the best
With regards
R. S. Iyer

 

[RSI]

I hope you will help us by making your software FIBOTRADER compatible.

Ranjit,
I am fed up with this uploading problem. I am splitting my next post in to two posts just to upload it here. Hope you donot mind it

 

[RSI]

Ranjit

I hope you will help us by making your software FIBOTRADER compatible.

Ranjit,
Difference between Fibotrader and FCharts (free version) on one hand and Amibroker and Metastock on the other hand is this.

Fibotrader and Fcharts (free version) will take only symbol, name, open, high, low, close and either volume or open interest data. You have to decide which one to import. I mean whether it is volume data or open interest data. In case of fibotrader, you will have to manually remove one column either volume or open interest. In case of fcharts (free version), it ignores the last column. This is the limitation of these two softwares.

 

[pkjha30]

Ranjit


Ranjit,
Difference between Fibotrader and FCharts (free version) on one hand and Amibroker and Metastock on the other hand is this.

Fibotrader and Fcharts (free version) will take only symbol, name, open, high, low, close and either volume or open interest data. You have to decide which one to import. I mean whether it is volume data or open interest data. In case of fibotrader, you will have to manually remove one column either volume or open interest. In case of fcharts (free version), it ignores the last column. This is the limitation of these two softwares.

Mr Iyer

This where eoddlc will provide flexibility (as of now it is for NSE/BSE EOD data. You can choose the sequence and fields and also if there should be header line and if so whether meta header ( like <ticker> ) or simple header (like ticker or symbol). You can decide the delimiter like comma or tab or quote though most accepted format is comma.

So one can decide his own conversion format in advanced view.I hope to extend this facility to other data sources like FNO/MCX/NCDEX

Pankaj