EOD Download/ Convert from NSE and BSE

[RSI]

ya date formate is fine .. no probs. 0 volume data shd be eliminated.. and yes I think commodity wise date will be more useful.... so dear friend pankaj when I can see the latest version.

chintan

Chintan,
I presume you are intending to say you need commoditywise data. Not datewise (I mean expiry datewise) commodity data. If you are intending to mean that let me inform you that if you import data in that format you will not get continuous chart. every chart will be broken at the end of contract period and new period will bring in new chart. No meaningful technical analysis can be done on that.
I hope this clarifies the request.
R. S. Iyer

 

[RSI]

Pankaj,

I donot know whether any comma is required in between headers such as <symbol> <name> etc. If it is needed for successful importing of data in to charting software database, then kindly add that also.

Please make provision for name also. Because some commodities are having strange symbols, reading of which will not help a trader to identify the commodity. For example (this I have taken up from NCDEX Bhav copy csv file) symbol "CHARJDDEL" denotes Chana contract.

Date format, I prefer mm/dd/yyyy

If possible, please provide commoditywise data provision as well.
Thanks for your efforts. I deeply appriciate your selfless services.
Thanks and regards
R. S. Iyer

 

[RSI]

Pankaj,
There seems to be some problem either in internet connection or with the length of message at my end. So I am splitting my next post in to three posts.

 

[RSI]

Pankaj,
Do not think that I am too greedy. Let us make your software a one stop for all need of data import software.

What I mean to say that you have almost finished (from what I have seen, it is 100% success) MCX Bhavcopy download and convert utility.

Now the only thing remains is NCDEX. NCDEX also provides EOD bhavcopy in csv format. Kindly download a sample bhavcopy. Except for addition of a few more columns, which we have to remove from the file at the time of conversion, it is almost similar to MCX Bhavcopy. Slight tweaking of one or two software codelines will be sufficient to handle downloading and converting NCDEX bhavcopies as well.

 

[RSI]

Why I am saying is, comparitively, agricultural commodities are traded more on NCDEX than on MCX. MCX is favourite for metals and energy. Some contracts such as Chana, Mentha (oil as well as seed), Guar and its derivates etc. are having good volatility and are favourites among the traders. They are actively traded on NCDEX.

Who knows, non availability of this kind of software and data might have prevented many members here from actively participating in commodity markets. Your software may be initiator and motivator for many members here.

So to put it in nutshell why can't provision for downloading and converting EOD data from NCDEX be made as well?

I request you to consider this in serious earnest.
Thanks and regards
R. S. Iyer

 

[RSI]

Only thing to be remembered is that while assigning ticker symbols, a different symbol than MCX symbol is to be assigned. This is because some commodities are having common symbols both on MCX and NCDEX. If one imports both data in to common database, earlier imported data will be overwritten by later imported data.

Avoiding this is quite simple. You have already assinged symbols such as GOLD-1 etc. to MCX tickers. Now you can assign either GOLD-i or GOLD-I or GOLD_1 to NCDEX tickers

 

[pkjha30]

One Indian in a remote village with a $875 laptop was looking at RSS4 Rubber price every day to update his villagers about current market price of rubber...

And now EODDLC will make tech analysis for the planters here on when to sell their stock!!!


BTW Will need to do TA on when to buy Rice, Onion, Potato and vegetables
Guess I dont have to rely any more on "vodafone alerts" to buy onion

Thanks a ton Pankaj for keeping the spirit of FREE software!!!

Amen to that


I will ask satya to provide online charting for commodities through investbulls.com free

so villagers can do TA easily.

Pankaj

 

[pkjha30]

Only thing to be remembered is that while assigning ticker symbols, a different symbol than MCX symbol is to be assigned. This is because some commodities are having common symbols both on MCX and NCDEX. If one imports both data in to common database, earlier imported data will be overwritten by later imported data.

Avoiding this is quite simple. You have already assinged symbols such as GOLD-1 etc. to MCX tickers. Now you can assign either GOLD-i or GOLD-I or GOLD_1 to NCDEX tickers

My Iyer

thanks for feedback. I will see how NCDEX can be incorporated.
MCX,NCDEX and BSE are more restrictive in their efforts to share information (that too in the age of transparency and with RTI Act they should be sharing such information more openly, also as it will benefit large number of users, especially retail investors and villagers, whose participation is more important for depth of these markets else these will be in shackles. )

They use asp with some more advanced tech so that automatic download is prevented. Sometimes it is referred to as data mining.

I am using what is easily available for general public use freely as accessed by internet and website. But writing html parser for data mining is something that needs continuous monitoring and support which is beyond my scope.

However, if I am able to download NCDEX then rest is easy part as codes to breakdown files is now within eoddlc so there will be no problem.

as for symbols, I think if I attach exchange name would it be ok??

like
GOLDMCX-1
GOLDNCDEX-1

GOLD-1 for MCX and GOLD_1 for NCDEX is also fine.

As for commodity wise list , yes it is possible within the present framework without much coding.

However I need one more clarification about dates. There are two dates in EOD. One is trade date/report date and another is expiry date. Expiry date is used for deciding contracts if it is near ,next or far (1,2,3 etc).
whereas trade date is used for out put and also as normal date field found in eod of NSE/BSE (indicating today's trade ) and used by TA softwares.

this is in response to chintan's post also.

I hope my understanding is clear here.

I will use user defined date format later.

But for release I will stick to the present output. As I am in the process of integrating other functions as pointed out for NSE and BSE data by users.Only portfolio marking is required for BSE so that some limit can be imposed for number of stocks downloaded for BSE data(other than bhavcopy to avoid too much strain on servers, let us not misuse their hospitality, )

The most difficult part of all this is to go to the code line by line several times yet keeping overall picture in mind and how each part is interacting with others as the code base has become big.


lets see how it works out.

Pankaj

 

[RANJITSINGH66]

hi pkjha30 saab,i hope you will help us by making your software FIBOTRADER compatible.
thanking you
RANJITSINGH66

 

[pkjha30]

hi pkjha30 saab,i hope you will help us by making your software FIBOTRADER compatible.
thanking you
RANJITSINGH66

Hi Ranjit

What is this saab??

just pk would do.

I think output from EODdlc i compatible with Fibotrader as you can see from sample taken from Soumya's thread

Sample for BSE

<ticker>,<name>,<date>,<open>,<high>,<low>,<close>,<volume>
526921,21ST CEN.MGM,1/25/2008,90.6,99,90.45,91.8,51216
532628,3I INFOTECH,1/25/2008,125,136.25,124,129.7,226930
523395,3M INDIA LTD,1/25/2008,1875,1980,1715.1,1896.45,3909
530499,A.K.CAPITAL,1/25/2008,400,415,400,404.4,5619
500009,A.SARABHAI,1/25/2008,30.25,33,30.1,31.4,76726
531190,A.V.COTTEX,1/25/2008,24.7,24.85,22.8,22.8,1100

sample for NSE
<ticker>,<name>,<date>,<open>,<high>,<low>,<close>,<volume>
3IINFOTECH,3I INFOTECH LTD.,1/25/2008,133,136,126,130.3,648449
3MINDIA,3M INDIA LIMITED,1/25/2008,1980,1980,1772,1899,1469
AARTIDRUGS,AARTI DRUGS LTD.,1/25/2008,60.1,63.3,60,62.9,7439
AARTIIND,AARTI INDUSTRIES LTD,1/25/2008,36,39,32.5,35,149683

Futures
<ticker>,<name>,<date>,<open>,<high>,<low>,<close>,<volume>
3IINFOTECH-III,3IINFOTECH-FAR MONTH,1/25/2008,118,134,118,134,27000
3IINFOTECH-I,3IINFOTECH-NEAR MONTH,1/25/2008,130,136.9,126.15,130.35,878850
3IINFOTECH-II,3IINFOTECH-NEXT MONTH,1/25/2008,130,137.4,127.75,131.8,108000
(here it uses III and FAR as part of Symbol and Name. ,don't know if really both are required. One should be enough but not checked )

Indices

This file is different and indices will be added in upcoming release.
and will come as part of EOD.
Right now I am working to calculate Indices volumes and deliverables so as to include that also.

I am also thinking of auto update of TA softwares through ODBC or DDE where it is supported.
AMIBroker and FCharts support this.It is like using eoddlc instead of quotetracker. But only for EOD data. This will be available for next release after current work is over. But data conceptualization has to be done now itself so as to avoid compatibility issues later.

Let us see.

pankaj