Data deluxe- mother of all data downloaders suggestions required

[DataRam]

Kaly

real time backfill of fno and commodity is no big issue if the trading terminal supports and the data are pure.

in fact, NOW,NEST,DIET ODIN, TradeTiger (ShareKhan) every one supports minimum 1 min tf data and u can do the backfill from there. if u know macro/vba then it is semi automated otherwise its purely manual but u will get the real time backfill data

sr

I agree with you . The only drawback with such backfill remains is that if we track even 20-25 scrips in F&O , Commodities together then the whole process becomes lengthy of taking the data listing from the chart and then backfilling it. However this work is always better than shelling out money for the same.

Trade Tiger is quite good in data which can be used for backfill . 30 days Intraday charts in 1 min. format is really helpful and also we can track all the markets , now even currency charts.

At this link http://www.aamaadmisoftware.com/download-installation/
I saw a video of [SQA - NOW - AMI] where the process of backfilling by automating the opening of charts and then taking the data listing and completing the backfill is shown. This seems quite good as most of the manual process has been automated.

If this can be done in Trade Tiger too then it will be really cool. :clap:

RR.

 

[sr114]

I agree with you . The only drawback with such backfill remains is that if we track even 20-25 scrips in F&O , Commodities together then the whole process becomes lengthy of taking the data listing from the chart and then backfilling it. However this work is always better than shelling out money for the same.

Trade Tiger is quite good in data which can be used for backfill . 30 days Intraday charts in 1 min. format is really helpful and also we can track all the markets , now even currency charts.

At this link http://www.aamaadmisoftware.com/download-installation/
I saw a video of [SQA - NOW - AMI] where the process of backfilling by automating the opening of charts and then taking the data listing and completing the backfill is shown. This seems quite good as most of the manual process has been automated.

If this can be done in Trade Tiger too then it will be really cool. :clap:

RR.

Rocky

both NOW and Nest Trader gives u the option of the data to be written in excel. so if u can write a vb application doing the formating and saving as txt or csv then the process is semi- automated.

also found that Yusi's utility [NOW2AMI] can do this process of backfill automatically as SQA does - so there is a process of doing this.

this way the data are pure . and in case of 20-25 scrips it will take not more than 4-5 min.

regards
sr

 

[josh1]

Rocky

both NOW and Nest Trader gives u the option of the data to be written in excel. so if u can write a vb application doing the formating and saving as txt or csv then the process is semi- automated.

also found that Yusi's utility [NOW2AMI] can do this process of backfill automatically as SQA does - so there is a process of doing this.

this way the data are pure . and in case of 20-25 scrips it will take not more than 4-5 min.

regards
sr

SR

NOW, NESTTrader and Zerodha Trader have been developed by the same organisation - Omnesys. (ZT is Nest Trader 3). They have not disclosed any API for the program. They are selling it. Otherwise I would have automated backfill two months back.

 

[mrkanitkar]

For your knowledge
--------------

Here is info from NSE
http://www.nseindia.com/content/press/prs_datavending.htm
EOD data - Capital Market and Futures & Option Segment 25,000 per year.
Historical eod data CM Rs. 15,000/- (individual) for per year, means if you want 10 yrs hist data, will cost you 1.5 Lac.
Check this
http://www.nseindia.com/content/press/RealTimeTariff.pdf
Software Products/ Websites 16,00,000/- per year for Cash market
Software Products/ Websites 16,00,000/- per year for FNO
++++ there is variable Rs. 300 per month per user. First 300 users free.
These are pure data costs, infrastructure costs extra.
NSE sells historical IEOD as well.
------------
If one do not have license, one should not get in distributing data. The utility allows user to download it from server where it's kept for customers / public.
Customers mean, if you are sharekhan, Zerodha customer and you use this utility to import, export to any charting product, you are doing on own.
This is reason, no broker (except interactive brokers as they have license for it) provides data to charting app.
Broker can provide charting within terminal as per CTCL (Computer to Computer (CTCL) trading / Internet based Trading) app rules.
There are only 12 CTCL vendors in India.
http://www.nseindia.com/content/fo/fo_inetasps.zip
Omnesys NEST and Financial Technologies ODIN have major market share.

Prominent vendors empanelled with NSE include Omnesys Technologies, in which the exchange itself holds 26 per cent stake through a subsidiary and Jignesh Shah-promoted Financial Technologies India, which is NSE's rival in the currency derivatives segment and Religare Techno Global Services.
Here is interview of Omnesys CEO given in 2010.

http://www.indiainfoline.com/Resear...rector-Omnesys-Technologies-Pvt.-Ltd/21044562

Note: MT4 /MT5 is not CTCL approved, so it is not provided by any broker in India.

 

[mrkanitkar]

I am evaluating to develop free utility for data download. I have used many utilities by now. I still feel few things are missing. Can you guys please provide your wishlist?
I am working on following idea.
1. Single click data download[/COLOR] and update chart database as well. This will update data of Amibroker, Metastock, Fcharts, MT4, Ninja
2. Download EOD, IEOD, Fundamental data.
3. Download Futures, Options, data
4. Symbols can be added/ removed in one file. It can be similar to Amiquote.
5. Support for RT data from RTD - NOW,NEST,Zerodhatrade, Sharekhan, PIB
6. Support for RT data from google, yahoo
7. bulk deal download
8. data for all world indices, any symbol supported by google, yahoo.
9. Bhavcopy support for BSE,NSE,MCX
10. data download can be scheduled at automated task for every day.
11. will provide historical adjusted data till utility launch date, for following
a. BSE (From 1984, Sensex from 1979 with volumes)
b. NSE (From its inception in 1994)
c. 40+ World Market Indices starting from 1980/1990
d. NSE IEOD cash, Futures 5 min from 2008, Nifty cash IEOD- from 2007.
Does this makes sense?

Here is update on development of the product.
The design for entire product is done. The development, coding is done for EOD product. Design for RT data (web & terminal) will start soon.
Testing is going on for eod module. We are working on seperate module for data processing for output formats.
Here is detail status.
1. Single click data download[/COLOR] and update chart database as well. This will update data of Amibroker, Metastock, Fcharts, MT4, Ninja.
Status: Single click download is done. update item is in development.
2. Download EOD, IEOD, Fundamental data.
Status: Yahoo EOD, IEOD (1min, 5min) with options to select days, Fundamental yahoo data done. Google EOD, IEOD (1min, 5 min, 15 min, 30 min, 75 min, 1 hr) with options to select days done.
3. Download Futures, Options, data
Status: NSE, BSE FNO data done.
4. Symbols can be added/ removed in one file. It can be similar to Amiquote.
Status: Done
5. Support for RT data from RTD - NOW,NEST,Zerodhatrade, Sharekhan, PIB
Status: not started.
6. Support for RT data from google, yahoo
Status: not started.
7. bulk deal download
Status: Done
8. data for all world indices, any symbol supported by google, yahoo.
Status: Done
9. Bhavcopy support for BSE,NSE,MCX
Status: Done for BSE, NSE.
10. data download can be scheduled at automated task for every day.
Status: not started.
11. will provide historical adjusted data till utility launch date, for following
a. BSE (From 1984, Sensex from 1979 with volumes)
b. NSE (From its inception in 1994)
c. 40+ World Market Indices starting from 1980/1990
d. NSE IEOD cash, Futures 5 min from 2008, Nifty cash IEOD- from 2007.
Status: not started.
----------
I added following more features, development done for all, testing is in progress.
1.NSE Security-wise Delivery Position
2.BSE Security-wise Delivery Position
3.NSE Bulk deals
4.NSE block deals
5.NSE combined reports
6.NSE FNO participants data
7.NSE India Vix
8.NSE India all indices
9.NSE Forex
10.NSE market reports
11.NSE FII activity reports
12.NSE equity list mapping
13.NCDEX bhav copy
14.Mutual fund NAV eod
15.Weekend support
16.NSE emerge - SME exchange bhavcopy
17.BSE SME exchange bhavcopy
18.Corporate announcements, actions, reports.
19.Board meeting announcements
20.Market capitalisation reports
21.List of stocks which have reached a new high or a new low
22.ETF bhavcopy
23.NSE Currency market reports, FNO
24.Retail Bond market bhavcopy.
25 Split info.
-----------------
Will add support for following
1.National spot exchange
2.MCX (already planned)
3.MCX,BSE indices
4.USE support
---------------
This will be free product for end user.
Do we need anything else on EOD data? Please feel free to suggest.
Please also give suggestions on RT web and RT terminal.
I do not see much deployments on Ninja, MT4, MT5 so thinking to drop idea to support it in first release.

 

[josh1]

Mr

Which platform/language are you using for developing EOD part? It will be good for me if it can be used cross platform for Windows /Linux.

 

[mrkanitkar]

C# dotnet.
Since most of charting products are on windows, we decided to support windows.
2 yrs back, my team built similar product on Java, we used Eclipse platform. That product still works but little bulky. It dumps data in mysql and also have support for flat files.
This new product is of just less than 200KB in size, it uses WPF.

 

[megapixel]

C# dotnet.
Since most of charting products are on windows, we decided to support windows.
2 yrs back, my team built similar product on Java, we used Eclipse platform. That product still works but little bulky. It dumps data in mysql and also have support for flat files.
This new product is of just less than 200KB in size, it uses WPF.

Is there any possibility to download MCX IEOD data with this tool ?

 

[mrkanitkar]

Is there any possibility to download MCX IEOD data with this tool ?

Yes, we can, you/anyone need to give link for it.
I did not find any site like yahoo / google finance provides MCX quotes.

 

[MANISH_DAMANI]

Sir when will mt4 to amibroker will start so that we are able to get nymex and comex data.It will be gr8 help.Sir I have tarde tiger and , odin. Tradetiger is good in terms of data backfill but in odin diet no backill is available. But if Mt4 to amibroker available then lot of people take advantage in Mcx commodity tarding as trend is decided by comex /nymex its $ parity works. Currently I am using manshi rt if you provide mt4 also with reasonable rates and and mcx data 60 days backfill because it is must for my trading decision then I can switch to you.