EOD Download/ Convert from NSE and BSE

Hello Pankaj

Thanks for making a very usefull software .It is teally v.good thanks a lot

I have one problem that is I am unable to download BSE indices all of then through it even after applying all that you have written in help PDF the indices which are downloded do not have any data and many among them fails to be downloaded only there CVC files are created

Pramendra
 
Dear Mr.PKJha,

I am thankful to you for your wonderful software.
I can save lot of time and energy, and think eveyrone would feel so.
I am new to this forum and hope to learn a lot from you all.

Shall I know what kind of charting software you use. Any free software you can recommand. I am using FCharts and Quotetracker for the last one week and I am not fully configured them to use.

Any tips from you and other members are welcome.

regards

bkkumar

Hi Traderji and all the Forum members

I am posting EODDwonloadConvert version 0.0.70 for forum members to

try and give their comments.Feel free to use it and give feedback. No warranties.Code is optimisation under progress.

It needs unzip32.dll which can be obtained from
http://www.mirrorservice.org/sites/ftp.info-zip.org/pub/infozip/WIN32/unz552dN.zip

It need to be placed in \windows\system ditrctory and register it.Rightclick on it will register it else use "regsvr32.exe unzip32.dll" in Run dialog of Start Button. Ignore any error. The version is unzip32.dll 5.5.2.0. . Only dll is available in zipped file in my briefcase.

The programme is compiled using vb6(sp6) and needs runtime files.Normally these are available on almost all systems now but on some older system it may not be available and give errors.You can download runtime files from here if required.

http://www.microsoft.com/downloads/...61-7A9C-43E7-9117-F673077FFB3C&displaylang=en

Its 1.0 MB download.

you need IE 5.5 or above installed.

Dependency Script is attached for your use to verify if required files are present.I have tested it on Win98SE and WINXP system . it should not give much trouble. The zipped file EODDownloadConvert.zip(142.5kb) may be downloaded from my briefcase.
http://briefcase.rediff.com/bn/br/pjhapondy


The programme should be extracted to a seperate folder of your choice. It will create folder structure and required files on first run . Settings related files are in BIN folder. My BIN folder is also placed in the Briefcase in case you may like to download some files such as Favourites.txt,holiday.txt etc. Programme icon is in BIN\images folder.

I have to thank http://www.planetsourcecode.com , www.freevbcode.com

and www.programmersheaven.org and www.info-zip.com for their code

snnipets and dll files. All thanks to respective programmers.My part was limited to certain areas and ideas specific to downloading files from nse and bse website and assemblling it to make a workable solution for members of Traderji.com

I have to thank satya whose dataextractor gave some inspiration for

this and his idea of thread to share software. My code will be freely available once it reaches version 1.0.0 and after your feed back is also incorporated.The programme has its own lifetime and utility.It may stop functioning if there are changes in URL structure of NSE and BSE websites.If newer operating system do not support older programmes. In such a situation it would need to be ported to newer versions.

What it seeks to do:-

Download:

For a given range of dates it will download bhavcopy for nse. For BSE it will download current Bhavcopy which is in zipped format. Itwill unzip the same without user intervention.You have to enter dates from and to in the field and click ADDUrl.

All URLs will be displayed in the box below with save path. As of now use default options it should be ok. For other options if there is any problem please tell me.

On the top there is option for providing download urls if you know it. If it is from BSE or NSE it will be stored in default folders. Other wise it should be available in Others folder.

Convert:

Initially use only default options.

If files are available in default folders then it will be visible in Input Files Box. Defaults are ok i.e. csv and nse or bse. Delimiter is always "," i.e. comma. leave it unchanged. You will see the first two lines of records in Select output options. If first line is header record you can check it . You can leave all other options as it is. On clicking Convert button it will convert all available file in NSE or BSE equities folder One folder will contain date wise EOD data for each date for all stocks, the other folder will contain
individual stock data for all dates available.

Datewise format is symbol,o,h,l,c,v,date
Stockwise format is o,h,l,c,v,date

First line is header line.

Perhaps these files can be used by members for importing into their :)
programmes. if any error is given I would like to know.

If you have downloaded many files you need to sort these records in the output files. Use sort button for this. It needs to be used for once only. thereafter if you are running your program after 5.30 or whatever time bse/nse post their latest data you dont need to sort.
I understand that bhavcopy is posted after 5.30 pm.

You can exclude any date by editing the holiday.txt file either directly. All dates are posted below respective month. Same pattern needs to be followed. the program will indicate if their is a failure of Download for whatever reason mostly URL is not available i.e. file is not found. It is trapped and failure.log is generated.

You can verify if URL exixts or not by using some other downloader.Error.log file wil keep records of two days errors and may be used for feed back.

Browser:

You can use this to browse your favourit websites wihtout overhead of IE. But you need IE 5.5 or above installed.

Some of the Links will give you direct access to nes/bse html files without going to home page. Loading will be faster. You can add favourites First input is for title and secon input is for URL. If you omit URL title can act as section header.

You can directly edit Favourites in your text editor. Odd lines will be titles and even lines should be URLs.

You can also check if your computer is connected to internet and if NSE website is up and running. Buttons are on Download page.

If this programme works it will be free and once it is optimised it will be given in source code form for maitainance by members if desired.

Future Wishlist

To optimise code.

To incorporate charts and other indicators based on the data collected on daily basis. Chart code and form is included but made invisible as further development is required.

To incorporate multi threaded downloading capability.

To take print( Presently you can print file in one go)

The name given to the executable is only tentative and members are invited to suggest better name. We may vote for name and whichever name gets highest vote will be made as programme name.


thanks are in order for Satya, joymitali,Saint, AmitBE,AMITKR murthymsr and otherboard members for inspiraton.

I dedicate this programme to the enterprising spirit of Traderji.com

and Moderator traderji.

With kind regards
Pankaj
 
Hi!

Dear Pankaj,

What a wonderful software. I could download NSE, BSE & even derivatives. It works excellent and even the converter is very powerful.

But I face few problems. I hope somebody may be able to solve it.

I am using MetaStock and the downloaded data for NSE works fine with that. But with BSE I can not put the converter sequence in proper order. Also with derivatives the sequence change is simply not available. Also can anybody tell me how to use derivatives in Metastock?

Regards

Vivaswan
Junior member (In all respects)
 

pkjha30

Well-Known Member
HI

First of all an apology is in order. Having released something for use of members and public It was my responsibility to attend to their problems.I could not do the same nor rectify the defects pointed out.

Reasons are plenty but can not be excused. So I am really sorry for the delay.

I can not say that the current version will be a perfect one as softwares are in constant evolution.

However some of the problems ,I have tried to attend.

1. Now the program is released as EODSetup.exe and uploaded to my briefcase.

http://briefcase.rediff.com/bn/br/pjhapondy///Shared/EODSetup.exe?d=0

This should be able to answer winxp vs. win98 and old vs. new system files problem.However is message is flashed that your system file is newer than the version being copied, kindly choose to retain your version.

2. A zipped version of EODDownloadConvert is also uploaded as upgrade but do take care to delete old Application.ini file found in BIN folder.
http://briefcase.rediff.com/bn/br/pjhapondy///Shared/EODDownloadConvert-Upgrade.zip?d=0

3. A cab setupfiles have also been uploaded EODDownloadConvert.zip Upon unzipping the file you will get setup files which will install the program.
http://briefcase.rediff.com/bn/br/pjhapondy///Shared/EODDownloadConvert.zip?d=0

Changes are based on feedback wherever I could attend to.

Folder layout is slightly changed. In the application folder you have BIN, BACKUP, DATA, Temp and Help Folders.
BACKUP folders have BSE ,NSE , Others as Folders. BSE and NSE again contains EQUITIES and FNO folders. These are the folders where files are backuped after conversion.

DATA folder contains BSE, NSE, Others Folders. NSE contains EQUITIES,FNO Indices, YIELD folders. BSE Contains EQUITIES, FNO, WL_INTRA, WL_DAILY, WL_MONTH, WL_WEEK

EQUITIES and FNO folders contains CONVERTED Folder. It contains files which are converted .

All downloaded files will go to EQUITIES folder for Equities and FNO folder for derivatives.

I would advise users to browse through the folders for familiarity.

Some bugs were reported in creation of files or folders. Hopefully these may not occur.

As regards FNO I got feedback that to maintain continuity it is better to append Near next and Far or 1, 2, 3 to the symbols. Now that option is provided. Another change is to check for zero values signifying no trades. This has considerably speeded up the conversion. Users may decide if it is useful. Since it is a fixed sequence conversion, there is no option for changing the sequence.


Now Data consistency check has been added. If data error is found it will be logged under respective date file under DATAERROR folder in BSE or NSE. No correction will be made. You have to verify manually and rectify it.

Another option is for Bonus and split checking . If it is found then it will be logged in BONUS folder under BSE or NSE folder with respective date file. Again it will only indicate suggestive ratios. You have to verify and use this info with your TA.While running I was able to catch few know cases of Bonus+split easily.

Another feature is to screen the stock based on open price and traded volume criteria. You can indicate min price or volume above which only stocks will be converted.

All the above features have to be used with check boxes.


Portfolio watchlist has problem if you add too many stock. This is not so much a limitation of the program as that of bandwidth.I would suggest to keep watchlist to manageable limits.

Now input and output settings for converter and downloader is hidden from view. You can also show or hide it. Default combination works good for most part.

There were a lot many things planned but presently I could not attend to them.I have tested this version on WINXP system. It should run on WIN 98 also as none of the system files are changed.

Auto updater is also changed to download EODSetup.exe rather than EODDownloadConvert.zip file. But existing updater should indicate presence of new file.

Please give your feedback which will be helpful in further development.


Pankaj
 

pkjha30

Well-Known Member
Hi pankaj,
Param may be right, has BSE changed its web address where it stores indices data? tried searching but no luck.
pl help.
warm regards
Hi Raghu

BSE does not give indices data. Indices download for NSE is only available. However there is a way out which is being used in the programme.If you use watchlist with Indices code it will download those indices also.
If there is some problem then may be they have changed web location. In that case I have to check .

Pankaj
 

pkjha30

Well-Known Member
Hi!

Dear Pankaj,

What a wonderful software. I could download NSE, BSE & even derivatives. It works excellent and even the converter is very powerful.

But I face few problems. I hope somebody may be able to solve it.

I am using MetaStock and the downloaded data for NSE works fine with that. But with BSE I can not put the converter sequence in proper order. Also with derivatives the sequence change is simply not available. Also can anybody tell me how to use derivatives in Metastock?

Regards

Vivaswan
Junior member (In all respects)
Default sequence for BSE should work fine.
For derivatives no sequence change option is available.
For metastock you can ask some of more experienced member.
Pankaj
 

pkjha30

Well-Known Member
Hello Pankaj

Thanks for making a very usefull software .It is teally v.good thanks a lot

I have one problem that is I am unable to download BSE indices all of then through it even after applying all that you have written in help PDF the indices which are downloded do not have any data and many among them fails to be downloaded only there CVC files are created

Pramendra
Hi I will try to verify that and get back to you
Pankaj
 

pkjha30

Well-Known Member
Hello Pankhaj

Thanks for your detailed reply. I find your utiity is the best one that may meet the needs of a user for all data sources under one roof. It only requires a few modifications to make it very useful and reliable.

I did reinstall again yesterday and downloaded the derivatives bhavcopies. It again got downloaded to the wrong folder- C:\Program Files\Download and Convert\DATA\NSE\Derivatives.

Similarly when reinstalled again it checked for updates automatically and software downloaded updates . But actually it did not download any updates. After downloads it asked for installing updates.Again nothing happened. I checked the version number. It was the old one. And programme was again asking to update to new vesions. Only when i manually updated from the updates obtained from your brief folder the SW was updated.

The derivatives data has to be seamlessly updated. ie- continous contracts-otherwise the data is not of much use.

I do have another suggestion for your consideration. Will it be possible to give a selection button for the following-

In the BSE and NSE bhavcopies there are lot of scrips which are not traded daily or regularly. Some for longer periods. There are many scrips which trade in miniscule volumes or in very low price levels. Some trade in paisas also.

Will it be possible to include a selection criteria in the SW itself so that before conversion of downoaded data these types of scrips can be eliminated. Say selection of converted output based on volume, price, and latest trade dates.
This will help a user to have more relevant and current data.

"Development was delayed due to family obligations and pressure of work at office."

Please take your time. we are not in a hurry.

Regards

Saji
Hi saji

Some of these are incorporated.. Do Check them and give your feedback
Pankaj
 

Similar threads