EOD Download/ Convert from NSE and BSE

[pkjha30]

Hi Saji and others who have written here

Your detailed response is worth considering. I will give reply to them as much as I can.

1. Setup problem

If a file is present on your system(whether old or new) you can safely click ignore button. No need to retry.

These problems will be taken care of in new version.

2.folder options seems to result in much confusion. I will try to stream line.
How ever I am following the structurer as below
..DATA>>NSE-->>Equities/Derivatives(->converted)/WL_INTRA/Converted

..DATA>>BSE-->>Equities/Derivatives(->converted)/WL_INTRA/Converted

..DATA>>Others>>/Converted

Files are downloaded to Equities/Derivatives/WL_INTRA and after conversion stored in Converted folder(for equities)
For Derivatives it is stored in DATA>>NSE>> Derivatives/Converted folder.

Downloaded files are not deleted. It is transfered to folder named DownloadedFiles in root folder.

3. folder selection dialog may be dispensed with in future version.

4. Derivatives data is converted in fixed format, i.e. sequence is predetermined based on user feedback and during conversion option for selection is not given. That is why field selector is not enabled.

5. The current version does not update derivatives seamlessly. But based on suggestion of few members I have coded the conversion in such a way that it will seamlessly update derivatives data. But we have to wait for next version.

6. In new version internet connection is checked only once. It can be rechecked if needed , manually.

7.You can tick appropriate checkboxes for stockwise or date wise selection. The choice will be retained next time.

8. NSE indices download is now integrated with the main download .The data will be inserted also in the converted files automatically( being worked on).

9.Another addition is to check for data errors and raise a warning in a seperate file.

10. Now bonus and split check will be done and recorded in seperate file. User can then manually verify the details and use the splitter functions of charting softwares .

11. Work under progress includes , in addition to above, download of nse intraday data.

12. Simplified user interface and one click download and another for convert. Advanced options are hidden from daily use.

13. Bugs reported by users are also being attended to.

Development was delayed due to family obligations and pressure of work at office.

Hope I am able to answer the questions. But a little more wait is there.

pankaj
Still more testing is required before it is released.

 

[saji oommen]

Hello Pankhaj

Thanks for your detailed reply. I find your utiity is the best one that may meet the needs of a user for all data sources under one roof. It only requires a few modifications to make it very useful and reliable.

I did reinstall again yesterday and downloaded the derivatives bhavcopies. It again got downloaded to the wrong folder- C:\Program Files\Download and Convert\DATA\NSE\Derivatives.

Similarly when reinstalled again it checked for updates automatically and software downloaded updates . But actually it did not download any updates. After downloads it asked for installing updates.Again nothing happened. I checked the version number. It was the old one. And programme was again asking to update to new vesions. Only when i manually updated from the updates obtained from your brief folder the SW was updated.

The derivatives data has to be seamlessly updated. ie- continous contracts-otherwise the data is not of much use.

I do have another suggestion for your consideration. Will it be possible to give a selection button for the following-

In the BSE and NSE bhavcopies there are lot of scrips which are not traded daily or regularly. Some for longer periods. There are many scrips which trade in miniscule volumes or in very low price levels. Some trade in paisas also.

Will it be possible to include a selection criteria in the SW itself so that before conversion of downoaded data these types of scrips can be eliminated. Say selection of converted output based on volume, price, and latest trade dates.
This will help a user to have more relevant and current data.

"Development was delayed due to family obligations and pressure of work at office."

Please take your time. we are not in a hurry.

Regards

Saji

 

[rpc]

hi Pankaj
Thanks for being back on the forum after a long time.Hope everything is fine at your end and we would get further enhancements to your excellent EODDC.
As Saji has mentioned Pls take your time and don't over-exhaust yourself
With Best Wishes
rpc

 

[rpc]

hi Saji
Extremely sorry by mistake I have mentioned your name wrongly as Aji instead of Saji.I sincerely apologise for the same.
Can someone guide how that can be rectified ?
Thank you
rpc

 

[pkjha30]

Thanks for writing saji and rpc,

It again got downloaded to the wrong folder- C:\Program Files\Download and Convert\DATA\NSE\Derivatives.

well that is the folder derivatives bhavcopies go to. Can you tell me where you expect it to go so that modifications can be made.

Similarly when reinstalled again it checked for updates automatically and software downloaded updates . ...Only when i manually updated from the updates obtained from your brief folder the SW was updated.

This is a known issue.Will try to rectify it.

Will it be possible to include a selection criteria in the SW itself so that before conversion of downoaded data these types of scrips can be eliminated. Say selection of converted output based on volume, price, and latest trade dates.

This is part of stock screener programme. However , it would not be too difficult to incorporate that here itself.

pankaj

 

[saji oommen]

Hello Pankaj,

You are correct about the folder. Since the progamme had a seperate folder named downloads i was under the impression all download files go there.I suppose after conversion these downloaded files are moved to downloads folder. Correct me if i am wrong.

Implementation of stock screener programme will be very useful and thanks for considering it.

Another suggestion i have is regarding default folder selections for NSE BSE, Derivatives for dwnloaded bhavcopies and converted data. The default settings done by you is OK but they are in C-drive if SW is installed in C-drive. There is a possibility of reinstallation of OS any moment.

Of course you have provided option for selection of alternate folders in other drives also. But can these selectons made once be fixed . What i mean is suppose i change BSE download path to a folder in E-drive and also the converted data path of BSE to another folder in E-drive or any other drive.

Then during next time or day i open the programmme say for downloading BSE data will the SW retain the earlier settings or will i have to reset again. I did try this . But the SW is reverting to default folders.

If such a facility is there a user can save the down loaded bhavcopies and converted data to a safe folder of his choice. May be you can provide such a facility at options, settings.

Or else you can do all these changes in load settings where by a user in the begining itself can set his download path and other converted data path settings which can be saved. So that next time SW is used all he has to do is to reload the settings and use the SW. The loaded settings can be common for NSE BSE, Derivatives data or individual settings for NSE cash, Derivatives or BSE.

For example you can even give provision for presetting output sequence required of data to be converted (including metastock format) in the load settings. This will make your SW convert screen simple . I feel you have put too many selection options in the screens.

This is something similar to use of ASCII import wizard in Amibroker.In such a case a user has to do his required settings only once.

Another point to note is regarding continous contracts in derivatives which expires on last thursday of every month.Some times this last thursday or day after it may be holidays. So expiry date is moved forward further.

Regards

Saji

 

[BirenG]

I wonder why BSE is not providing EOD data free of cost for download at their web site. When NSE provides bhav copy on day to day basis which traders find useful, what keeps BSE from extending this facility. We all know that BSE is the oldest exchange and has several other facilities to boast of, having greater number of listings etc. Besides, they have recently redesigned their website but this facility has been left out - advertently or inadvertently. Hope this reaches the attention of authorities concerned and we can expect a bhav copy of transactions available for download without any further delay.

 

[saji oommen]

Hello Pankaj,

Just few minutes before i downloaded todays BSE bhavcopy alone. Just for your information these are the things that happened. More important no earlier bhavcopies or converted data in any of EOD convertor folders. SW installed in C-drive programme files.

1. Download of BSE bhav successful

2. Went to convert page -CSV file visible in input files column

3. In input and output sequence colums all details like scrip name, open, high etc visible. Selection controls also active

4. For conversion i had changed destination path only to E-BSE Ascii folder.

5. After changing detination path in input and output sequence colums all details like scrip name, open, high etc disappears. Both columns fully blank. Selection controls active. CSV file visible in input files column. Convert button active.

6. I press the convert button. The convertor flashes a message that " No user output sequence prescribed- using defaults" Press Ok. The conversion goes on

7.On checking E-BSE Ascii folder i find all my BSE ASCII files of 6 months converted yesterday erased . When conversion was going on in the status bar i had noticed numbers running through. I did not know it was deleting the files.The status bar was also not showing ticker name, date, open, etc while converting as is the case during default settings. Message conersion over.

8. In E-BSE Ascii folder the only files left are date wise and stockwise of todays BSE data only. Since i use only stock wise when i opened it had details of first scrip only. ie 20th century finance. But datewise has all the scrip details but date was 20060324.

9. For trial again i removed downloaded bhavcopy and download again. No problem. I use the default settings in convert page. Everything works fine and converted outputs has no defects. ACC only one file.

10. Again once more i try using E-BSE Ascii folder. Same story with the exception that there is nothing in stockwise folder. Date wise wrong date again.

11. I go back again and try to use stored csv file in downloads folder without downloading bhavcopy again. But csv file erased. I tried this method since you had mentioned earlier that downloaded bhavcopies will be stored in root folder. I also wanted to check what if i want to convert all data again if i required it or lost it accidently by erasing it.

Normally it is only going back and using the downloaded bhavcopies to convert it again.Currently it seems there is no second chance since the csv files itself are getting erased. This is what i had mentioned in my earlier posts.

12. Some errors in code lines seems to be erasing files.

My doubt is how did it erase the ASCII files in another drive folder and also the csv file in root folder. One more thng is that today there was only one ACC file. So there must be problems with batch processing also.


Any way you can find out the reason. Sorry for the constant trouble. This was my 8th Installation of your SW. I keep coming back to it. Attached outputs.

Anyway my 5th try during this letter. Default settings everything working properly. Change destination folder. Now no stockwise or datewise output and csv file in root folder erased.


Regards

Saji

 

[saji oommen]

Hello Pankaj

Here is the correct converted data using default settings.

Saji

 

[pkjha30]

Hi saji

Input folder needs to have bhav copies. If you use eoddlc to download bhavcopies then you need not change the source folder.That is the reason all boxex were empty.
Of course eoddlc did not check that(should have done so normally).

Files in download folders should not have been erased. I will go through it.

Since the settings were apparently changed(due to empty boxes) the program rewrote the files and hence earlier data were erased.(Needsto work on that.)

Thanks for trying it. I will certainly address these issues.Meanwhile I will suggest using defaults.

pankaj