Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts MS

[TracerBullet]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

It is not easy.

1. No api from NEST to get data for past.
So, Instead you will have to get it from VWAP statistics / Chart data table manually. josh was trying to get it automatically using Autohotkey like tool.

2. It wont be easy to find and fill gaps in the data. Need to study AB api + figure out how to identify gaps.
It would be simpler to just get all data for today and import it. Even then you will probably need to delete todays data ( if any ) to avoid duplicate volume issues.

I only follow 2 scrips which makes it easier.

 

[shubh03]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Dear Josh Sir

it works very well

but one problem

it save data in tick form while i need in 1min form

so pls suggest how i do it ?

 

[josh1]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Dear Josh Sir

it works very well

but one problem

it save data in tick form while i need in 1min form

so pls suggest how i do it ?

Open Settings.ini and change Bar Period to 60000. However, the application will update Amibroker after completion of each minute.

 

[TracerBullet]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Open Settings.ini and change Bar Period to 60000. However, the application will update Amibroker after completion of each minute.

i think he was maybe using excel.

Can you attach sample VWAP + Chart data from NOW. Put it in a file as it is.

 

[josh1]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

If possible, add links to the program in your main page as options for NEST/NOW. thanks

Main Page cannot be edited. Added it in my signature.:thumb::thumb::thumb:

 

[TracerBullet]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Main Page cannot be edited. Added it in my signature.:thumb::thumb::thumb:

The dropbox link was for temporary build but i wont remove it. It does not have backfill.

Once i fix backfill with NOW, i will release new veriosn in github with binary as well. Please change to it then.thanks.

 

[josh1]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Backfill does not have any advantage over excel, so it shouldn't matter much.

Anyway backfill code is very simple. If you learn some basic c++ you can do it too.
Once this issue is fixed, ill stop monitoring this thread. If you ever find a bug pm me and ill check it in free time.

It is not easy.

1. No api from NEST to get data for past.
So, Instead you will have to get it from VWAP statistics / Chart data table manually. josh was trying to get it automatically using Autohotkey like tool.

2. It wont be easy to find and fill gaps in the data. Need to study AB api + figure out how to identify gaps.
It would be simpler to just get all data for today and import it. Even then you will probably need to delete todays data ( if any ) to avoid duplicate volume issues.

I only follow 2 scrips which makes it easier.

i think he was maybe using excel.

Can you attach sample VWAP + Chart data from NOW. Put it in a file as it is.

Open Settings.ini and change Bar Period to 60000. However, the application will update Amibroker after completion of each minute.

Do not bother about Backfill. I am halfway thru it. Will do it soon.

After changing Bar period to 60000, I found that NestRTD sends candle data to Ami only once a minute. I wanted it create 1 min bar internally but send to Ami every second and refresh. The time stamp should be in HH:mm format. For example -
Say first tick collected is at 9:15:00. O, H, L, C will all be equal LTP since it is first tick of the minute. Whatever ticks collected during one minute will modify H or L and C will always be equal to LTP.
The same bar will be sent to Ami each second with time Stamp 9:15. When Ami sees, same time stamp each second, it will keep overwriting earlier candle and refresh.
At 9:16:00 O,H,L,C will again be equal to LTP. Ami will create next candle since time stamp will be 9:16.
If you get time, please do this.
Edit- The advantage is that, there will be 375 quotes in Ami per day. Deleting these before backfill thru COM is faster. If there are 375*60 quotes, in database, it takes much longer time to delete.

 

[TracerBullet]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Do not bother about Backfill. I am halfway thru it. Will do it soon.

Do you mean its working or that you are fixing it for now.

After changing Bar period to 60000, I found that NestRTD sends candle data to Ami only once a minute. I wanted it create 1 min bar internally but send to Ami every second and refresh. The time stamp should be in HH:mm format. For example -
Say first tick collected is at 9:15:00. O, H, L, C will all be equal LTP since it is first tick of the minute. Whatever ticks collected during one minute will modify H or L and C will always be equal to LTP.
The same bar will be sent to Ami each second with time Stamp 9:15. When Ami sees, same time stamp each second, it will keep overwriting earlier candle and refresh.
At 9:16:00 O,H,L,C will again be equal to LTP. Ami will create next candle since time stamp will be 9:16.
If you get time, please do this.
Edit- The advantage is that, there will be 375 quotes in Ami per day. Deleting these before backfill thru COM is faster. If there are 375*60 quotes, in database, it takes much longer time to delete.

Yes, Once a month i purge the data and replace it with 1min data from another source to keep db small.

But,
1. I will start with intraday trading looking at 3 min charts. I need 15 second charts as well ( ex - to see volume impact ).

2. I think we can have a configuration for this to allow minimum time resolution to hh:mm.

But in this case we will have to change code a bit. Currently the O/H/L values are reset after sending a bar. This will have to changed to reset only after LTT changes ( hh:mm ). If LTT does not change, just set the O/H/L/V of current bar but dont change the previous bar.

I will try to add this in free time but if it takes too long will stop (shouldnt).
I think the change should be simple. Lets see if i can do it today.

You can try to do it yourself too. I want people to change and fix code rather than just me. Changes can be merged with git code.

1. Worker:: processRTDData() - only take hh::mm in ltt
2. Worker:: amibrokerPoller() - Remove 2nd if check. Instead move it to the last two lines in the loop to avoid reset.
3. Need to read extra setting to toggle behaviour in (1).
4. In settings.ini remove seconds from TimeFormat for scrips with no ltt. We will use this as our toggle (3)
I think thats it.

Its possible i may add a bug in volume / candle price with this. Please test it properly - prefereably run both old and new version for entire day (with different alias for same scrip) and make sure that the 1 min candles price/volume match

This will be the last change i make in this application for next few months
Everything else will go to TODO untill i start trading profitably and have free time.
If you add something / Fix a bug send it to me with pm.

Edit
3. Are you sure deleting is slow? if i delete from quote editor, its pretty fast.
Also please make sure, you auto backfill tool will delete second bars too even if slow.

 

[shubh03]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Open Settings.ini and change Bar Period to 60000. However, the application will update Amibroker after completion of each minute.

i get data from TT

so where to find setting.ini

 

[TracerBullet]

Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

I tried it. it works but there is a problem that cannot be easily solved. so i have abandoned it for now.

I have two threads. Thread A maintains O/H/L/C of each scrip.
Thread B takes current bar data, sends it to AB and resets O/H/L/C for next bar
Two threads are used so that AB calls dont block getting data from NEST.

Problem - If we use hh::mm and send data to AB every second, Thread B will have to do O/H/L/C reset only after LTT changes ( ie after minute changes ) to maintain correct O/H/L/C for the minute. Previously we did it on every AB call.

This change will cause us to loose some data after minute change
why? - say Thread B sends data at 10:00:58 and then 10:00:59. Because minute is still 00 it wont reset O/H/L/C.

Then we get 3 ticks from Nest, 1 of which is say at 10:01:01. All three ticks are read and set to O/H/L/C by Thread A with LTT at 10:01. When Thread B reads this data, it sees that LTT changed, so it will send the data till 10:00:59 + the 3 ticks to AB and then reset O/H/L/C for next minute.

So the first candle update for 10:01 will actually be duplicate of 10:00 + 3 ticks. Next update in 10:01 will overwrite this first candle update.

Solution will need invasive changes that i dont want to do now. Maybe in a few months

1. Option 1 is for thread A to call thread B synchronously whenever LTT changes to next minute. This will still be complicated as we have more than 1 scrips only some of which may have LTT changed to next minute.
Anyway this will block thread A which i dont want

2. Option 2 is for thread A to not maintain O/H/L/C. Instead it collects all ticks in an array for each scrip and thread B constructs O/H/L/C and differentates between 10:00 and 10:01
This is cleaner. It will probably need more cpu time but thats ok.

I hope you understood. You can look at code and suggest simpler ways.
Alternative - Create an AB scrip that compresses its database to 1min (if possible)

1 positive outcome is i found a bug. Index scrips have no ltt and for those the bar overwrite problem i think still happens. This should be simple fix.