Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts MS

Status
Not open for further replies.

Raghuveer

Well-Known Member
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Hey guys....!! I am new to all this...
I have meddled with the settings in both RTDMan and Autobackfill.... Done everything as instructed in the video.
The backfill is ready but do I get the tick data during the market hours....??
If everything is done as per the two help files (Setup Instructions.docx and RTDMAN Tutorial.pdf) you will get the same data as in NEST/NOW in real time.
Make sure your broker has enabled RTD for your account, if not send email to them requesting RTD.

If by "tick data" you mean the real tick by tick data, you will not get it.

If there are any problems:
-post versions of software you are using
-post images of terminal, DOS window of RTDMan, Windows date settings, Amibroker settings etc (the more the better)
-post settings.ini in CODE tags

To rule out any obvious mistakes you can post these things on Sunday itself.
 
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

If everything is done as per the two help files (Setup Instructions.docx and RTDMAN Tutorial.pdf) you will get the same data as in NEST/NOW in real time.
Make sure your broker has enabled RTD for your account, if not send email to them requesting RTD.

If by "tick data" you mean the real tick by tick data, you will not get it.

If there are any problems:
-post versions of software you are using
-post images of terminal, DOS window of RTDMan, Windows date settings, Amibroker settings etc (the more the better)
-post settings.ini in CODE tags

To rule out any obvious mistakes you can post these things on Sunday itself.
Thank you...:)

I was able to get the back fill on AmiBroker and was able to verify that the charts on Nest and Amibroker somehow match...:-D . So seems my whole meddling according to the video instructions has worked....
Apart from that, I guess only the live market will tell me how the RTD setup will be right... And I do not know whether RTD is enabled, I will mail Zerodha right away....

Since I won't be getting a tick data, how does the chart update. Do I need to keep refreshing. That way it will be harder on 5 minutes time frames. The chart settings on Amibroker is set to 1 minute while in the video it was set to 1 tick. I downloaded Amibroker from their site and it seems I guess it's a trial version or I don't know...
Please do let me know....
Thank You guys again....
 

Raghuveer

Well-Known Member
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

My Ami is not trial and I do not know how trial version behaves. Ami downloaded from Amibroker site will be trial version.

Set Ami db interval to tick.
Ami will auto update chart, no need to do any "refreshing".
Forget "tick data". Ami will get whatever data NEST/NOW will be getting.

Backfill is easier and entirely different from RTDMan. It's possible to have backfill working and RTDMan not working.

Thank you...:)

I was able to get the back fill on AmiBroker and was able to verify that the charts on Nest and Amibroker somehow match...:-D . So seems my whole meddling according to the video instructions has worked....
Apart from that, I guess only the live market will tell me how the RTD setup will be right... And I do not know whether RTD is enabled, I will mail Zerodha right away....

Since I won't be getting a tick data, how does the chart update. Do I need to keep refreshing. That way it will be harder on 5 minutes time frames. The chart settings on Amibroker is set to 1 minute while in the video it was set to 1 tick. I downloaded Amibroker from their site and it seems I guess it's a trial version or I don't know...
Please do let me know....
Thank You guys again....
 
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Josh, I am using Angel Broking ODIN. Can you please help in understanding if your utility can be used to extract data from ODIN to Amibroker? Also which file to be downloaded for the same?

Thanks..
Josh / Seniors,
Please help me in pointing the setup files for ODIN to Ami.
 

Raghuveer

Well-Known Member
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Josh / Seniors,
Please help me in pointing the setup files for ODIN to Ami.
Download "RTNOW-2014-07-04.zip". It has "Setup Instructions.docx", go through it.
 
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

My Ami is not trial and I do not know how trial version behaves. Ami downloaded from Amibroker site will be trial version.

Set Ami db interval to tick.
Ami will auto update chart, no need to do any "refreshing".
Forget "tick data". Ami will get whatever data NEST/NOW will be getting.

Backfill is easier and entirely different from RTDMan. It's possible to have backfill working and RTDMan not working.
So, how do I verify if RTDMan is working...?
The dos did show that it was connected to all the scrips that I saved in the settings file in the RTDMan. And I guess since it's a trial version, I will have a month of time as leverage before I get to any sort of deadlines. In this span, I guess Zerodha should have their software neat and clean. I am hoping only to get through till that time with a reliable charting tool other than nest.
 
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

@ Raghuveer and Josh

I did everything with the RTDMan.... The backfill is working and the ticks are coming in too...

But I have a problem, the ticks come separately in a window. The autofill works on the main Amibroker that I opened.
So basically when I open RTDMan, it opens an extra window which shows all the ticks and following this, if I run the backfill, the backfill happens on the main Amibroker I wont see them on the window opened by RTDMan.... Its not possible to trade with just todays data coming in tick by tick.... But in the instructions video, I see the price variations happening on the backfilled chart as well....

My datatable is filled will all the data for nearly 27 charts until Friday close...

How do I solve this?...
 

Raghuveer

Well-Known Member
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Some one had a doubt about the correct way to start RTDMan and Amibroker.

Default for settings.ini is to comment out database path. Keep it that way unless you are sure something else is better.
To see/change what is the default database path for Ami: Tools->Data->Default database path

Start in this order for RTDMan:
1. Start NEST/NOW.
2. Start RTDMan. RTDMan will auto open Ami. Do not start Ami manually.

Make sure there is only one instance of Ami open. All data will be saved to this Ami (default database path).

Close in this order:
1. Save Ami. Immediately close RTDMan dos window. Ami will be auto closed. If this done after market hours Ami will always auto close immediately. If this is done during market hours OR if there are a lot of scrips to be saved then Ami may take some minutes to auto close. Wait for auto close of Ami. (In very rare case manually close Ami).
2. Close NEST/NOW.

Backfill:
If Ami is already opened either manually or by RTDMan: Backfill will use that Ami database path to save.
If Ami is not already open it will be auto opened with default database path.
 
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Some one had a doubt about the correct way to start RTDMan and Amibroker.

Default for settings.ini is to comment out database path. Keep it that way unless you are sure something else is better.
To see/change what is the default database path for Ami: Tools->Data->Default database path

Start in this order for RTDMan:
1. Start NEST/NOW.
2. Start RTDMan. RTDMan will auto open Ami. Do not start Ami manually.

Make sure there is only one instance of Ami open. All data will be saved to this Ami (default database path).

Close in this order:
1. Save Ami. Immediately close RTDMan dos window. Ami will be auto closed. If this done after market hours Ami will always auto close immediately. If this is done during market hours OR if there are a lot of scrips to be saved then Ami may take some minutes to auto close. Wait for auto close of Ami. (In very rare case manually close Ami).
2. Close NEST/NOW.

Backfill:
If Ami is already opened either manually or by RTDMan: Backfill will use that Ami database path to save.
If Ami is not already open it will be auto opened with default database path.
I have observed that if I let RTDMan start the Amibroker instance, then at the time of stopping RTDMan, it crashes the Nest for me..

I have the default Ami Database as "Data - EOD"... and have edited the settings for RTDMan, to use "Data - Nest" as the Ami Database.. I always start Amibroker myself, switch the database to "Data - Nest" and then begin RTDMan... This works perfectly fine for me... Also helps me to start another Ami instance and view the EOD data as and when I wish and close that instance as well... without any problems.. :)
 
Re: Free RealTime Data NOW Nest ODIN Trade Tiger Google Yahoo to AmiBroker, Fcharts M

Hi Satish ji.. I have edited NestVwapBackFill_0.5.au3 to take care of some of the annoyances I observed...

Edit on lines 318-325 is related to backfill using Nest Plus. For me, the utility had consecutively pushed data to wrong scrips in Ami once.. Hence, adding an additional check of presence of Trading Symbol text in DataTable Window title..

Edit on lines 410-415 is to provide further delay when no data is found using VWAP Statistics... I regularly face the issue, hence the need...

Code:
; #RequireAdmin
#cs
	Developed by Josh1

	Purpose - to backfill quotes automatically from Nest/Zerodha Trader
	from Market watch window and Hourly Statistics
	30th June, 2014

#ce


Global $Title = IniRead(@WorkingDir & "\NestVwap.ini", "NestTrader", "Title", "NEST Trader 3")
Global $DataDir = IniRead(@WorkingDir & "\NestVwap.ini", "NestTrader", "CSVFolder", "R:\")
Global $FNO = IniRead(@WorkingDir & "\NestVwap.ini", "NestTrader", "Future", "")
Global $TSymbol = -1, $Exchange = -1, $Series = -1, $Instrument = -1, $Symbol = -1, $Optype = -1, $Strike = -1, $iListView, $hiListView, $hgui

#Region ............ Included Scripts
#include-once
#include <GUIConstants.au3>
#include <Array.au3>
#include <GuiListView.au3>
#include <Date.au3>
#include <DTC.au3>
#include <file.au3>
#EndRegion ............ Included Scripts

#Region ............ Options settings
Opt('WinWaitDelay', 100)
Opt('WinDetectHiddenText', 1)
Opt('MouseCoordMode', 0)
Opt("SendKeyDelay", 15)
AutoItSetOption("WinTitleMatchMode", 2)
Opt("GUIOnEventMode", 1) ; Change to OnEvent mode
#EndRegion ............ Options settings

_Gui1()

Func _Gui1()
	#Region ### START Koda GUI section ### Form=
	If Not $hgui Then
		Global $hgui = GUICreate("Nest Vwap Backfill", 365, 440, 800, 90, BitOR($GUI_SS_DEFAULT_GUI, $WS_MAXIMIZEBOX, $WS_SIZEBOX))

		;#comments-start
		$settingsmenu = GUICtrlCreateMenu("&Settings")
		;	GUICtrlSetState(-1, $GUI_DISABLE)
		$helpmenu = GUICtrlCreateMenu("Help")
		;	GUICtrlSetState(-1, $GUI_DISABLE)
		$infoitem = GUICtrlCreateMenuItem("Info", $helpmenu)
		GUICtrlSetOnEvent($infoitem, "_OnInfo")
		$settingitem = GUICtrlCreateMenuItem("Settings", $settingsmenu)
		GUICtrlSetOnEvent($settingitem, "_OnSettings")
		GUICtrlCreateMenuItem("", $settingsmenu, 2) ; create a separator line
		;#ce
		Local $h = 10
		Local $v = 5
		Local $vh = 18

		GUISetFont(8, 400, 0, "MS Sans Serif")
		GUISetOnEvent($GUI_EVENT_CLOSE, "CLOSEButton")

		$Label5 = GUICtrlCreateLabel("Backfill Gap from", $h, $v + 5, 80, 18)
		Global $T1 = GUICtrlCreateDate(_NowCalcDate() & " 09:15:00", $h + 85, $v + 5, 70, 18, $DTS_SHORTDATEFORMAT)
		$Label6 = GUICtrlCreateLabel("  Up To", $h + 160, $v + 5, 50, 18)
		Global $T2 = GUICtrlCreateDate(_NowCalc(), $h + 210, $v + 5, 70, 18, $DTS_SHORTDATEFORMAT) ;$DTS_TIMEFORMAT

		GUICtrlCreateGroup("Backfill from -", $h, $v + 25, 300, 20)
		Global $Npy = GUICtrlCreateRadio("NEST Plus", 150, $v + 25, 70, 18)
		Global $Npn = GUICtrlCreateRadio("VWAP/Stat", 250, $v + 25, 50, 18)

		;	GUICtrlSetState($Npn, $GUI_CHECKED)
		GUICtrlSetOnEvent($Npy, "_OnPlusbackfill")
		GUICtrlSetOnEvent($Npn, "_OnPlusbackfill")

		; to select a specific default format
		$DTM_SETFORMAT_ = 0x1032 ; $DTM_SETFORMATW
		$style = "HH:mm:ss"
		GUICtrlSendMsg($T1, $DTM_SETFORMAT_, 0, $style)
		GUICtrlSendMsg($T2, $DTM_SETFORMAT_, 0, $style)
		GUICtrlSetResizing(-1, $GUI_DOCKBORDERS)

		GUICtrlSetResizing(-1, $GUI_DOCKRIGHT + $GUI_DOCKSIZE + $GUI_DOCKBOTTOM)

		$Button1 = GUICtrlCreateButton("Select All", $h, $v + 380, 90, 25)
		$Button2 = GUICtrlCreateButton("DeSelect All", $h + 100, $v + 380, 90, 25)
		$Button3 = GUICtrlCreateButton("Do Backfill", $h + 210, $v + 380, 105, 25)
		GUICtrlSetOnEvent($Button1, "_SelectAll")
		GUICtrlSetOnEvent($Button2, "_DeselectAll")
		GUICtrlSetOnEvent($Button3, "_On_DoBackfill_Click")

		$Label7 = GUICtrlCreateLabel("Date from", $h, $v + 45, 50, 20)
		Global $D1 = GUICtrlCreateDate(_NowCalc(), $h + 55, $v + 45, 90, 18, $DTS_SHORTDATEFORMAT)
		$Label8 = GUICtrlCreateLabel("To", $h + 155, $v + 45, 50, 20)
		Global $D2 = GUICtrlCreateDate(_NowCalc(), $h + 200, $v + 45, 90, 18, $DTS_SHORTDATEFORMAT)
		$style1 = "yyyy/MM/dd"
		GUICtrlSendMsg($D1, $DTM_SETFORMAT_, 0, $style1)
		GUICtrlSendMsg($D2, $DTM_SETFORMAT_, 0, $style1)
		ControlDisable($hgui, "", $D1)
		ControlDisable($hgui, "", $D2)

		GUISetState()
	EndIf
	WinSetOnTop($hgui, "", 1)
	#EndRegion ### END Koda GUI section ###

	#Region .............. Set Title for Nest main window and get its handle if open
	If Not WinExists($Title) Then
		MsgBox(262144, "Nest Error", "Is Nest Trader Open? " & @CRLF & "If yes- Please Copy title of Nest Trader Window Title" & @CRLF & "it should match exact upto version number  ")
		_OnSettings()
	Else
		WinActivate($Title)
		Global $hWnd = WinGetHandle($Title) ; Get handle for NOW main window
		Global $hWatch = ControlGetHandle($hWnd, "", 1003) ;Get handle of Market Watch Control
		Global $hStat = 9999
	EndIf
	#EndRegion .............. Set Title for Nest main window and get its handle if open
EndFunc   ;==>_Gui1
While 1

	Sleep(100)
WEnd


Func _FillList()

	If Not WinExists($Title) Then
		MsgBox(262144, "Nest Error", "Is Nest Trader Open? " & @CRLF & "Open it before this application ... Exiting ")
		_OnSettings()
	EndIf

	_Check_MarketWatchColumns($hWatch)
	Global $MwatchScrips = _Get_Scrips_from_Mwatch($hWatch)
	;_ArrayDisplay($MwatchScrips,"Scrips found")

	Global $iRows = UBound($MwatchScrips, 1)
	Global $iCols = UBound($MwatchScrips, 2)
	If $iListView Then GUICtrlDelete($iListView)

	$iListView = GUICtrlCreateListView("Select Symbols to backfill", 10, 75, 350, 300); Created Listview to display and select scrips
	$hiListView = GUICtrlGetHandle($iListView) ; obtained handle for the listview
	_GUICtrlListView_SetExtendedListViewStyle($hiListView, BitOR($LVS_EX_FULLROWSELECT, $LVS_EX_CHECKBOXES), $LVSCW_AUTOSIZE)


	;_GUICtrlListView_InsertColumn($hiListView, 0, "Symbol ", 150)
	For $i = 0 To $iRows - 1 ; Fill Listview with Marketwatch scrips
		GUICtrlCreateListViewItem($MwatchScrips[$i][$TSymbol], $iListView)
	Next
	_GUICtrlListView_InsertColumn($hiListView, 2, "Progress", 95)
	_GUICtrlListView_InsertColumn($hiListView, 3, "Time", 95)

EndFunc   ;==>_FillList

Func _On_DoBackfill_Click()

	Global $Plusbackfill = ControlCommand($hgui, "", $Npy, "IsChecked")
	;	Get scrips selected for backup from hourly Statistics
	$Scrips = _Select_Scrips_from_Mwatch($hWatch)
	_Get_Text_from_VWAP_Statistics($Scrips)
	WinClose($hStat)
	;MsgBox (64,"","Updation completed :-)")

EndFunc   ;==>_On_DoBackfill_Click

Func _Select_Scrips_from_Mwatch($hListView)
	Local $aReturn = 0
	Local $aReturn[$iRows]
	;Copy selected scrips indices instead of deleting unselected scrips
	Local $size = 0
	For $j = 0 To ($iRows - 1) Step +1
		If _GUICtrlListView_GetItemChecked($hiListView, $j) = True Then
			$aReturn[$size] = $j
			$size = $size + 1;
		EndIf
	Next

	ReDim $aReturn[$size]
	Return SetError(Number($aReturn[0] = 0), 0, $aReturn)
EndFunc   ;==>_Select_Scrips_from_Mwatch

Func _Get_Scrips_from_Mwatch($hListView)
	Local $iItemCount = _GUICtrlListView_GetItemCount($hListView), $i, $j
	Local $iColumnCount = _GUICtrlListView_GetColumnCount($hListView)
	;MsgBox(64,"Items in Mktwatch","Itemcount- " & $iItemCount & "Columns - " & $iColumnCount)
	Local $aReturn[$iItemCount][$iColumnCount] ;= [[$iItemCount, $iColumnCount]]
	For $i = 0 To $iItemCount - 1
		;ControlListView($hWnd,"",1003,"Select", $i)
		For $j = 0 To $iColumnCount - 1
			;$aReturn[$i][$j] = _GUICtrlListView_GetItemText($hListView, $i, $j)
			$aReturn[$i][$j] = ControlListView($hWnd, "", 1003, "GetText", $i, $j)
		Next
		;ControlListView($hWnd,"",1003,"DeSelect", $i)
	Next
	;_ArrayDisplay($aReturn,"$areturn from MktWatch")
	Return SetError(Number($aReturn[0][0] = 0), 0, $aReturn)
EndFunc   ;==>_Get_Scrips_from_Mwatch

Func _Check_MarketWatchColumns($hListView)

	Local $iColumnCount = _GUICtrlListView_GetColumnCount($hListView)

	Local $aColumns[] = [], $aReturn[6]
	For $i = 0 To $iColumnCount - 1
		$aColumns = _GUICtrlListView_GetColumn($hListView, $i)
		Local $Name = $aColumns[5]

		Switch $Name
			Case "Trading Symbol"
				$TSymbol = $i
			Case "Exchange"
				$Exchange = $i
			Case "Instrument Name"
				$Instrument = $i
			Case "Symbol"
				$Symbol = $i
			Case "Option Type"
				$Optype = $i
			Case "Strike Price"
				$Strike = $i
			Case "Series/Expiry"
				$Series = $i
		EndSwitch

	Next

	If $TSymbol < 0 Then
		MsgBox(64, "Columns", "Trading Symbol must be present Market Watch ")
		Exit
	ElseIf $Exchange < 0 Then
		MsgBox(64, "Columns", "Exchange must be present Market Watch ")
		Exit
	ElseIf $Instrument < 0 Then
		MsgBox(64, "Columns", "Instrument Name must be present Market Watch ")
		Exit
	ElseIf $Symbol < 0 Then
		MsgBox(64, "Columns", "Symbol must be present Market Watch ")
		Exit
	ElseIf $Optype < 0 Then
		MsgBox(64, "Columns", "Option Type must be present Market Watch ")
		Exit
	ElseIf $Strike < 0 Then
		MsgBox(64, "Columns", "Strike Price must be present Market Watch ")
		Exit
	ElseIf $Series < 0 Then
		MsgBox(64, "Columns", "Series/Expiry must be present Market Watch ")
		Exit
	EndIf

	;_ArrayDisplay($aReturn,"hahaha")
EndFunc   ;==>_Check_MarketWatchColumns


Func _Get_Text_from_VWAP_Statistics(ByRef $Scrips)
	;_ArrayDisplay($Scrips,"$Scrip Selected")
	; Find total rows in Mktwatch
	Local $iItemCount = ControlListView($hWnd, "", 1003, "GetItemCount")

	;	Iterate selections for each selected scrip
	For $i = 0 To UBound($Scrips, 1) - 1
		ControlListView($hWnd, "", 1003, "SelectClear")
		Sleep(1000)

		For $iRows = 0 To $iItemCount - 1 ;search scrip through all rows of Mktwatch
			; Get TradingSymbol from each row
			$TempScrip = ControlListView($hWnd, "", 1003, "GetText", $iRows, $TSymbol)

			Local $index = $Scrips[$i];

			If $MwatchScrips[$index][$TSymbol] = $TempScrip Then

				If Not $Plusbackfill Then
					If $MwatchScrips[$index][$Exchange] = "MCX" Then
						_GUICtrlListView_SetItemText($hiListView, $iRows, "MCX - Use Plus", 1)
						_GUICtrlListView_SetItemText($hiListView, $iRows, "Backfill", 2)
						ExitLoop 2
					EndIf
					_GUICtrlListView_SetItemText($hiListView, $iRows, "Opening Vwap", 1)
				Else
					_GUICtrlListView_SetItemText($hiListView, $iRows, "Opening DataTable", 1)
				EndIf

				For $tr = 1 To 5
					_GUICtrlListView_SetItemText($hiListView, $iRows, "Try-" & $tr, 1)
					_GUICtrlListView_SetItemFocused($hWatch, $iRows)

					$Str = $MwatchScrips[$i][$TSymbol]
					$Occur = ControlListView($hWnd, "", 1003, "FindItem", $Str)

					ControlListView($hWnd, "", 1003, "Select", $Occur)

					If Not $Plusbackfill Then
						; Workaround - This removes the failures on 1st try for VWAP stats
						;_GUICtrlListView_ClickItem($hWatch,$iRows,"right", true )			; Without this line - If we Click on MW and then backfill - VWAP stats does not get focus
						ControlClick($hWnd, "", 1003, "Right") ; No need to right click on row as we will set comboboxes too later

						ControlSend($hWnd, "", 1003, "tv")
						Sleep(1000)
						$hStat = WinWaitActive("[Title:Vwap Statistics; Class:#32770]", "", 5) ;Wait to open Vwap Statistitics and get handle

						If ($hStat <> 9999 Or $hStat <> 0) Then
							If WinExists($hStat, "") Then
								WinActivate($hStat, "")
								WinSetOnTop($hStat, "", 1)
								_GUICtrlListView_SetItemText($hiListView, $iRows, "Got it", 1)
								ExitLoop
							Else
								$hStat = 9999
							EndIf
						ElseIf $tr = 5 Then
							MsgBox(64, "Vwap Backfill", "Vwap window refused to open ... Exiting" & @CRLF & @CRLF _
									 & "Check whether opens manually, ...Start again or try other scrip")
							Exit
						EndIf

					Else
						_GUICtrlListView_ClickItem($hWatch, $iRows, "right", True) ; ZT - This fails to launch menu on first try of first scrip everytime
						Sleep(500)
						ControlSend($hWnd, "", 1003, "pd")

						#cs - Aditya
						Using Nest Plus, sometimes backfill is done for wrong scrip. Hence, adding additional requirement of text of Trading Symbol in Title window
						as it is available in every DataTable window.

						Original line:
						;$hStat = WinWaitActive("[Title:DataTable; Class:#32770]", "", 10) ; Wait to open Data Table and get handle
						#ce - Aditya
						$hStat = WinWaitActive("[Title:DataTable : " & $TempScrip & "; Class:#32770]", "", 10) ; Wait to open Data Table and get handle

						If ($hStat <> 9999 Or $hStat <> 0) Then

							If WinExists($hStat, "") Then
								WinActivate($hStat, "")
								WinSetOnTop($hStat, "", 1)
								_GUICtrlListView_SetItemText($hiListView, $iRows, "Got it", 1)
								ExitLoop
							Else
								$hStat = 9999
							EndIf

						ElseIf $tr = 5 Then
							MsgBox(64, "NestPlus Backfill", "Data Table refused to open ... Exiting" & @CRLF & @CRLF _
									 & "Check whether opens manually, ...Start again or try other scrip")
							Exit
						EndIf

					EndIf
				Next

				If Not $Plusbackfill Then
					WinActivate($hStat, "")

					; Select Combo box options - ZT didnt select them
					ControlCommand($hStat, "", "ComboBox1", 'SelectString', $MwatchScrips[$index][$Exchange]) ; Exchange

					; Series = Intrument for NFO and = Series for NSE Equity
					If ControlCommand($hStat, "", "ComboBox2", 'FindString', $MwatchScrips[$index][$Instrument]) <> 0 Then
						ControlCommand($hStat, "", "ComboBox2", 'SelectString', $MwatchScrips[$index][$Instrument]) ; Instrument
					Else
						ControlCommand($hStat, "", "ComboBox2", 'SelectString', $MwatchScrips[$index][$Series]) ; Series Expiry
					EndIf

					ControlCommand($hStat, "", "ComboBox3", 'SelectString', $MwatchScrips[$index][$Symbol]) ; Symbol

					If ControlCommand($hStat, "", "ComboBox4", 'IsEnabled', "") <> 0 Then
						ControlCommand($hStat, "", "ComboBox4", 'SelectString', $MwatchScrips[$index][$Series]) ; Expiry Date
					EndIf
					If ControlCommand($hStat, "", "ComboBox5", 'IsEnabled', "") <> 0 Then
						ControlCommand($hStat, "", "ComboBox5", 'SelectString', $MwatchScrips[$index][$Optype]) ; Opt Type
					EndIf
					If ControlCommand($hStat, "", "ComboBox6", 'IsEnabled', "") <> 0 Then
						ControlCommand($hStat, "", "ComboBox6", 'SelectString', $MwatchScrips[$index][$Strike]) ; Strike Price
					EndIf

					ControlSetText($hStat, "", "Edit1", "1")
					Sleep(1000)
					;WinActivate($hStat,"")
					ControlFocus($hStat, "", "[CLASS:Button; INSTANCE:1]")
					ControlClick($hStat, "", "[CLASS:Button; INSTANCE:1]", "Left", 1) ;Click on 'Get Statistics' Button
					Sleep(4000)
				EndIf

				; Get handle of ListView /Data and check column order
				Local $hStLV = ControlGetHandle($hStat, "", "[Class:SysListView32; INSTANCE:1]")
				#cs
					Local $hStLVColumns [6] = ["Start Time","Open Rate","High Rate","Low Rate","Close Rate","Differential Vol"]
					Local $aColumns = ""
					For $iCols = 0 To 5
					$aColumns = _GUICtrlListView_GetColumn($hStLV, $iCols)
					If $aColumns[5] <> $hStLVColumns[$iCols] Then
					MsgBox(64,"VWAP Statistics Columns Order","Column - " & $i+1 & " is- " & $aColumns[5] & @LF _
					& " It should be - " & $hStLVColumns[$iCols])
					Exit
					EndIf
					Next
				#ce
				Local $Sname

				If $MwatchScrips[$index][$Instrument] = "" Then ;Change Scripname
					$Sname = $MwatchScrips[$index][$Symbol]
				ElseIf StringInStr($MwatchScrips[$index][$Instrument], "FUT") And ($FNO <> "") Then
					$Sname = $MwatchScrips[$index][$Symbol] & $FNO
				Else
					$Sname = $MwatchScrips[$index][$TSymbol]
				EndIf

				_GUICtrlListView_SetItemText($hiListView, $iRows, $Sname, 0)
				_GUICtrlListView_SetItemText($hiListView, $iRows, "GettingData", 1)
				Global $pos = $iRows
				ControlFocus($hStat, "", "[CLASS:SysListView32; INSTANCE:1]")
				Sleep(1000)

				;Edit start - Aditya -- Additional delay when the data takes a while to populate in VWAP Statistics
				Local $STLVitemcount = _GUICtrlListView_GetItemCount($hStLV)
				If $STLVitemcount = 0 Then ; If there is no data in Hourly Stat/Datatable then provide further delay
					_GUICtrlListView_SetItemText($hiListView, $iRows, "Further delay...", 1)
					Sleep(8000)
				;Edit end - Aditya
					Local $STLVitemcount = _GUICtrlListView_GetItemCount($hStLV)
					If $STLVitemcount = 0 Then ; If there is no data in Hourly Stat/Datatable then exit
						_GUICtrlListView_SetItemText($hiListView, $iRows, "No Data found", 1)
						WinClose($hStat)
					EndIf
				EndIf

				;	Start Collecting Data from Hourly Statistics
				If $i = 0 Then
					MsgBox(262144, "Set Backfill Time", "Set Time Gap to be filled and Press OK")

					If Not $Plusbackfill Then
						Global $DayDeleteFrom = _Date_Time_Convert(_NowCalcDate() & GUICtrlRead($T1), "yyyy/MM/ddHH:mm:ss", "yyyyMMddHHmmss")
						Global $DayDeleteUpto = _Date_Time_Convert(_NowCalcDate() & GUICtrlRead($T2), "yyyy/MM/ddHH:mm:ss", "yyyyMMddHHmmss")
					Else
						Global $DayDeleteFrom = _Date_Time_Convert(GUICtrlRead($D1) & GUICtrlRead($T1), "yyyy/MM/ddHH:mm:ss", "yyyyMMddHHmmss")
						Global $DayDeleteUpto = _Date_Time_Convert(GUICtrlRead($D2) & GUICtrlRead($T2), "yyyy/MM/ddHH:mm:ss", "yyyyMMddHHmmss")
						;MsgBox(64, "Dates", "From = " & $DayDeletefrom & " To = " & $DayDeleteUpto)
					EndIf

				EndIf

				Local $T = ""
				_GUICtrlListView_SetItemText($hiListView, $pos, "Processing", 2)
				$ts = TimerInit()

				For $j = 0 To $STLVitemcount - 1
					$sText = ""

					If Not $Plusbackfill Then

						For $k = 0 To 5

							$sText &= ControlListView($hStat, "", "[CLASS:SysListView32; INSTANCE:1]", "GetText", $j, $k)
							;				  	$sText &= _GUICtrlListView_GetItemText($hStLV, $j,$k)

							If $k = 0 Then ;Convert 12 Hour to 24 Hour format
								Local $Hour = Int(StringLeft($sText, "2"))

								If ($Hour <> 12 And StringRight($sText, 2) = "PM") Then
									$Hour = $Hour + 12
									$sText = $Hour & StringMid($sText, 3, 6)
								Else
									$sText = StringLeft($sText, 8)
								EndIf

								$Checktime = @YEAR & @MON & @MDAY & StringReplace(StringLeft($sText, 8), ":", "")
								If Not (($Checktime) >= ($DayDeleteFrom) And ($Checktime) <= ($DayDeleteUpto)) Then
									;_GUICtrlListView_SetItemText($hiListView,$pos,$DayDeleteFrom,1)
									;_GUICtrlListView_SetItemText($hiListView,$pos,$DayDeleteUpto,2)
									ContinueLoop 2
								EndIf

							EndIf

							$sText &= ","
						Next

						$T &= $Sname & "," & @YEAR & @MON & @MDAY & "," & $sText & @CRLF ;Format CSV data
					Else

						For $k = 1 To 6
							$sText1 = ControlListView($hStat, "", "[CLASS:SysListView32; INSTANCE:1]", "GetText", $j, $k)
							;				  	$sText1 = _GUICtrlListView_GetItemText($hStLV, $j,$k)

							If $k = 1 Then ;Split Date Time into separate fields and format them to yyyymmdd,HHmmss
								$sText1 = StringRegExpReplace($sText1, '(\d{2})-(\d{2})-(\d{4}) (\d{2}):(\d{2}):(\d{2})', '$3$2$1,$4$5$6')
								$Checktime = StringReplace($sText1, ",", "") ;

								If Not (($Checktime) >= ($DayDeleteFrom) And ($Checktime) <= ($DayDeleteUpto)) Then
									;_GUICtrlListView_SetItemText($hiListView,$pos,$Checktime,1)
									;_GUICtrlListView_SetItemText($hiListView,$pos,$DayDeleteUpto,2)
									ContinueLoop 2
								Else
									;_GUICtrlListView_SetItemText($hiListView,$pos,$Checktime,2)
								EndIf

							EndIf
							$sText &= $sText1 & ","
							; _GUICtrlListView_SetItemText($hiListView,$pos,$sText,1)
						Next

						$T &= $Sname & "," & $sText & @CRLF ;Format CSV data
					EndIf

				Next

			EndIf

		Next

		$td = Int(TimerDiff($ts)) / 1000
		$td &= " seconds"
		_GUICtrlListView_SetItemText($hiListView, $pos, $td, 2)
		_GUICtrlListView_SetItemText($hiListView, $pos, "Data retrieved", 1)

		FileDelete($DataDir & $Sname & ".txt")
		$Filename = FileOpen($DataDir & $Sname & ".txt", 1)
		FileWrite($Filename, $T)
		FileClose($Filename)
		_DeleteTics($Sname)
		WinClose($hStat)
		$hStat = 9999
	Next

	MsgBox(64, "Backfill", "Backfill of scrips completed")
EndFunc   ;==>_Get_Text_from_VWAP_Statistics

Func _DeleteTics($Scrip)
	Local $Filename = $DataDir & $Scrip & ".txt"
	Local $oAB = ObjCreate("Broker.Application")
	$oAB.Visible = 1
	If StringRight($DayDeleteFrom, 6) <> 000000 Then
		$oStocks = $oAB.Stocks
		Local $Qty = $oStocks.Count
		Local $i = 0
		Local $j = 0
		Local $k = 0

		For $i = 0 To ($Qty - 1)
			$Stock = $oStocks($i)
			;      $k = 0
			Local $ScripName = $Stock.Ticker
			$QCount = $Stock.Quotations.Count
			$Quotes = $Stock.Quotations
			If $ScripName = $Scrip Then
				_GUICtrlListView_SetItemText($hiListView, $pos, "Deleting pr.quotes", 1)
				$ts = TimerInit()
				;			MsgBox(0,"","Removing quotes " & $Scrip)
				For $j = ($QCount - 1) To 0 Step -1
					$Quote = $Quotes($j)
					$tmpDateNum = $Quote.Date

					If (($tmpDateNum >= $DayDeleteFrom) And ($tmpDateNum <= $DayDeleteUpto)) Then
						;						$k+=1
						;_GUICtrlListView_SetItemText($hiListView,$pos,$tmpDateNum,1)
						;_GUICtrlListView_SetItemText($hiListView,$pos,$DayDeleteFrom,2)
						$Quotes.Remove($j)
					ElseIf ($tmpDateNum < $DayDeleteFrom) Then
						ExitLoop
					EndIf
				Next
				$td = Int(TimerDiff($ts)) / 1000
				$td &= " seconds"
				;_GUICtrlListView_SetItemText($hiListView,$pos,$td,2)
			EndIf
		Next

		With $oAB
			.Import(0, $Filename, "rtd.format")
			.RefreshAll
			.SaveDatabase
		EndWith

	Else

		Local $sString = FileReadToArray($Filename)
		Local $aString = StringSplit($sString, @CRLF, 2)

		$aRet = _ArraySortInOrder($sString)
		;	If Not @error Then _ArrayDisplay($aRet, "Sorted Array")
		_FileWriteFromArray($Filename, $aRet, @CRLF)

		With $oAB
			.Import(0, $Filename, "rtd2.format")
			.RefreshAll
			.SaveDatabase
		EndWith

	EndIf
	_GUICtrlListView_SetItemText($hiListView, $pos, "Done", 1)

EndFunc   ;==>_DeleteTics


Func _ArraySortInOrder($aStr)
	; Description - Sorts an array or a space separated string in the order of the characters listed in a given string.
	; Parameters:
	;       $aStr - The array, or the space separated string to be sorted.
	;       $sAlphabetOrder - A string containing the sorting order of the characters which appear in the array or string.
	; Requirement: #include <Array.au3>
	; Returns: A sorted array.
	;

	Local $sAlphabetOrder = "0123456789abcdefghijklmnopqrstuwxyz"
	$sAlphabetOrder = StringStripWS($sAlphabetOrder, 8) ; Cannot sort on a space.
	Local $aAlpha = StringSplit($sAlphabetOrder, "", 2)

	;--- Convert the array ($aStr) to a string or convert the string ($aStr) to an array.----
	Local $sToSort
	If IsArray($aStr) Then
		For $i = 0 To UBound($aStr) - 1
			$sToSort &= $aStr[$i] & " "
		Next
		$sToSort = StringTrimRight($sToSort, 1) ; Remove trailing space.
	ElseIf IsString($aStr) Then
		$sToSort = StringStripWS($aStr, 7)
		$aStr = StringSplit($sToSort, " ", 2)
	Else
		Return SetError(0, 0, 1)
	EndIf
	;--->End of Convert array_string.----

	; Convert all characters in $sToSort to $sAlphabetOrder characters.
	; Note; "#>" was randomly chosen to prevent previous converted characters to be converted again.
	For $i = 0 To UBound($aAlpha) - 1
		$sToSort = StringRegExpReplace($sToSort, $aAlpha[$i] & "(?!\d{0,2}#>)", StringRight("00" & $i, 3) & "#>")
	Next

	;Create 2D array with $sAlphabetOrder character words in column 0, and given words in the next column.
	Local $aToSort = StringSplit($sToSort, " ", 2)
	Local $aArr[UBound($aStr)][2]

	For $i = 0 To UBound($aStr) - 1
		$aArr[$i][0] = ($aToSort[$i])
		$aArr[$i][1] = $aStr[$i]
	Next
	;--->End of Create 2D array -------------------------------------------------------------

	_ArraySort($aArr) ; Sort on column 0

	; Copy the 2nd column of the 2D array into a 1D array.
	For $i = 0 To UBound($aStr) - 1
		$aStr[$i] = $aArr[$i][1]
	Next

	Return $aStr
EndFunc   ;==>_ArraySortInOrder


Func _SelectAll()
	For $i = 0 To $iRows - 1
		_GUICtrlListView_SetItemChecked($hiListView, $i)
	Next
EndFunc   ;==>_SelectAll

Func _DeSelectAll()
	For $i = 0 To $iRows - 1
		_GUICtrlListView_SetItemChecked($hiListView, $i, 0)
	Next
EndFunc   ;==>_DeSelectAll

Func _OnPlusbackfill()
	Local $PlusState = ControlCommand($hgui, "", $Npy, "IsChecked")
	If $PlusState Then
		ControlEnable($hgui, "", $D1)
		ControlEnable($hgui, "", $D2)
	Else
		GUICtrlSetData($D1, _NowCalc())
		GUICtrlSetData($D2, _NowCalc())
		ControlDisable($hgui, "", $D1)
		ControlDisable($hgui, "", $D2)
	EndIf

	_FillList()
EndFunc   ;==>_OnPlusbackfill

Func _OnSettings()
	Global $settingsgui = GUICreate("Backfill Settings", 365, 120, 800, 130, BitOR($GUI_SS_DEFAULT_GUI, $WS_SIZEBOX), "")
	GUISetOnEvent($GUI_EVENT_CLOSE, "CLOSESettings", $settingsgui)
	Local $h = 10
	Local $v = 5
	Local $vh = 18

	GUISetFont(8, 400, 0, "MS Sans Serif")

	$Label1 = GUICtrlCreateLabel("Nest Trader Window Title", $h, $v, 140, $vh)
	Global $Input1 = GUICtrlCreateInput(IniRead(@WorkingDir & "\NestVwap.ini", "NestTrader", "Title", "NEST Trader 3"), $h + 145, $v, 150, $vh)
	;NEST Trader 3.10.57.97.7
	$Label2 = GUICtrlCreateLabel("Temporary CSV Folder Path", $h, $v + 20, 140, 18)
	Global $Input2 = GUICtrlCreateInput(IniRead(@WorkingDir & "\NestVwap.ini", "NestTrader", "CSVFolder", "R:\"), $h + 145, $v + 20, 150, 18)

	$Label3 = GUICtrlCreateLabel("Rename Future as" & @CRLF & "Symbol+  " & "-1/-F1/_F1 etc.", $h, $v + 36, 140, 36)
	Global $Input3 = GUICtrlCreateInput(IniRead(@WorkingDir & "\NestVwap.ini", "NestTrader", "Future", ""), $h + 145, $v + 40, 150, 18)

	$Label4 = GUICtrlCreateLabel("Options and Equity will remain as Trading Symbol", $h, $v + 62, 400, 18)

	$Button1 = GUICtrlCreateButton("Save", $h, $v + 82, 90, 25)
	GUICtrlSetOnEvent($Button1, "_OnSave")

	$Button2 = GUICtrlCreateButton("Exit", $h + 105, $v + 82, 90, 25)
	GUICtrlSetOnEvent($Button2, "CLOSEButton")

	GUISetState()
	WinSetOnTop($settingsgui, "", 1)

EndFunc   ;==>_OnSettings

Func _OnSave()
	Do
		$Title = GUICtrlRead($Input1)
		$DataDir = GUICtrlRead($Input2)
		$FNO = GUICtrlRead($Input3)

		If StringRight($DataDir, 1) <> "\" Then $DataDir &= "\"
		Local $FileExists = FileOpen($DataDir & "MyFile.txt", $FO_CREATEPATH + $FO_OVERWRITE)

		;If Not $FileExists then
		If $FileExists = -1 Then
			MsgBox(64, "CSV folder Path incorrect", " Please give correct CSV folder path and save")
			ExitLoop
		Else
			FileClose($FileExists)
			FileDelete($DataDir & "MyFile.txt")
		EndIf

		;Write defaults to ini file
		IniWrite(@WorkingDir & "\NestVwap.ini", "NestTrader", "Title", $Title)
		IniWrite(@WorkingDir & "\NestVwap.ini", "NestTrader", "CSVFolder", $DataDir)
		IniWrite(@WorkingDir & "\NestVwap.ini", "NestTrader", "Future", $FNO)

		GUIDelete($settingsgui) ; If it was this GUI - we just delete the GUI <<<<<<<<<<<<<<<
		_Gui1()

	Until 1
EndFunc   ;==>_OnSave

Func CLOSEButton()
	Exit
	#cs
		Switch @GUI_WINHANDLE ; See which GUI sent the CLOSE message
		Case $hgui
		Exit ; If it was this GUI - we exit <<<<<<<<<<<<<<<
		Case $settingsgui
		;MsgBox(0,"","Close pressed")
		GUIDelete($settingsgui) ; If it was this GUI - we just delete the GUI <<<<<<<<<<<<<<<

		EndSwitch
	#ce
EndFunc   ;==>CLOSEButton

Func _OnInfo()
	ShellExecute("Readme-Backfill.doc")
EndFunc   ;==>_OnInfo

Func CLOSESettings()

	GUIDelete($settingsgui) ; If it was this GUI - we just delete the GUI <<<<<<<<<<<<<<<
	_Gui1()
EndFunc   ;==>CLOSESettings

I feel there is scope for lots of further improvements, like - the utility shouldn't delete the data if no data is found, better log text to show that there is a problem with specific symbols, etc.. Would share my edits as and when I make them.. :)

I also feel that we could host the code on Github/BitBucket... That would show you the diffs visually, have someone to blame for a particular buggy line, better issue and bug management, pull requests and much more..
 
Status
Not open for further replies.

Similar threads