Free tick data available for Amibroker, Ninjatrader and more

[meheustbruno]

Alternatively, if you are looking to import bar data into Ninjatrader (such as 1-minute bars), according to the Ninjatrader manual (http://www.ninjatrader.com/support/help ... orting.htm), NT bar data supports the following format:

yyyyMMdd HHmmss;open price;high price;low price;close price;volume

To achieve this in Tickstory, you can export with the following settings:

Timeframe: 1-Minute
Include headers: No
Custom Format: {BarBeginTime:yyyyMMdd} {BarBeginTime:HHmmss};{Open};{High};{Low};{Close};{Volume}

Please let us know how you go - we can add this as a pre-defined option to make things easier in the future.

Regards.

Hi there, thank you for this great project and what was achieved

Would you be kind enough to include this NinjaTrader Bar format (that works for minute based intervals) as a pre-defined setting on next release ? Thanks in advance

[tickstory]

Hi meheustbruno,

No problem - we've added this item to go into the next release.

Kind regards.

[jollydragon]

How about OANDA and ThinkForex?

[tickstory]

Hi Jollydragon,

If you're referring to acquiring data from these platforms, we're investigating other sources for quality data. Ideally we would like to create a data import routine for MT4 which should cover all supporting brokers such as ThinkForex.

Regards.

[ratsh]

Hi Ratsh,

Many thanks for your comprehensive post. We are aware of the fact that Amibroker is very flexible and can handle different formats. In the future we hope to create a data provider that can feed Amibroker through its plug-in architecture. Until then, we are trying to "support" Amibroker by making it as simple as possible for users to import data with the minimal amount of effort (that is, using Amibroker's default import definitions where possible). Of course, more advanced users will find your information highly invaluable.

Thanks!

Any news on the bold one?

[tickstory]

Hi Ratsh,

Unfortunately no news on this one yet. We will know more once we release our next major version of Tickstory. Thanks for staying tuned!

Regards.

[Threshold]

Longterm AmiBroker user speaking here:

FYI AmiBroker is not dependent on any predefined fix format to get data imported to it. There is no such thing as "fix format". AB works with any format defined. It also provides millisecond data which is not even covered in those two predefined formats. In short those two formats supposed to work with Amiquote are just two of thousands possibilities as there are in addition any other standard data file extensions supported also. Also you don't need Amiquote to get data files imported since you can use any custom data plugin or OLE import script or the two manual file import options in File menu of AB. This is just meant to clarify to remove misunderstanding of how AB works. It's not like MT or NT who only work with fix datafile formats. It is highly flexible. So saying this or that 3rd party software now supports Amibroker for data import sounds quite funny to insiders.

Side note: aqi, aqd, aqh,aqfe other Amiquote file extensions are just standard comma separated ascii data files also and are readable with any texteditor. So there is no mystery about them and no hidden protection.

Hope this clears up a few tings.


##################################################################################################################
Here is one (out of many) examples showing how to import bid/ask price and volume with millisecond timestamp to AmiBroker at once.
##################################################################################################################

In Tickstory choose:

Timeframe: Tick
Timezone: whatever you need
Output format: Custom

Check include header

Header info:
Timestamp,Bid price,Bid volume,Ask Price,Ask volume

Data format (timestamp is again just one of many possiblities):
{Timestamp:yyyyMMdd HH:mm:ss.fff},{BidPrice},{BidVolume},{AskPrice},{AskVolume}

File name: for example use {Symbol}.csv

and start downloading

##################################################################################################################
With notepad++ create a file with following content (this is a "to be done just one time" procedure for each format you need):
##################################################################################################################

Code: Select all

$FORMAT Date_YMD,Time,Close,Volume,Aux1,Aux2
$SKIPLINES 1
$SEPARATOR ", "
$CONT 1
$GROUP 255
$AUTOADD 1
$DEBUG 1
$TICKMODE 1
$TIMESHIFT 0


and save it with file extension .format (for example TickstoryBidAsk.format) and copy that file to ...\AmiBroker\Formats subfolder

Note:
Timshift line is set to zero but if you want to shift to other time zone during import then for example change to $TIMESHIFT 2 which would shift the time stamp by +two hours while $TIMESHIFT -4 would shift by 4 hours backwards (for example if you want import different timezone as the one you have set in tickstory download. Of course you need to take care of summer/winter time, so you would have to split data file if summer/winter time range is part of same file and need to adjust $TIMESHIFT line for summer and winter time import. Anyway $TIMESHIFT is not needed if you are happy with downloaded timezone. Just showing that it's possible to change time zone after download also.)

Go to same subfolder AmiBroker\Formats, open import.types file placed in there and add line (for example):
TickstoryTickBidAsk (*.csv)|*.csv|TickstoryBidAsk.format
(you can see that the format file info at the end of the line needs to have same file name as you have saved your format file to before)

save that import.types file

Now you are finished with preparations.
Open AmiBroker, create a Tickstory database, go to File --> Import Ascii, choose file format "TickstoryTickBidAsk" in drop down menu and browse to your tickstory download folder to open your previously downloaded Bid/Ask data file(s) separated or all at once (by marking them).

Of course you can use programmatic import too but that's another short story.

One thing to do while creating your Database and being related to upper example is setting data source to local database since it is off-line data and setting base time interval to tick there. In addition go to File --> Database settings --> Intraday settings of our database and set Aux1 to Last and Aux2 to Sum (since latter one is ask volume). Also set start minutes to "00" from "30" to start hourly TF at 00.
see picture

I followed your instructions. Importation goes well, no errors. Takes some time, but its obviously because of the size of these tick files which average atleast around 10gigs each. I imported just 1 for a test. Worked. So to save it, I shut down Amibroker, which took a long, laggy time, about 10 minutes or so to shut down on my pc with 4 cores @ 4ghz and 16 gigs of ram. Then I restarted the program, start up took equally long with similar lag. Then I tried to pull up the chart and this seems to be an impossible task. Memeory usage skyrockets to max, program becomes unresponsive, pc lags. I let this go on for about to hour to see if it would load eventually and then had to end process.

Do you not experience any issues with your tick files? I download the fill length, 2003-now for EURUSD. That was my test file. I thought Amibroker was even better than MT4 @ handling data, backtesting, and optimizing. It seems I'm unable to even pull up a chart.

Thoughts?

[tickstory]

Hi Threshold,

I'll add a brief comment on this as you're probably going to get much better answers to this on the Amibroker forum. In my experience, you will need to break up your data into manageable periods. Amibroker will attempt to evaluate any formulas on the chart in one go, meaning that if you are displaying tick data it will traverse the entire dataset before it plots anything. Keeping your datasets to a minimum is the best way to reduce lag/plot times and also assessing whether it is meaningful to be plotting tick data within Amibroker in the first place. That is, are you actually using this data to make trading decisions or are you actually using 15-min candles, 144-ticks, etc? If so, adjust your import granularity to suit - import only 1-minute candles, for example, if you're working in intraday bar time-frames.

Hope this helps.

[ratsh]

I followed your instructions. Importation goes well, no errors. Takes some time, but its obviously because of the size of these tick files which average atleast around 10gigs each. I imported just 1 for a test. Worked. So to save it, I shut down Amibroker, which took a long, laggy time, about 10 minutes or so to shut down on my pc with 4 cores @ 4ghz and 16 gigs of ram. Then I restarted the program, start up took equally long with similar lag. Then I tried to pull up the chart and this seems to be an impossible task. Memeory usage skyrockets to max, program becomes unresponsive, pc lags. I let this go on for about to hour to see if it would load eventually and then had to end process.

Do you not experience any issues with your tick files? I download the fill length, 2003-now for EURUSD. That was my test file. I thought Amibroker was even better than MT4 @ handling data, backtesting, and optimizing. It seems I'm unable to even pull up a chart.

Thoughts?

First of all make sure that you don't use cracked 32-bit software from 20 years ago downloaded from some warez website but make sure to use most recent 64-bit (!!) software from official AmiBroker website (vrs. 6.00 is most recent official one). Also read this AmiBroker performance guide https://www.amibroker.com/guide/x_performance.html from top to bottom. Here is another one on multithreading https://www.amibroker.com/guide/h_multithreading.html

And FYI, AB database is in-RAM database system http://en.wikipedia.org/wiki/In-memory_database
Hedgefunds use similar technology i.e. StreamBase or kdb+ or their own solutions to slurp off data of disk once and then process in-memory to have as little I/O as possible.
So it's high performance data base system.

In short the only limit is the hardware.

And yes, AB is miles ahead of MT4.

For any other problem go to their support channel at support@amibroker.com

[tickstory]

Hi all,

A new video is available that describes how to export Tickstory data and import into Amibroker:

https://www.youtube.com/watch?v=T2WDg2sBInQ

Enjoy!