MetaTrader History Data Importing and Converting Tutorial for Quality Backtesting
Backtesting MetaTrader expert advisors on historical data is a good way to test a strategy. But testing on the limited data supplied with MT4 installation gives a very poor quality of testing (usually below 50%). So, how to achieve a 90% quality in backtesting of MT4 expert advisors? It’s not that hard really, just follow this simple tutorial and you will able to test any MetaTrader EA.
1. Download a new MetaTrader 4 installation and install it to a separate folder. Be advised that a good MetaTrader historical data take up a lot of disk space, so install it to a drive with at least 1-2 Gbytes of free space. For download MT4 installation I recommend official MetaQuotes site.
2. After you install your MT4, don’t run it yet. Go to the folder you have installed it to. Go the ‘history’ folder there and delete all folders there (the actual folders names inside the ‘history’ folder may be different from those on these pictures):
3. Run the newly installed MetaTrader and open a free demo account with whatever broker it will be offering to you (official MetaQuotes installation offers random brokers for demo account opening in its MT4). Login to your account and close all open charts (if any).
4. Adjust the MetaTrader options to allow larger charts (more bars in history):
The close your MetaTrader.
5. Download a quality historical data for every currency pair you want to test your expert advisor on. I recommend Alpari History Center for this – download M1 data for each of the currency pairs you need.
6. Unzip the archive with M1 historical data somewhere. Rename it to comply with MT4 standard naming – i.e. EUR/USD M1 chart should be named EURUSD1.hst, GBP/USD M1 should be renamed to GBPUSD1.hst, etc.
7. Copy the renamed history file to your MetaTrader’s ‘history’ folder into your broker’s folder (overwrite if prompted):
8. Launch MetaTrader 4 again. Don’t open any charts on-line, open your downloaded M1 chart offline (example shown for EUR/USD chart):
9. It might take some time for the actual chart to load, but it’s OK. Now you need to convert M1 history to every timeframe you will need. For this you need to use a period_converter script which is a standard feature of all MT4 platforms. Run it from the Scripts tab and set its ExtPeriodMultiplier input parameter to the number of minutes in the desired output timeframe. For M5 enter 5, for M15 enter 15, for M30 enter 30, for H1 enter 60, for H4 enter 240, for D1 enter 1440, for W1 enter 10080, MN isn’t convertible as there is no fixed amount of minutes in one month:
10. Wait for the script to finish its work (may take up some time; it’s more than 2 years of data after all):
11. Repeat steps 8-10 for all timeframes you want to get.
12. Repeat steps 5-11 for all currency pairs you want to test your MT4 EAs on.
13. Voilà! Now you can enjoy a 90% quality backtesting and test every expert advisor you want on the time periods of more than 2 years. Good luck with your trading!
Tags: MetaTrader, MT4, tutorial
January 5th, 2008 at 1:52 pm
very good manual! I have a queston.When I converted the 1H timeframe to 2H timeframe.How to choice 2H timeframe in strategy tester?I can select m1,m5,m15,m30,H1,4 daily but the 2H timeframe is not present in startegy tester,but i converted it and i see when i opening chart offline.
Sorry for my english .
Thanks
January 5th, 2008 at 2:12 pm
First, thanks for using this tutorial. Second, it’s better create all timeframes from M1 timeframe. Third, unfortunately, there is no H2 timeframe in MT4, so you must use one of the standard timeframes…
January 5th, 2008 at 2:27 pm
Thanks for your answer.
I interesing the visual mode in strategy tester ,while i see what happening on every each tick .When i runing expert advisor on 1H timeframe i dont see what happening in my custom timeframe.Have you any idea what can i do ,to fix this problem?
Thanks
January 5th, 2008 at 3:09 pm
Sorry, I don’t use the visualization in strategy tester. Maybe there are no orders to draw?
January 5th, 2008 at 3:33 pm
When i using the strategy tester i must select a timeframe(I can not select my custom timeframe)
I think so ,the only way that i see redrawing the chartis the visual mode.
January 5th, 2008 at 5:07 pm
I have a new problem:) I try to open the 1M history date but in chart windows write me with big font WAIT,and not happening anything .I make all as your manual .
What i do wrong?
January 5th, 2008 at 5:25 pm
Did you rename it as mentioned in the tutorial? Did you copy it to the proper history folder?
February 20th, 2008 at 12:06 pm
how can i put take profit in my software. pls
February 20th, 2008 at 12:32 pm
kingsley, What software are you using?
May 20th, 2008 at 7:16 am
databank.alpari.org is down at the present
May 20th, 2008 at 8:24 am
William, I hope it will get back up soon.
June 18th, 2008 at 11:45 am
thanks a lot for this tutorial
)
last version of metatrader propose wti chart! also brent chart. unfortunatly, the fistory for these new charts (and other new charts) is very poor. i’m looking on the web where i could find a web site, as Alpari History Center who provide these datas. if i find one out i will post it here
thanks again
August 17th, 2008 at 4:03 pm
has some one been able to find another history center? alpari databank is still down, pls HELP !
August 20th, 2008 at 8:15 pm
Yo! I tried Alpari 1-Min data, it’s real bad, too much missing 1-min bars!!!
jb
September 7th, 2008 at 8:46 am
Great! These info are what I need! Thanks. However, I believe alpari databank is still down.
September 19th, 2008 at 5:36 pm
Alpari databank is up now…enjoy!!
November 16th, 2008 at 9:20 pm
Thanks for the great and detailed info. However, I am unable to download the history zip file from Alpari download center. I can\’ download any of them. Does anybody know an alternative download site? Also, does anybody notice that when testing with Meta quotes history that the longer back you go to test an EA the more losing trades the tester says are made? When testing for only a month back you get more accurate results. Two months back it gives you less accurate results, 3 months back even less accurate,etc….. And the farther back you do the testing the bigger the time gaps are that the EA doesn’t trade! Any help will be appreciated. Thanks!
November 26th, 2008 at 2:27 am
This info is very useful, but I can’t download history data from alpari data center, it’s looks like they are down or there is a reason that the data is only for members ?
I get the word “Forbidden”, any other history data center ?
Thanks…
January 6th, 2009 at 8:20 am
How long should backtesting take? I’m running my first try at it now. Trying to run on data from 1/1/2008 thru 12/31/2006 on M30. It’s been running for 14 hours and appears to be only about 3/4 done. Running on Windows XP, Pentium 4, 2.20 GHz, 1 GB of RAM. If it runs this slowly, on 1 test, can’t imagine how long an optimisation would take.
January 6th, 2009 at 10:16 am
That depends on the EA you are testing and the amount of output it produces to the Experts log. Usually, it takes so long only if there are many error messages are generated.
March 30th, 2009 at 5:28 am
Thanks for a great tutorial. I’ve been spending so much time recently to setup MT4 with good testing data and your guide helped me to understand this better. However, after following all steps I’m getting 82.79% modelling quality, and there are numerous “unmatched data eror” messages when starting the test. Why so? I downloaded M1 data from Alpari and converted all timeframes. Is this because Alpari data is not of high qualifty? My EA works on a daily timeframe. It is really flustrating to setup all of this properly. I have an impression that MetaQuotes is not interested in improving this aspect of their software. Thanks.
March 30th, 2009 at 8:17 am
Lukasz,
“Unmatched data” errors happen because of the errors in the initial M1 data. Try using some other period or just don’t mind those errors if there are not many of them they won’t influence the results much.
April 1st, 2009 at 12:15 am
Andrei, but I’m getting the same error even if the M1 data is correct. I explained this in more detail on MQL4 Forum:
http://forum.mql4.com/20934#154236
April 1st, 2009 at 3:58 am
Please see this also:
http://forum.mql4.com/20973#154246
April 1st, 2009 at 9:13 am
Lukasz,
On what timeframe do you receive these “mismatched” errors? If it’s not M1, where did you get your data for the higher timeframes? Also, which Model for testing do you use?
April 2nd, 2009 at 12:40 am
Andrei,
I’m testing my EA on a D1 time frame (every tick model). I found out that there is a bigger issue (checked build 222 and most recent 223) – the modeling quality is not constant for the same test. It is not repeatable. Please see my recent post:
http://forum.mql4.com/20973
April 2nd, 2009 at 7:19 am
Lukasz,
Actually, when you run the tester, it tries to download some data. I am not sure when it does and when it doesn’t but in my experience if I have all M1 data for the period and the higher timeframes are converted from this M1 data there is no difference in the subsequent runs if I explicitly set a period.
April 3rd, 2009 at 1:15 am
Andrei,
I’n my subsequent tests I verified that even though the data is complete for the tested period, I’m still getting “n/a” for the modeling quality. Can you tell me which build of MT4 are you using and where you obtained the data? If you’re using Alpari, can you please try running a test from 2001.01.01 till 2009.01.01 on EUR/USD (very tick, D1 time frame)? I’m thinking about getting data from MetaStock. Thanks.
April 3rd, 2009 at 4:28 am
Andrei,
I found out that tester downloads EURUSD1.hst file again from the broker (this is one of the two files that got updated under the MT4_HOME/history/.) EURUSD1440.hst got also updated but the file size is the same. I even narrowed the testing period to 2003.01.01 – 2008.01.01. This is the problem. I guess this happens when testing period is D1.
April 3rd, 2009 at 4:44 am
I explained here how to reproduce:
http://forum.mql4.com/21059
April 3rd, 2009 at 5:33 am
Andrei,
I finally solved this mistery. Please see my followup to the above initial post. Basically, every icon in the history center must be double clicked on to get these higher timeframes recalculated properly. What a relief.
April 3rd, 2009 at 2:29 pm
Actually, I take this back. The tester still downloads data from the broker which causes mismatched data errors. I will try to download, then delete broker’s account to see if this prevents downloading any data.
April 30th, 2009 at 7:19 pm
How do I convert Daily data to get Monthly data?
April 30th, 2009 at 7:22 pm
You should convert Minute data to get Monthly. But I think using 30 as multiplier for the script will convert daily to monthly.
May 14th, 2009 at 3:12 am
Has anyone found a site for EURUSD data as Alapri seems to be down?
May 17th, 2009 at 9:28 pm
Hello,
Alpari datacenter doen not work for me.
I?ve found other sites to download historyc data, but I don?t know how to setup the download settings.
Also I don?t know how to transform .csv or .txt files into .hst files.
I tried to put the link to the 2 sites but the blog system does not allw me to put their urls.
Best Regards
May 17th, 2009 at 9:52 pm
www fin-rus com / analysis / export / default.asp
www dukascopy net / english / ddf_main / rdata / ExportCSV /
These are the urls of my previous post.
Can anyone help me with this?.
how to configure to correctly download the historic data in order to use it with mt4?
also, how to transform csv or txt files into .hst files?
Thanks in advance.
May 27th, 2009 at 8:38 am
Here is another source for MT4 historical data:
http://www.onecorpfx.com/historical_data.php