Encog Performance

Performance is a frequently discussed topic on the Encog forums. Machine Learning software can consume considerable CPU resources. Because of this it is important for a machine learning method to be implemented as efficiently as possible. This page is meant to be a collection of my own performance measurements. Here I compare Encog platforms, as well as compare Encog to other machine learning frameworks.

Contents

 [hide] 

• 1 Notes on Third Party Frameworks
• 2 Benchmark Used
• 3 Benchmark Results
• 4 Benchmarks Done by Others
• 5 Notes & Conclusions
• 6 Downloads

Notes on Third Party Frameworks

Because this page contains comparisons between Encog and other third-party machine learning frameworks I provide as much detail as possible about the tests I performed. I want this to be as fair as possible. If anyone sees anything in my implementations that could improve the score of one framework, please let me know! The same thing goes for cross-language comparisons. I have no particular allegiance to Java, C# or C++. I've spent many hours programming in each. I want each Encog platform to be as fast as it possibly can be.

All third party tests were done with a JAR/DLL compiled by the creators of the framework.

Benchmark Used

This benchmark preforms a backpropagation train of a feedforward neural network with the following specs.

• Input Layer Size: 10
• Output Layer Size: 1
• Training Iterations: 50
• Hidden Layers: 1 (with 20 neurons)
• Training Elements: 100,000
• Activation Function: Sigmoid Activation Function

The following platforms were used:

• Java: jdk1.6.0_25
• .Net: Microsoft Visual Studio 2010 w/ .Net 4
• OS: Microsoft Windows 7 (64-bit)
• Hardware: i7 QuadCore (more information at Jeff's Computers

All tests were run from the command line using release builds. All tests were run in 64-bit mode.

Benchmark Results

This section shows the benchmark results sorted from fastest to slowest.

Benchmark Speed Results (lower is better)

Framework	Platform	Result
Encog 3.0 Multi-Threaded	Java-WIN64	2,661ms
Encog 3.0 Multi-Threaded	C#-WIN64	5,156ms
Encog 3.0 Single-Threaded	Java-WIN64	8,693ms
Encog 3.0 Single-Threaded	C#-WIN64	17,591ms
AForge 2.1.5	C#-WIN64	22,785ms

Benchmarks Done by Others

These are third party tests not performed by me (Jeff Heaton).

• Compares the speed of Encog, JOONE & Neuroph
• Compares the speed of Encog & Mathlab

Notes & Conclusions

I began this comparison to evaluate the performance between Encog Java and Encog .Net. I frequently get questions from people with no language bias. They simply want the fastest platform. Until performing these tests, I was not sure. Clearly it seems to be that Java has the best performance. I would like to tighten up the performance between Encog Java and Encog C#. At first I was concerned that perhaps I was doing something terribly wrong with C#. Which is why I through AForge into the mix. AForge is a popular C# AI framework that has been around for much longer than Encog. I expected that AForge would beat Encog C# by as much as the Java version did. Which would have clearly indicated I, and the other Enocg C# programmers/translators, had something new to learn about C# performance. However, this does not seem to be the case. Encog is considerably faster than AForge both in single and multi-threaded mode. AForge does not have a multi-threaded training mode, so I added the Encog single-threaded stat to provide a more level comparison.

I hope to compare to other platforms as well. Two that are on the list are Weka and FANN. I expect FANN to be faster than Encog, as FANN is written in C code. Multi-threaded may help, but I am not sure at this point.

Downloads

• Download this Benchmark