.Net Core 2.0 Performance Notes Revisited
30 Dec 2017This post is part of the F# Advent Calendar 2017 series. Many thanks to Sergey Tihon for organizing these.
Over the past few weeks I've been submitting improvements to some of the F# programs in the Benchmarks Game. In a previous post I did this for the C# programs.
Since that post things have moved on and C# is currently faster than Java for 8 out of 10 of the programs.
Java is faster for regex-redux as .Net Core doesn't yet have a compiled regex implementation.
For k-nucleotide Java makes use of a dictionary well suited to the program not available to C#.
Most of the submissions to the F# programs were ports of the C# code that had recently been optimised.
For fasta and k-nucleotide further optimisations were discovered.
ArrayPool is very useful in the case of fasta.
For k-nucleotide the largest dictionary can be constructed more efficiently in four parallel parts.
Another tempting optimisation was there being a one to one replacement in F# to use native pointers for arrays e.g. Array.get a i becomes NativePtr.get a i.
This only actually provided a small improvement in most cases and wasn't always done.
I feel I must plug Expecto's Expect.isFasterThan. It's a quick way of checking that one implementation is truly faster than another and has proven invaluable.
Results
C# vs Java, F# vs C#, F# vs Java, F# vs Haskell, F# vs OCaml, F# vs Python
Program |
C# |
F# |
Java |
Haskell |
OCaml |
Python |
|---|---|---|---|---|---|---|
pidigits |
3.03 |
3.05 |
3.12 |
Error |
Error |
3.43 |
reverse-complement |
0.78 |
0.82 |
1.03 |
1.40 |
0.79 |
3.26 |
fannkuch-redux |
14.44 |
16.65 |
17.26 |
15.40 |
16.12 |
565.97 |
binary-trees |
8.26 |
8.54 |
8.34 |
23.66 |
10.03 |
93.55 |
n-body |
21.37 |
22.86 |
22.10 |
21.43 |
21.67 |
838.39 |
mandelbrot |
5.83 |
6.66 |
6.04 |
11.69 |
55.18 |
225.24 |
fasta |
2.09 |
1.67 |
2.33 |
9.36 |
6.00 |
59.47 |
k-nucleotide |
11.47 |
10.43 |
8.70 |
35.01 |
21.63 |
77.65 |
regex-redux |
30.74 |
31.02 |
10.34 |
Error |
24.66 |
15.22 |
spectral-norm |
4.07 |
4.22 |
4.23 |
4.04 |
4.31 |
180.97 |
Conclusion
As mentioned in the previous post there are some caveats to these results. They represent the current state of a set of programs on a specific test machine. However, there is enough evidence for some general conclusions.
The overall results for .Net Core 2.0 are very impressive compared to other managed platforms.
F# performance in the worst case is only 15% behind C#. F# is a high-level language that results in simpler and shorter code. It's good that even in the extreme of a low-level performance benchmark it is not too far behind C#.
F# in fact shows very good performance against Java resulting in a five all draw. This means F# would be expected to perform better than Scala or Kotlin if they were to participate in the benchmarks.
F# looks to have the best performance among the functional languages. This is due to the performance of .Net Core 2.0 and being able to write F# in a functional-first style.
Hopefully 2018 will see continued adoption of both .Net Core 2.0 and F#.
Happy New Year!