Performance benchmarks¶

This page is trying to go through some performance metrics and see how better/worse Datafaker is in compare with Java Faker and other similar projects.

Hardware & Software¶

All the tests are done with help of JMH at Fedora 37 for different JDKs (mentioned in tables below). The laptop has 32Gb of RAM, AMD Ryzen 7 PRO 5850U with Radeon Graphics.

Datafaker (1.7.0) vs Datafaker (1.4.0) vs Java Faker (1.0.2) vs Kotlin-faker (1.13.0) vs JFairy (0.6.5)¶

Originally Datafaker started as a fork of java-faker. For java-faker there have been mentions of the poor performance of the library, and this document will compare the performance of the respective libraries against each other.

Recently, the 1.7.0 version of Datafaker has been released, and we'll look into what this says about performance improvements.

To demonstrate the performance, we're going to use JMH benchmarks. All the code is available as a separate project because of JMH license reasons.

Since it's hardly possible and reasonable to compare each method's performance, we'll focus on a subset of features in this article. In the below, you can see that Datafaker is about 10x-100x times faster for several cases.

JDK effects¶

Also, there is an interesting fact: in case of Datafaker moving from jdk8 to jdk19 performance is improving up to 25%, while for Java Faker it is not happening. It seems in Java Faker there are some time-consuming operations blocking such improvements from updating java version which partially were fixed in Datafaker.

There is also Kotlin-faker, which a performance comparison table in README page.

Unfortunately there is not so much information what kind of benchmarks have been executed. After some search across the project I was able to find only this page So, it looks like the only test which were done is checking Faker.name().name() performance.

Ok, let's start with the similar test here. Similar, because we are going to use JMH which was not used in their test. If applicable we try to execute same tests we did for previous section. So let's start with the original test from Kotlin Faker

Original Kotlin Faker Test¶

(for different libs there should be different classes, for more details look in the code): net.datafaker.benchmark.kotlinfakerbenchmark

ATTENTION! In this test we measure not throughput but the average time of one operation => lower is better.

1.7.0 Datafaker looks much faster than others in this test.

Initialization¶

It's worth to measure since initially during initialization of Faker object it requires to initialise all the providers objects and read all the yaml files for providers.

Tests for initialization could be found at net.datafaker.benchmark.initialization Initialization:

Simple methods fullname, firstname, address¶

Performance of simple method calls like fullname, firstname, address.

Tests could be found at net.datafaker.benchmark.simplemethods

Firstname:¶

Fullname:¶

StreetAddress:¶

String template operations¶

From one side Kotlin Faker and JFairy do not support expressions, from the other side Kotlin Faker supports numerify/bothify/letterify and regexify operations.

So, the tests are done based on Kotlin Faker's doc page

Numerify:¶

Letterify:¶

Bothify:¶

Regexify:¶

Expressions¶

Since both Java Faker and Datafaker provide expression functionality like letterify, bothify, regexify, it would make sense to see the difference. Here, it is also worth mentioning that for different patterns score could be different, however it is impossible to check all.

net.datafaker.benchmark.templatestrings.DatafakerTemplateStrings

Bothify expression:¶

Letterify expression:¶

Numerify expression:¶

Regexify expression:¶

Method invocations¶

Method invocations for the case of Datafaker and Javafaker are covered by simple method calls like full name and address since under the hood there are method invocations.

It makes sense to keep in mind that these tests do not cover all possible use cases and could be considered only as a starting point for analysis.