Skip to content
This repository has been archived by the owner on Sep 23, 2021. It is now read-only.

Commit

Permalink
Project Upgrade/Simplification (#12)
Browse files Browse the repository at this point in the history
* Upgrade project

* Uses the auto-generated client instead of a custom one

* Moves PersonServiceHandler moved to server implicits

* Updates README to 20 iter and w-iter instead of 10

* Brings the new benchmark results based on 20 iterations

* Upgrades radar chart

* Removes Thread.sleep. It also:
* Provides a new bar chart diagram.
* Update all the benchmark results
  • Loading branch information
juanpedromoreno authored and raulraja committed Oct 30, 2017
1 parent fdc3d8e commit 92a6069
Show file tree
Hide file tree
Showing 18 changed files with 1,200 additions and 912 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,7 @@ project/plugins/project/
/out/
.idea_modules/

# Benchmark results
bench/*.txt

.DS_Store
1,695 changes: 1,075 additions & 620 deletions BENCHMARK_RESULTS.md

Large diffs are not rendered by default.

71 changes: 40 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ We are using the [Java Microbenchmark Harness (JMH)](http://openjdk.java.net/pro
* `argonaut`, version `6.2`.
* RPC services stack based on:
* `freestyle`, version `0.4.1`.
* `frees-rpc`, version `0.1.1` (atop of [gRPC](https://grpc.io/), version `1.6.1`).
* `frees-rpc`, version `0.1.2` (atop of [gRPC](https://grpc.io/), version `1.6.1`).

### http

Expand All @@ -56,10 +56,10 @@ sbt "http/runMain metrifier.http.server.HttpServer"
* Run Benchmarks:

```bash
sbt "bench/jmh:run -i 10 -wi 10 -f 2 -t 1 metrifier.benchmark.HttpBenchmark"
sbt "bench/jmh:run -o http-benchmark-results.txt -i 20 -wi 20 -f 2 -t 1 metrifier.benchmark.HttpBenchmark"
```

Which means "10 iterations", "10 warmup iterations", "2 forks", "1 thread".
Which means "20 iterations", "20 warmup iterations", "2 forks", "1 thread".

### frees-rpc

Expand All @@ -72,10 +72,10 @@ sbt "frees-rpc/runMain metrifier.rpc.server.RPCServer"
* Run Benchmarks:

```bash
sbt "bench/jmh:run -i 10 -wi 10 -f 2 -t 1 metrifier.benchmark.RPCBenchmark"
sbt "bench/jmh:run -o rpc-benchmark-results.txt -i 20 -wi 20 -f 2 -t 1 metrifier.benchmark.RPCBenchmark"
```

Which means "10 iterations", "10 warmup iterations", "2 forks", "1 thread".
Which means "20 iterations", "20 warmup iterations", "2 forks", "1 thread".

### Benchmark Results

Expand All @@ -97,24 +97,27 @@ Expanded version is in the [BENCHMARK_RESULTS.md](BENCHMARK_RESULTS.md) file.
* HttpBenchmark Raw output:

```bash
[info] # Run complete. Total time: 00:05:23
[info] Benchmark Mode Cnt Score Error Units
[info] HttpBenchmark.createPerson thrpt 20 9.178 ± 0.050 ops/s
[info] HttpBenchmark.getPerson thrpt 20 1680.386 ± 577.062 ops/s
[info] HttpBenchmark.getPersonLinks thrpt 20 1349.666 ± 174.447 ops/s
[info] HttpBenchmark.listPersons thrpt 20 1811.600 ± 400.316 ops/s
[info] HttpBenchmark.programComposition thrpt 20 8.492 ± 0.089 ops/s
# Run complete. Total time: 00:08:52

Benchmark Mode Cnt Score Error Units
HttpBenchmark.createPerson thrpt 40 861.052 ± 106.109 ops/s
HttpBenchmark.getPerson thrpt 40 1569.770 ± 153.491 ops/s
HttpBenchmark.getPersonLinks thrpt 40 1072.277 ± 131.565 ops/s
HttpBenchmark.listPersons thrpt 40 1196.642 ± 153.626 ops/s
HttpBenchmark.programComposition thrpt 40 156.592 ± 14.936 ops/s
```

* RPCBenchmark Raw output:

```bash
[info] # Run complete. Total time: 00:06:06
[info] Benchmark Mode Cnt Score Error Units
[info] RPCBenchmark.createPerson thrpt 20 9.298 ± 0.044 ops/s
[info] RPCBenchmark.getPerson thrpt 20 3544.518 ± 970.798 ops/s
[info] RPCBenchmark.getPersonLinks thrpt 20 2708.222 ± 277.988 ops/s
[info] RPCBenchmark.listPersons thrpt 20 3042.332 ± 340.310 ops/s
[info] RPCBenchmark.programComposition thrpt 20 8.755 ± 0.064 ops/s
# Run complete. Total time: 00:09:32

Benchmark Mode Cnt Score Error Units
RPCBenchmark.createPerson thrpt 40 2127.367 ± 230.981 ops/s
RPCBenchmark.getPerson thrpt 40 2859.026 ± 205.242 ops/s
RPCBenchmark.getPersonLinks thrpt 40 1883.806 ± 127.808 ops/s
RPCBenchmark.listPersons thrpt 40 1762.097 ± 161.918 ops/s
RPCBenchmark.programComposition thrpt 40 272.358 ± 18.218 ops/s
```

##### Summary
Expand All @@ -123,40 +126,46 @@ Expanded version is in the [BENCHMARK_RESULTS.md](BENCHMARK_RESULTS.md) file.

Source | Mode | Cnt | Score | Error | Units
--- | --- | --- | --- | --- | ---
HttpBenchmark.createPerson | thrpt | 20 | 9.178 | 0.050 | ops/s
RPCBenchmark.createPerson | thrpt | 20 | 9.298 | 0.044 | ops/s
HttpBenchmark.createPerson | thrpt | 40 | 861.052 | 106.109 | ops/s
RPCBenchmark.createPerson | thrpt | 40 | 2127.367 | 230.981 | ops/s

###### **getPerson**

Source | Mode | Cnt | Score | Error | Units
--- | --- | --- | --- | --- | ---
HttpBenchmark.getPerson | thrpt | 20 | 1680.386 | 577.062 | ops/s
RPCBenchmark.getPerson | thrpt | 20 | 3544.518 | 970.798 | ops/s
HttpBenchmark.getPerson | thrpt | 40 | 1569.770 | 153.491 | ops/s
RPCBenchmark.getPerson | thrpt | 40 | 2859.026 | 205.242 | ops/s

###### **getPersonLinks**

Source | Mode | Cnt | Score | Error | Units
--- | --- | --- | --- | --- | ---
HttpBenchmark.getPersonLinks | thrpt | 20 | 1349.666 | 174.447 | ops/s
RPCBenchmark.getPersonLinks | thrpt | 20 | 2708.222 | 277.988 | ops/s
HttpBenchmark.getPersonLinks | thrpt | 40 | 1072.277 | 131.565 | ops/s
RPCBenchmark.getPersonLinks | thrpt | 40 | 1883.806 | 127.808 | ops/s

###### **listPersons**

Source | Mode | Cnt | Score | Error | Units
--- | --- | --- | --- | --- | ---
HttpBenchmark.listPersons | thrpt | 20 | 1811.600 | 400.316 | ops/s
RPCBenchmark.listPersons | thrpt | 20 | 3042.332 | 340.310 | ops/s
HttpBenchmark.listPersons | thrpt | 40 | 1196.642 | 153.626 | ops/s
RPCBenchmark.listPersons | thrpt | 40 | 1762.097 | 161.918 | ops/s

###### **programComposition**

Source | Mode | Cnt | Score | Error | Units
--- | --- | --- | --- | --- | ---
HttpBenchmark.programComposition | thrpt | 20 | 8.492 | 0.089 | ops/s
RPCBenchmark.programComposition | thrpt | 20 | 8.755 | 0.064 | ops/s
HttpBenchmark.programComposition | thrpt | 40 | 156.592 | 14.936 | ops/s
RPCBenchmark.programComposition | thrpt | 40 | 272.358 | 18.218 | ops/s

##### Comparing both in Charts

You can find the following charts in [this jsfiddle](http://jsfiddle.net/juanpedromoreno/sjw5jgrj/).

##### Comparing both in a Radar Chart
* Bar Chart
![bar-chart-bench](chart-bar.png)

![bench](radar-chart.png)
* Radar Chart
![radar-chart-bench](chart-radar.png)

#### Conclusion

Expand Down
102 changes: 36 additions & 66 deletions bench/src/main/scala/RPCBenchmark.scala
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
package metrifier
package benchmark

import freestyle._
import freestyle.implicits._
import org.openjdk.jmh.annotations._
import metrifier.rpc.client.RPCClient
import metrifier.shared.model._
import metrifier.benchmark.rpc.implicits._
import java.util.concurrent.TimeUnit

import metrifier.rpc.protocols.PersonService
import monix.eval.Task

import scala.concurrent.duration.Duration
Expand All @@ -19,84 +17,56 @@ import scala.concurrent.Await
@OutputTimeUnit(TimeUnit.SECONDS)
class RPCBenchmark {

@Benchmark
def listPersons: PersonList = {

def program[M[_]](implicit APP: RPCClient[M]): FreeS[M, PersonList] = APP.listPersons

Await.result(program[RPCClient.Op].interpret[Task].runAsync, Duration.Inf)
}
val client: PersonService.Client[Task] = implicitly[PersonService.Client[Task]]
val p5: Person = Person(
id = "5",
name = PersonName(title = "ms", first = "valentine", last = "lacroix"),
gender = "female",
location = Location(
street = "1494 avenue du fort-caire",
city = "orléans",
state = "aveyron",
postCode = 91831),
email = "valentine.lacroix@example.com",
picture = None
)

@Benchmark
def getPerson: Person = {

def program[M[_]](implicit APP: RPCClient[M]): FreeS[M, Person] = APP.getPerson("1")

Await.result(program[RPCClient.Op].interpret[Task].runAsync, Duration.Inf)
}
def listPersons: PersonList = Await.result(client.listPersons("foo").runAsync, Duration.Inf)

@Benchmark
def getPersonLinks: PersonLinkList = {

def program[M[_]](implicit APP: RPCClient[M]): FreeS[M, PersonLinkList] = APP.getPersonLinks("1")

Await.result(program[RPCClient.Op].interpret[Task].runAsync, Duration.Inf)
}
def getPerson: Person = Await.result(client.getPerson("1").runAsync, Duration.Inf)

@Benchmark
def createPerson: Person = {
def getPersonLinks: PersonLinkList =
Await.result(client.getPersonLinks("1").runAsync, Duration.Inf)

def program[M[_]](implicit APP: RPCClient[M]): FreeS[M, Person] =
APP.createPerson(
id = "5",
nameTitle = "ms",
nameFirst = "valentine",
nameLast = "lacroix",
gender = "female",
locationStreet = "1494 avenue du fort-caire",
locationCity = "orléans",
locationState = "aveyron",
locationPostCode = 91831,
email = "valentine.lacroix@example.com",
pictureLarge = None,
pictureMedium = None,
pictureThumbnail = None
)

Await.result(program[RPCClient.Op].interpret[Task].runAsync, Duration.Inf)
}
@Benchmark
def createPerson: Person =
Await.result(
client
.createPerson(p5)
.runAsync,
Duration.Inf
)

@Benchmark
def programComposition: PersonAggregation = {

def program[M[_]](implicit APP: RPCClient[M]): FreeS[M, PersonAggregation] = {
def clientProgram: Task[PersonAggregation] = {
for {
personList <- APP.listPersons
p1 <- APP.getPerson("1")
p2 <- APP.getPerson("2")
p3 <- APP.getPerson("3")
p4 <- APP.getPerson("4")
p1Links <- APP.getPersonLinks(p1.id)
p3Links <- APP.getPersonLinks(p3.id)
pNew <- APP.createPerson(
id = "5",
nameTitle = "ms",
nameFirst = "valentine",
nameLast = "lacroix",
gender = "female",
locationStreet = "1494 avenue du fort-caire",
locationCity = "orléans",
locationState = "aveyron",
locationPostCode = 91831,
email = "valentine.lacroix@example.com",
pictureLarge = None,
pictureMedium = None,
pictureThumbnail = None
)
personList <- client.listPersons("foo")
p1 <- client.getPerson("1")
p2 <- client.getPerson("2")
p3 <- client.getPerson("3")
p4 <- client.getPerson("4")
p1Links <- client.getPersonLinks(p1.id)
p3Links <- client.getPersonLinks(p3.id)
pNew <- client.createPerson(p5)
} yield (p1, p2, p3, p4, p1Links, p3Links, personList.add(pNew))
}

Await.result(program[RPCClient.Op].interpret[Task].runAsync, Duration.Inf)
Await.result(clientProgram.runAsync, Duration.Inf)
}

}
6 changes: 1 addition & 5 deletions bench/src/main/scala/rpc/rpc.scala
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
package metrifier
package benchmark

import cats.implicits._
import freestyle.implicits._
import freestyle.asyncCatsEffect.implicits._
import freestyle.rpc.client._
import freestyle.rpc.client.implicits._
import monix.eval.Task
import io.grpc.ManagedChannel
import metrifier.rpc._
import metrifier.rpc.client._
import metrifier.rpc.protocols._

import scala.concurrent.Future
Expand All @@ -32,9 +31,6 @@ package object rpc {

implicit val personServiceClient: PersonService.Client[Task] =
PersonService.client[Task](channel)

implicit val rpcClientHandler: RPCClientHandler[Task] =
new RPCClientHandler[Task]
}

object implicits extends Implicits
Expand Down
Binary file added chart-bar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added chart-radar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
48 changes: 22 additions & 26 deletions demo/src/main/scala/rpc/RPCDemoApp.scala
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,44 @@ package metrifier
package demo
package rpc

import freestyle._
import metrifier.shared.model._
import metrifier.rpc.client.RPCClient
import metrifier.demo.rpc.implicits._
import metrifier.rpc.protocols.PersonService
import monix.eval.Task

import scala.concurrent.Await
import scala.concurrent.duration._

object RPCDemoApp {

def clientProgram[M[_]](implicit APP: RPCClient[M]): FreeS[M, PersonAggregation] = {
def clientProgram(implicit client: PersonService.Client[Task]): Task[PersonAggregation] = {
for {
personList <- APP.listPersons
p1 <- APP.getPerson("1")
p2 <- APP.getPerson("2")
p3 <- APP.getPerson("3")
p4 <- APP.getPerson("4")
p1Links <- APP.getPersonLinks(p1.id)
p3Links <- APP.getPersonLinks(p3.id)
pNew <- APP.createPerson(
id = "5",
nameTitle = "ms",
nameFirst = "valentine",
nameLast = "lacroix",
gender = "female",
locationStreet = "1494 avenue du fort-caire",
locationCity = "orléans",
locationState = "aveyron",
locationPostCode = 91831,
email = "valentine.lacroix@example.com",
pictureLarge = None,
pictureMedium = None,
pictureThumbnail = None
)
personList <- client.listPersons("foo")
p1 <- client.getPerson("1")
p2 <- client.getPerson("2")
p3 <- client.getPerson("3")
p4 <- client.getPerson("4")
p1Links <- client.getPersonLinks(p1.id)
p3Links <- client.getPersonLinks(p3.id)
pNew <- client.createPerson(Person(
id = "5",
name = PersonName(title = "ms", first = "valentine", last = "lacroix"),
gender = "female",
location = Location(
street = "1494 avenue du fort-caire",
city = "orléans",
state = "aveyron",
postCode = 91831),
email = "valentine.lacroix@example.com",
picture = None
))
} yield (p1, p2, p3, p4, p1Links, p3Links, personList.add(pNew))
}

def main(args: Array[String]): Unit = {

val result: PersonAggregation =
Await.result(clientProgram[RPCClient.Op].interpret[Task].runAsync, Duration.Inf)
Await.result(clientProgram.runAsync, Duration.Inf)

println(s"Result = $result")

Expand Down
5 changes: 1 addition & 4 deletions demo/src/main/scala/rpc/package.scala
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ import freestyle.implicits._
import freestyle.config.implicits._
import freestyle.asyncCatsEffect.implicits._
import freestyle.rpc.client._
import freestyle.rpc.client.implicits._
import monix.eval.Task
import io.grpc.ManagedChannel
import metrifier.rpc._
import metrifier.rpc.client._
import metrifier.rpc.protocols._

import scala.concurrent.Future
Expand Down Expand Up @@ -41,9 +41,6 @@ package object rpc {

implicit val personServiceClient: PersonService.Client[Task] =
PersonService.client[Task](channel)

implicit val rpcClientHandler: RPCClientHandler[Task] =
new RPCClientHandler[Task]
}

object implicits extends Implicits
Expand Down
Loading

0 comments on commit 92a6069

Please sign in to comment.