ReaderT 101
This blog post is about dependency injection (d.i.) using the Reader monad in Scala. I won’t explain what a monad is nor will I explore any category theory (mostly because I don’t know how to explain any of that). In this post I just want to show the mental model I have when using monadic style with ReaderT.
Note: this post turned out to be quite big. It’s not very dense though! Especially if you’re familiar with Scala you should be able to whisk through most of it.
Dependency injection
Code needs other code. That’s what d.i. is for me. We write separate pieces of code. Often one bit needs to use the other. I’ll use the following example source for this post:
case class Hero(name: String)
// imagine this is a database
case class Ooo(importantSettings: Unit) {
private[this] val finn = Hero("Finn")
private[this] val jake = Hero("Jake")
def findHero(name: String): Hero = {
// Imagine all kinds of database processing here
finn
}
def friendsRegistry(): Map[String, Hero] = {
// Moar processing
Map(finn.name -> jake)
}
def evalAdventure(hero1: Hero, hero2: Hero): String = {
// Jake always saves the day, he's a magic dog!
if (hero1 == jake || hero2 == jake) "awesome" else "disappointing"
}
}
// The instance of Ooo we want to inject everywhere
val ooo = Ooo(())
// This is a piece of 'business' logic
object AdventureTime {
def getHero(name: String): Hero = ooo.findHero(name)
def getBestFriend(hero: Hero): Hero = ooo.friendsRegistry()(hero.name)
def goOnAdventure(hero1: Hero, hero2: Hero): String = {
val result = ooo.evalAdventure(hero1, hero2)
s"Adventure time with ${hero1.name} and ${hero2.name} was $result!"
}
}
Adventure Time – land of Ooo
Instead of a stuffy real-world example I’m using Adventure Time. Think of Ooo as a database repository and AdventureTime as some piece of business logic. I assume this code is relatively simple and understandable. The problem is this: how does AdventureTime get a reference to Ooo? In other words, we want to inject Ooo into AdventureTime and possibly other parts of the code.
First, an example of how one could have an adventure:
import AdventureTime._
val hero1 = getHero("Finn")
val hero2 = getBestFriend(hero1)
val result = goOnAdventure(hero1, hero2)
// result -> "Adventure time with Finn and Jake was awesome!"
A global variable and/or the Singleton
The example above illustrates one of the easiest ways of doing this: use a global variable and refer to that. This works great for small programs but when your program gets a bit larger, or your codebase is a bit older, this becomes very painful. Globals are difficult to maintain, they’re not very flexible, and they make code difficult to unit-test. You can also see in the example that the dependency is kind of hidden.
DI frameworks
Thankfully the industry has moved on from globals (right?) and frameworks like Spring and Guice have been invented to help. I won’t go into details about how they work, but they’re usually similar to constructor injection.
Constructor injection
In OO languages we can use the constructor of an object to provide it with the needed dependency. The AdventureTime object is now a class.
class AdventureTime(ooo: Ooo) {
def getHero(name: String): Hero = ooo.findHero(name)
def getBestFriend(hero: Hero): Hero = ooo.friendsRegistry()(hero.name)
def goOnAdventure(hero1: Hero, hero2: Hero): String = {
val result = ooo.evalAdventure(hero1, hero2)
s"Adventure time with ${hero1.name} and ${hero2.name} was $result!"
}
}
val at = new AdventureTime(ooo)
val hero1 = at.getHero("Finn")
val hero2 = at.getBestFriend(hero1)
val result = at.goOnAdventure(hero1, hero2)
This is a bit better than using global variables. Note that we still need some way to actually get ooo to where we create our at object, but in this post I want to focus on where the dependency is used. You can see that AdventureTime now has an explicit dependency on Ooo.
One caveat of this approach is that your class file should not become too large, otherwise you’re basically back to using a global variable! Constructor injection is not bad, it’s been used to create large systems. It’s fairly flexible, although you usually can’t change the dependency after it’s set. In order to test this you’d need to create a mock implementation or use a mocking library to mock the dependency.
What we actually want
We actually would like to pass the dependency as a parameter to every function that might need it.
object AdventureTime {
def getHero(ooo: Ooo, name: String): Hero = ooo.findHero(name)
def getBestFriend(ooo: Ooo, hero: Hero): Hero = {
ooo.friendsRegistry()(hero.name)
}
def goOnAdventure(ooo: Ooo, hero1: Hero, hero2: Hero): Unit = {
val result = ooo.evalAdventure(hero1, hero2)
s"Adventure time with ${hero1.name} and ${hero2.name} was $result!"
}
}
import AdventureTime._
val ooo = Ooo(())
val hero1 = getHero(ooo, "Finn")
val hero2 = getBestFriend(ooo, hero1)
val result = goOnAdventure(ooo, hero1, hero2)
This is a very flexible approach, we could change the dependency with each function call. We don’t need an instance variable to hold the dependency which makes this approach very suitable for, well, functions. We obviously see a pattern in these functions, but we can’t really abstract over it to remove the repetition.
Ice King – he’s so misunderstood
Monads
Let’s see how we can use some functional programming and the Reader monad to improve this. Before we do that though, let’s quickly refresh how monads work. We use an all time favourite, the Option monad. Feel free to skip this explanation if you’re familiar with it.
The example code is actually not very null-safe.
val hero1 = getHero(ooo) // <- hero1 could be null // which would probably make getBestFriend throw an NPE val hero2 = getBestFriend(ooo, hero1) // hero2 can also be null... val result = goOnAdventure(ooo, hero1, hero2)
One way to handle this would be something like:
val hero1 = getHero(ooo, "Finn")
if (hero1 != null) {
val hero2 = getBestFriend(ooo, hero1)
if (hero2 != null) {
val result = goOnAdventure(ooo, hero1, hero2)
} else {
println("No adventure today")
}
} else {
println("No adventure today")
}
This kind of clutters up things and distracts from what the code is actually trying to do. The Option monad represents the possibility that something can be null. We can encode this optional behaviour into the types. The monad then let’s us concentrate on the actual happy-path of the code while handling the boiler-plate around null-checking for us.
case class Ooo(importantSettings: Unit) {
// It's possible the hero can't be found, so it's optional
def findHero(name: String): Option[Hero] = {
Some(finn)
}
def friendsRegistry(): Map[String, Hero] = {/* same as before */}
def evalAdventure(hero1: Hero, hero2: Hero): String = {
/* same as before */
}
}
object AdventureTime {
// Another Option here.
def getHero(ooo: Ooo, name: String): Option[Hero] = ooo.findHero(name)
// Yet another one. Types tend to ripple through a codebase
def getBestFriend(ooo: Ooo, hero: Hero): Option[Hero] = {
ooo.friendsRegistry().get(hero.name)
}
def goOnAdventure(ooo: Ooo, hero1: Hero, hero2: Hero): String = {
/* same as before */
}
}
import AdventureTime._
val ooo = Ooo(())
val result: Option[String] = for {
hero1 <- getHero(ooo, "Finn")
hero2 <- getBestFriend(ooo, hero1)
} yield goOnAdventure(ooo, hero1, hero2)
println(result.getOrElse("There was no adventure :("))
The Option monad does exactly what we want. If there are no nulls, everything works as before. If there is a null somewhere in the process, it kind of ‘sticks’. I.e., no subsequent code is executed and a None is returned. It’s not exactly ‘as before’, we’ve obviously switched to a for comprehension.
We’ve enhanced the return types of our functions to deal with a kind of ‘secondary’ logic so we can focus on the main functionality that we’d like to express. That sounds familiar. What if we could encode our dependency into the return type as well?
Enter the Reader
The Reader monad basically encodes a simple function. It’s type definition is:
type Reader[E, A] = ReaderT[Id, E, A]
Let’s forget the right hand side of that type alias for now. Reader just expresses a function that takes a parameter of type E and returns a value of type A. Think of it as:
def func(e: E): A = {
// create some A using e
}
// or
val func = (e: E) => {
new A(e.foo())
}
You see how we could use that to express a dependency. The first type parameter E stands for ‘environment’. In our code E is Ooo and A is whatever our functions return. E.g., an Option[Hero] or a String. The type signature of getHero would become def getHero(name: String): Reader[Ooo, Option[Hero]]. Read: “getHero is a function that returns a function. When the returned function is supplied an Ooo it will return an Option of Hero“.
Let’s add this to our example. Note that all the functions in AdventureTime have the same dependency, so we make a little type alias for it. I’m assuming the reader is familiar with the various ways of creating lambda functions in Scala.
// Warning: this is not the final example, don't write code like this!
type OooReader[X] = Reader[Ooo, X]
object AdventureTime {
def getHero(name: String): OooReader[Option[Hero]] = Reader{
(ooo: Ooo) => ooo.findHero(name)
}
def getBestFriend(hero: Hero): OooReader[Option[Hero]] = Reader{
_.friendsRegistry().get(hero.name)
}
def goOnAdventure(h1: Hero, h2: Hero): OooReader[String] = Reader{
(ooo: Ooo) =>
val resultOfAdventure = ooo.evalAdventure(h1, h2)
s"Adventure time with ${h1.name} and ${h2.name} was $resultOfAdventure!"
}
}
import AdventureTime._
val res = for {
hero1 <- getHero("Finn")
hero2 <- getBestFriend(hero1.get) // .get !? ick...
result <- goOnAdventure(hero1.get, hero2.get)
} yield result
This looks similar to before, but we’ve managed to remove all the ooo parameters. Hang on, where are we injecting ooo now? Well, we’re not. This code seems to not do anything. If you inspect the type of res you’ll see it’s scalaz.Kleisli[scalaz.Id.Id,Ooo,String]. 😱
Remember that getHero returns an OooReader, i.e., a function taking an Ooo and returning an Option[Hero]. getBestFriend actually has the same signature. Just like Option, using Reader in a for comprehension sequences the monads into a ‘bigger’ one. For Option this means combining potentially absent values. For Reader it just means: “keep passing the dependency to the next function”. We’ve basically combined all three function calls into one big Reader.
If we want to execute the code we need to supply it with an Ooo using the run function of Reader.
res.run(Ooo(())) // --> scalaz.Id.Id[String] = Adventure time with Finn and Jake was awesome!
Monad Transformer
We’ve run into a problem though. We had to resort to the evil get function for unwrapping our Options. So the Reader basically undid all the Option monad goodness. Ideally the code should handle both monads at once. Fortunately there is a monad transformer for Reader called ReaderT.
What was that weird type signature and what is this Id stuff? Remember the right hand side of the Reader type alias? It was ReaderT[Id, E, A]. It turns out that instead of working with functions of type E => A, we usually work with functions like E => M[A], where M is some kind of monad. ReaderT expresses just that. Reader is actually an alias for ReaderT where M is the Id monad. I see Id as the ‘does nothing’ monad.
ReaderT looks like this:
type ReaderT[F[_], E, A] = Kleisli[F, E, A]
What? Another type alias? Yes, ReaderT is actually equivalent to Kleisli, which is what scalaz uses. Kleisli also adds many convenience functions for combining Kleislis.
Let’s rewrite our example using Kleisli instead:
object AdventureTime {
// Kleisli[Option, Ooo, Hero] 'represents' Ooo => Option[Hero]
def getHero(name: String) = kleisli[Option, Ooo, Hero](_.findHero(name))
def getBestFriend(hero: Hero) = kleisli[Option, Ooo, Hero]{
_.friendsRegistry().get(hero.name)
}
def goOnAdventure(h1: Hero, h2: Hero) = kleisli[Option, Ooo, String]{
(ooo: Ooo) =>
val resultOfAdventure = ooo.evalAdventure(h1, h2)
Some(s"Adventure time with ${h1.name} and ${h2.name} " +
s"was $resultOfAdventure!")
}
}
import AdventureTime._
val res = for {
hero1 <- getHero("Finn")
hero2 <- getBestFriend(hero1)
result <- goOnAdventure(hero1, hero2)
} yield result
res.run(Ooo(()))
Before we had Reader just wrapping a function that matches the desired type. There is no such constructor for ReaderT, probably just because kleisli already does exactly the same. In other words, one can create a ReaderT using the kleisli function. The type parameters in order are: the monad of the return value, the environment of the function, and the type of the return value.
The Future
This all looks nice but we might not be convinced yet. Sit tight, I’ll show you a great advantage of using Reader. We’ll have to go even more functional though.
Our for comprehension should belong in some function in the logic layer of our program. We’ve abstracted the dependency on Ooo through the Reader but the sample code still strongly couples to AdventureTime. Let’s remove that by passing the necessary functions as parameters instead!
object SomeFancyLogic {
def startEpicAdventure(
getHero: (String) => ReaderT[Option, Ooo, Hero],
getBestFriend: (Hero) => ReaderT[Option, Ooo, Hero],
goOnAdventure: (Hero, Hero) => ReaderT[Option, Ooo, String])
(name: String): ReaderT[Option, Ooo, String] = {
for {
hero1 <- getHero(name)
hero2 <- getBestFriend(hero1)
result <- goOnAdventure(hero1, hero2)
} yield result
}
}
// We usually 'wire up' the parameter group containing the
// functions first
val startEpicAdventureWired = SomeFancyLogic.startEpicAdventure(
AdventureTime.getHero _,
AdventureTime.getBestFriend _,
AdventureTime.goOnAdventure _) _
startEpicAdventureWired("Finn").run(Ooo(()))
Let’s also make our ‘database’ a bit more realistic. In the server world we like to avoid blocking, so APIs for external services usually return Futures.
// The land of Ooo of the future
case class Ooo(importantSettings: Unit) {
// findHero now returns a Future
// for simplicity I'm ignoring the Option stuff.
def findHero(name: String): Future[Hero] = {
Future.successful(finn) // again, just simulating here..
}
def friendsRegistry(): Future[Map[String, Hero]] = {
Future.successful(Map(finn.name -> jake))
}
def evalAdventure(hero1: Hero, hero2: Hero): Future[String] = {
Future.successful{
if (hero1 == jake || hero2 == jake) "awesome" else "disappointing"
}
}
}
// The rest of the code stays almost the same!
// Just change the Monad type parameter from Option to Future
object AdventureTime {
def getHero(name: String) = kleisli[Future, Ooo, Hero](_.findHero(name))
def getBestFriend(hero: Hero) = kleisli[Future, Ooo, Hero]{
_.friendsRegistry().map(_(hero.name))
}
def goOnAdventure(h1: Hero, h2: Hero) = kleisli[Future, Ooo, String]{
(ooo: Ooo) =>
ooo.evalAdventure(h1, h2).map{result =>
s"Adventure time with ${h1.name} and ${h2.name} was $result!"
}
}
}
object SomeFancyLogic {
def startEpicAdventure(
getHero: (String) => ReaderT[Future, Ooo, Hero],
getBestFriend: (Hero) => ReaderT[Future, Ooo, Hero],
goOnAdventure: (Hero, Hero) => ReaderT[Future, Ooo, String]
)(name: String): ReaderT[Future, Ooo, String] = {
for {
hero1 <- getHero(name)
hero2 <- getBestFriend(hero1)
result <- goOnAdventure(hero1, hero2)
} yield result
}
}
/* wiring as before, snipped for brevity o_O */
val future = startEpicAdventureWired("Finn").run(Ooo(()))
Await.result(future, 2.seconds)
A pattern is emerging here! We can actually abstract out the monad! We can also abstract away the dependency on Ooo. It looks like this:
object SomeFancyLogic {
def startEpicAdventure[M[_]: Monad, E](
getHero: (String) => ReaderT[M, E, Hero],
getBestFriend: (Hero) => ReaderT[M, E, Hero],
goOnAdventure: (Hero, Hero) => ReaderT[M, E, String]
)(name: String): ReaderT[M, E, String] = {
for {
hero1 <- getHero(name)
hero2 <- getBestFriend(hero1)
result <- goOnAdventure(hero1, hero2)
} yield result
}
}
E is now the generic type for the dependency. M[_] is a type that is actually a type constructor. Look at it as a type with a hole that needs another type to be whole. E.g., Option[String] or Future[Hero]. We also specify that there needs to be an implementation for the Monad type class for M.
The cherry on top
Wildberry is not a cherry but she is pretty.
Testing this piece of logic now becomes pretty easy. Of course the logic is really simple here.
A unit test should only test the code-under-test. With our new function parameters this means we can easily instruct our test without using any mock libraries. We test Popjam using ScalaCheck to do extensive property based testing. Also note that while the database is using Futures, we don’t actually want to test the asynchronous behaviour of the code, just the logic. Moreover, creating tests with concurrency in them usually leads to brittle time-dependent tests.
Here’s how we could test our logic:
def testEpicAdventure() = {
// our 'mocked' functions. Usually we would make them return
// more useful results obviously
val getHero = (name: String) => kleisli[Id, Unit, Hero]{
_ => Hero(name)
}
val getBestFriend = (h: Hero) => kleisli[Id, Unit, Hero]{
_ => Hero("Jake")
}
val goOnAdventure = (h1: Hero, h2: Hero) => kleisli[Id, Unit, String]{
_ => "Test adventure"
}
val wired = startEpicAdventure(getHero, getBestFriend, goOnAdventure) _
val result = wired("Finn").run(())
result aka "how did the adventure test go" should equal("Test adventure")
}
We can just use Id for our monad and Unit for the database. I’ve found this way of testing to be a lot more fun than setting up complicated mock, stub, or spy objects.
There are a lot more things we can do with scalaz and ReaderT. Like MonadReader ask for instance. I encourage you to go on that adventure yourself!
Pingback: 1 – ReaderT 101 | Exploding Ads