Sunday, July 30, 2017

Domain Models - Late Evaluation buys you better Composition

In the last post we talked about early abstractions that allow you to design generic interfaces which can be polymorphic in the type parameter. Unless you abuse the type system of a permissive language like Scala, if you adhere to the principles of parametricity, this approach helps you implement abstractions that are reusable under various contexts. We saw this when we implemented the generic contract of the mapReduce function and its various specializations by supplying different concrete instances of the Monoid algebra.

In this post we will take a look at the other end of the spectrum in designing functional domain models. We will discuss evaluation semantics of model behaviors - the precise problem of when to commit to specific concrete evaluation semantics. Consider the following definition of a domain service module ..
type ErrorOr[A] = Either[String, A]

trait PaymentService {
  def paymentCycle: ErrorOr[PaymentCycle]
  def qualifyingAccounts(paymentCycle: PaymentCycle): ErrorOr[Vector[Account]]
  def payments(accounts: Vector[Account]): ErrorOr[Vector[Payment]]
  def adjustTax(payments: Vector[Payment]): ErrorOr[Vector[Payment]]
  def postToLedger(payments: Vector[Payment]): ErrorOr[Unit]
Such definitions are quite common these days. We have a nice monadic definition going on which can be composed as well to implement larger behaviors out of smaller ones ..
def processPayments() = for {
  p <- paymentCycle
  a <- qualifyingAccounts(p)
  m <- payments(a)
  a <- adjustTax(m)
  _ <- postToLedger(a)
} yield ()
Can we improve upon this design ?

Committing to the concrete early - the pitfalls ..

One of the defining aspects of reusable abstractions is the ability to run it under different context. This is one lesson that we learnt in the last post as well. Make the abstractions depend on the least powerful algebra. In this example our service functions return Either, which is a monad. But it's not necessarily the least powerful algebra in the context. Users may choose to use some other monad or may be even applicative to thread through the context of building larger behaviors. Why not keep the algebra unspecified at the service definition level and hope to have specializations in implementations or even in usages at the end of the world ? Here's what we can do ..
// algebra
trait PaymentService[M[_]] {
  def paymentCycle: M[PaymentCycle]
  def qualifyingAccounts(paymentCycle: PaymentCycle): M[Vector[Account]]
  def payments(accounts: Vector[Account]): M[Vector[Payment]]
  def adjustTax(payments: Vector[Payment]): M[Vector[Payment]]
  def postToLedger(payments: Vector[Payment]): M[Unit]
A top level service definition that keeps the algebra unspecified. Now if we want to implement a larger behavior with monadic composition, we can do this ..
// weaving the monad
def processPayments()(implicit me: Monad[M]) = for {
  p <- paymentCycle
  a <- qualifyingAccounts(p)
  m <- payments(a)
  a <- adjustTax(m)
  _ <- postToLedger(a)
} yield p
Note that we are using only the monadic bind in composing the larger behavior - hence the least powerful algebra that we can use is that of a Monad. And we express this exact constraint by publishing the requirements of the existence of an instance of a Monad for the type constructor M.

What about Implementation ?

Well, we could avoid the committment to a concrete algebra in the definition of the service. What about the implementation ? One of the core issues with the implementation is how you need to handle errors. This is an issue which often makes you commit to an implementation when you write the interpreter / implementation of a service contract. You may use Failure for a Try based implementation, or Left for an Either based implementation etc. Can we abstract over this behavior through a generic error handling strategy ? Some libraries like cats offers you abstractions like MonadError that helps you implement error reporting functionality using generic monadic APIs. Here's how we can do this ..
class PaymentServiceInterpreter[M[_]](implicit me: MonadError[M, Throwable])
  extends PaymentService[M] {


  def payments(accounts: Vector[Account]): M[Vector[Payment]] = 
    if (accounts.isEmpty) me.raiseError(
      new IllegalArgumentException("Empty account list"))
    else //..
Note we needed a monad with error handling capabilities and we used MonadError for that. Note that we have kept the error type in MonadError as Throwable, which may seem a bit unethical in the context of pure functional programming. But it's also true that many libraries (especially Java ones) or underlying abstractions like Future or Try play well with exceptions. Anyway this is just a red herring though it has nothing to do with the current topic of discussion. The moot point is that you need to supply a MonadError that you have instances of.

Here's how cats defines the trait MonadError ..
trait MonadError[F[_], E] extends ApplicativeError[F, E] with Monad[F] { //..
.. and that's exactly what we will commit to. We are still dealing with a generic Monad even in the implementation without committing to any concreate instance.

End of the World!

The basic purpose why we wanted to delay committing to the concrete instance was to allow the users the flexibility to choose their own implementations. This is what we call the principle of delayed evaluation. Abstract early, evaluate late and decouple the concerns of building and the evaluation of the abstractions. We have already seen the 2 of these principles - we will see that our design so far will accommodate the third one as well, at least for some instances of M.

The user of our API has the flexibility to choose the monad as long as she supplies the MonadError[M, Throwable] instance. And we have many to choose from. Here's an example of the above service implementation in use that composes with another service in a monadic way and choosing the exact concrete instance of the Monad at the end of the world ..
import cats._
import cats.implicits._

// monix task based computation
object MonixTaskModule {
  import monix.eval.Task
  import monix.cats._

  val paymentInterpreter = new PaymentServiceInterpreter[Task]
  val emailInterpreter = new EmailServiceInterpreter[Task]

  for {
    p <- paymentInterpreter.processPayments
    e <- emailInterpreter.sendEmail(p)
  } yield e

// future based computation
object FutureModule {
  import scala.concurrent.Future
  val paymentInterpreter = new PaymentServiceInterpreter[Future]
  val emailInterpreter = new EmailServiceInterpreter[Future]

  for {
    p <- paymentInterpreter.processPayments
    e <- emailInterpreter.sendEmail(p)
  } yield e

// try task based computation
object TryModule {
  import scala.util.Try

  val paymentInterpreter = new PaymentServiceInterpreter[Try]
  val emailInterpreter = new EmailServiceInterpreter[Try]

  for {
    p <- paymentInterpreter.processPayments
    e <- emailInterpreter.sendEmail(p)
  } yield e
Monix Task is an abstraction that decouples the building of the abstraction from execution. So the Task that we get from building the composed behavior as in the above example can be executed in a deferred way depending of the requirements of the application. It can also be composed with other Tasks to build larger ones as well.

Vertical Composition - stacking abstractions

When you have not committed to an implementation early enough, all you have is an unspecified algebra. You can do fun stuff like stacking abstractions vertically. Suppose we want to implement auditability in some of our service methods. Here we consider a simple strategy of logging as a means to audit the behaviors. How can we take an existing implementation and plugin the audit function selectively ? The answer is we compose algebras .. here's an example that stacks the Writer monad with an already existing algebra to make the payments function auditable ..
final class AuditablePaymentService[M[_]: Applicative](paymentService: PaymentService[M]) 
  extends PaymentService[WriterT[M, Vector[String], ?]] {

  def paymentCycle: WriterT[M, Vector[String], PaymentCycle] =

  def qualifyingAccounts(paymentCycle: PaymentCycle): WriterT[M, Vector[String], Vector[Account]] =

  def payments(accounts: Vector[Account]): WriterT[M, Vector[String], Vector[Payment]] =


val auditablePaymentInterpreter = new AuditablePaymentService[Future](
  new PaymentServiceInterpreter[Future]
We took the decision to abstract the return type in the form of a type constructor early on. But committed to the specific type only during the actual usage of the service implementation. Early abstraction and late committment to implementation make great composition possible and often in ways that may pleasantly surprise you later ..


Benoit said...

Thanks alot for this post!

Philippe Derome said...

Very nice blog! People on the conferences circle or significant OSS contributors may find this elementary but I think it's very useful for most Scala developers who are working in the enterprise (me) or for startups. Debasish is easier to follow than several other writers on FP in Scala arena by introducing sample code in context and discussing full context in sufficient detail. Not everybody knows how to compose abstractions with Writer (or Reader) monads as is done at the end of the blog. It's a beauty to see such a concise and clear exposure of non-trivial concepts. Maybe you could expand the WriterT at the end a little more to show the method calls that are being made? Or discuss how such a Writer Monad could be used to collect latency statistics for the audited calls (e.g. publishing to a service like Logstash)?

Readers should understand that a EmailService would be defined in way similar to PaymentService (not shown in blog).

If you like this blog, you'll probably like his book Functional and Reactive Domain Modeling at Manning. Readers may also like work Raul Raja Martinez on Freestyle project (I believe it's sponsored by 47 Degrees); he's made similar arguments on error handling as is made here.

zagubiony said...

Really good post, I have one question though. You parameterize with higher kinded type. Does it imply that you have to implement it every time you want to test it? Be it Future , Try or Task. Doesn't it make it not reusable ? I mean if I want to test it with unit test I don't want to return Future if all the data is stubbed, I would rather return Try but then I need Try -ish implementation which in turn requires from me implementing all the logic again. Is it really effective and worth it?

Debasish Ghosh said...

@zagubiony - You don't have to implement anything repeatedly. Note the entire implementation is in PaymentServiceInterpreter and it is completely independent of the instance of Monad that you are using. Look at the modules FutureModule or TryModule - you just instantiate with the type of Monad - Try or Future. This gives you the reusability.

zagubiony said...


thank you for your response. It's still not clear for me yet. Let me explain with some examples:
PaymentServiceInterpreter has various methods like payments and postToLedger let's assume that former needs DB and latter talks to 3rd party API/module. I can either implement those myself in PaymentServiceInterpreter itself (ie make the calls myself, fetch data from DB) or use the interface supplied in the constructor of the class. I am therefore dependent on those implementations or the return type of the calls I make.

I am really trying to understand it as I have had to cheat with Future.successful in my mocks in tests many times before and I really don't like this approach.

It would be much easier if you had maybe a repository with examples?

Thank you

Debasish Ghosh said...

@zagubiony - I think you are conflating 2 issues. In the implementation of the interpreter you can use any types that get returned from your third party library. e.g. your db access library may return a Try. The thing that we are trying to parameterize here is the contract between the interpreter and the external world. Suppose you have a method in your interpreter class as ..

// DB is an abstraction from your third party library
def qualifyingAccounts(p: PaymentCycle, db: DB): M[Vector[Account]] = {
// use DB to fetch from database
// do whatever you like with the result, use Try, Future etc.
// Finally when u return a Vector[Account] v, do this or
// call another function that returns a vector wrapped in an M monad

The contract of compositionality is just between the functions of the interpreter and possibly with other interpreters (we have not yet discussed it though). The only constraint that you need to honor is that the functions of the class return an M[_] and all errors are reported using MonadError. So the important part is to keep in mind is that within your implementation when you return errors, you need to use an abstraction that has an instance of MonadError.