Getting Func-ey Part 5 - Basics of Final-tagless / ZIO

29 minute read Published: 2020-02-27

They are originally based on a series of 5 presentations I ran at my company in an attempt to get in front of some anti-Scala sentiment that pervaded much of the org at the time. The org suffered from the typical problem of hiring Java devs and then telling them to write to Scala. It assumes someone at least has a few months of Scala programming experience.

Recap

Up until now we've been looking at little pieces, or what I like to call FP in the small:

We end up here because we want referential transparency (the ability to reason locally by being able replace an expression with it's value). From Part 4 we learned that we can do this with IO. We can build a description of what we want to happen, separate from execution, that is referentially transparent. This has nothing to do with whether or not the same call to a webserver, within IO, will return the same result. This is not referential transparency. Referential transparency only refers to being able to do the substitution trick. All bets are off once we interpret or execute the IO program (say using .unsafeRunSync).

Effects

There are also transformed examples that stack two effects like EitherT[Future, Throwable, Result]:

At the end of this post, in the bonus section, I show how we can stack effects using monad transformers and how unpleasant this is to due in Scala. For a number of reasons, this style is much more common in Haskell (better type system, much better optimization). What we are looking at in this post (final-tagless, or ZIO) are much more ergonomic alternatives in Scala that retain some of the properties we want (with tradeoffs).

Procedural Effects

Procedural effects are side-effecting non-deterministic partial interactions with the real world:

Functional effects

A functional effect is an immutable data structure that describes procedural effects. They are later interpretted in some runtime into the procedural effects they describe. If you use Monix Task and take a look at the source for Task you can see this clearly: 

/** [[Task]] state describing an immediate synchronous value. */
private[eval] final case class Eval[A](thunk: () => A) extends Task[A]
private[eval] final case class Suspend[+A](thunk: () => Task[A]) extends Task[A]
/** Internal [[Task]] state that is the result of applying `flatMap`. */
private[eval] final case class FlatMap[A, B](source: Task[A], f: A => Task[B]) extends Task[B]
/** Internal [[Coeval]] state that is the result of applying `map`. */
private[eval] final case class Map[S, +A](source: Task[S], f: S => A, index: Int)
  extends Task[A] with (S => Task[A]) {

Unlike Future a call to map or flatMap is not a method call. Instead they are data structures describing what needs to happen. The first time I saw this my mind was blown.

What we want

We want a way to describe:

We do this by creating an embedded DSL in our language:

As I said in a previous post, OOP and FP are two sides of the same coin. It's like skiing vs. snowboarding. They are different but you are still going down the same mountain.

A minimal example for IO / ZIO / FP

Let's put all this pieces together. We will accomplish this by looking at portions of a tiny webservice using akka-http and Future, along with some of the tools we've learned. To show that it is actually not scary to go further off the FP deep end, we'll compare this same service with one using a functional HTTP library called Http4s using cats-effect in the final-tagless style. We will also look at what parts of this look like in ZIO which is an alternative to cats-effect with different trade-offs and ergonomics.

This is just a taste of what this looks like, and I don't explain all the syntax.

I pick no favourites here between the FP libraries and the example is deliberately simplified to fit to a blog post. As always remember: two programmers, three opinions. I do a lot of hand-waving here to get the example going and leave some of the explanation to existing documentation.

Finally, I'm still new to this stuff too. Hopefully this can help some other beginners, and porting this akka-http service at work was the first time I even used ZIO. Please hit me up on twitter if I've said something wrong/misleading!

Cats Effect / Final Tagless and ZIO

Cats-Effect and ZIO both provide IO capabilities. They are both runtimes that allow us to describe a program and later interpret them. It turns out they are both (handwavy) m:n threaded environments living on top of the JVM. Instead of threads they talk about Fibers (in cats, in zio) which are not mapped 1:1 to native threads. Instead there is a runtime in-between. This runtime has a ton of functional primatives for concurrent programming that are beyond the scope of this blog post but worth looking into. We can spin up many, many fibers (in the same way you can spin up many, many actors). We can wait on them cheaply. We can cancel them easily compared to Native threads. There is some interop between ZIO and cats-effect.

Tagless Final

We write Algebras (think: Services) parameterized on some abstract F[_]:

trait console[F[_]] {
  def put(v: String): F[Unit]
  def get: F[String]
}

This shares the same encoding as a typeclass but is not a typeclass since we may define more than one interpretation for a given type. A major con of the Tagless Final approach is that we have to differentiate between algebras and typeclasses ourselves in type signatures --this is unfortunate.

The algebra above represents printing something to the console. We can create an interpreter on some concrete type, in this case Task but it could have been IO or ZIO:

implicit object TaskInterpretter extends Console[Task] {
  def put(v: String): Task[Unit] = Task.delay(println(v))
  def get: Task[String] = Task.eval(scala.io.StdIn.readLine())
}

The general idea is:

Running Example

Let's look at our akka-http service defined by three endpoints around interfacing with some Social Profile API. You can imagine that there would be some interface like the following that would end up as a dependency of the Routes class in akka-http:

trait AuthService {
  def authorize(req: ValidAuthorizeRequest): Future[AuthorizeResponse]
  def authenticate(req: ValidAuthenticateRequest): Future[AuthenticateResponse]
  def getManageableSocialProfiles(accessToken: AuthAccessToken): Future[SocialProfilesResponse]
}

What would this look like in the Final-Tagless approach? Not very different except we are parameterized on F[_] instead of Future:

trait AuthService[F[_]] {
  def authorize(req: ValidAuthorizeRequest): F[AuthorizeResponse]
  def authenticate(req: ValidAuthenticateRequest): F[AuthenticateResponse]
  def getManageableSocialProfiles(accessToken: AuthAccessToken): F[SocialProfilesResponse]
}

Let's say we are using Sttp to make remote calls to the third party API. How would the constructors look different between the two approaches for an implementation? In the OOP case, we would have some concrete class:

class LiveAuthService(
  private implicit val client: SttpBackend[Future, ...not important]
  private implicit val ec: ExecutionContext @@ DefaultContext
) extends AuthService { .. }

In the Final-tagless approach we would also have some business logic, but still parameterized on some abstract F.

class LiveAuthService[F[_]: MonadError[*[_], Throwable]](
  private implicit val client: SttpBackend[F, Nothing, WebSocketHandler]
) extends AuthService[F] { .. }

Not too different I hope. We have a concrete class, but we still don't know our F. The final-tagless version has the benefit that we don't have to pass along the execution context everywhere (yay). We know that remote calls can fail, which is encoded in the Failure channel of Future.

But what about our F? We need to use a typeclass to give our F more capabilities. That is, we need whatever F to be supplied to have the ability to handle errors. This is the ugly verbose MonadError. In practice, you usually use a type alias to make this more ergonomic e.g.

type MonadThrow[F[_]] = MonadError[F, Throwable]
class LiveAuthService[F[_]: MonadThrow]](
  private implicit val client: SttpBackend[F, Nothing, WebSocketHandler]
) extends AuthService[F] {

Hopefully this doesn't look that much more scary than the regular Future based one. The interesting thing here is that all we know about F is that we can throw and recover from errors and that F is a monad (so we will have access to flatMap, and since every Monad is also a Functor, we have access to map).

We know much less than we do than the version using a concrete Future. Our code can only use what we have provided to F, that is, it's much more constrained than Future. Someone isn't going to be able to embed some arbitrary effect because the type system won't allow it. There is nothing stopping someone with the Future from being able to do Future(sneakyMineBitCoin(panamaAccountId)) for example.

Now let's look at the actual implementations of the logic for the authorize method. In the Future approach:

override def authorize(req: ValidAuthorizeRequeset): Future[AuthorizeResponse] = {
  val authorizeBody: Map[String, String] = Map(
     // ... post body here .... 
  )

  basicRequest
    .body(authorizeBody)
    .post(AuthService.authorizeBase)
    .response(asJson[AuthAccessToken])
    .send()
    .flatMap(
      response =>
        response.body.fold(
          error => Future.failed(ApiError.fromResponse(response.code.code, error)),
          token =>
            Future
              .successful(AuthorizeResponse(AuthService.oauthVersion, token.token.toString, None, None))
      )
    )
}

We're using sttp here. The Future may timeout or have some other error. There may be some parsing error on the response.body when we try to parse the json to an AuthAccessToken, if there is, we turn that into some domain specific ApiError. If everything works out correctly, we construct an AuthorizeResponse.

Looking at the final-tagless version:

override def authorize(req: ValidAuthorizeRequest): F[AuthorizeResponse] = {
  val authorizeBody: Map[String, String] = Map(
      ... post body ... 
  )

  basicRequest
    .body(authorizeBody)
    .post(AuthService.authorizeBase)
    .response(asJson[AuthAccessToken])
    .send()
    .flatMap(
      response =>
        response.body.fold(
          error => ApiError.fromResponse(response.code.code, error).raiseError[F, AuthorizeResponse],
          token => AuthorizeResponse(AuthService.oauthVersion, token.token.toString, None, None).pure[F]
      )
    )
}

This looks pretty similar! One big reason of this is because Sttp gives a nice interface to being able to plug in a whole bunch of backends. Anything from akka-http client to clients for Cats / ZIO / Task.

Our F is still abstract, but we know we have MonadError which implies Monad. Hence we have access to flatMap. The main difference is how errors are raised, which we do through the MonadError typeclass. To raise an error we use raiseError. To return a value, we use pure.

Not that different and the other methods of the service play out the same way.

Routing

If we were using akka-http as our webserver, we would wire this service into some routes along the lines of: 

class AuthServiceRoutes(authService: AuthService)
 (implicit val ec: ExecutionContext @@ DefaultContext) {
  private val authorize = path("oauth" / "authorize") {
    Route.seal(post {
      entity(as[ValidAuthorizeRequest]) { request =>
        onComplete(authService.authorize(request)) { response =>
          logger.info(s"POST /oauth/authorize")
          complete(response)
        }
      }
    })(exceptionHandler = baseExceptionHandler)
  }

  // ...
  // ...
  // rest of the routes
}

Note here I'm using MacWire for dependency injection.

If we were going more functional using Http4s, it would look like:

final class AuthRoutes[F[_]: Sync]
  (authService: AuthProgram[F]) extends Http4sDsl[F] {
    // ...
    case req @ POST -> Root / "authorize" =>
      req.decodeR[ValidAuthorizeRequest] { authReq =>
        authService
          .authorize(authReq)
          .flatMap(r => Ok(r.asJson))
          .recoverWith(errorHandler)
      }
      
    // .. 
    // ..
    // rest of the routes 
}

Sync is a monad that can suspect execution of side effects in the F[_] context. It's powerful and should be used sparingly in your program since it is as unconstrained as Future, e.g. you can embed anything in it. A good discussion is found here including the gist and responses to the original post.

Hopefully this still doesn't look too different. But at what point does F become concrete? We see that in the programs Main method. In the Future approach we'll be setting up our runtime dependency injection to make it all work in some Main.scala's run method or similar:

    // Ensure MDC context information is passed
    val _ = MDCContextLocal.create
    implicit val system: ActorSystem = ActorSystem("channelinstagrambasic")
    implicit val mat: ActorMaterializer = ActorMaterializer()
    implicit val globalEc: ExecutionContext @@ DefaultContext =
      ContextLocalPropagatingExecutionContextWrapper(system.dispatchers.lookup("execution-contexts.global"))
        .taggedWith[DefaultContext]

    @SuppressWarnings(Array("org.wartremover.warts.Any")) // it's confused
    lazy implicit val sttp: SttpBackend[Future, Source[ByteString, Any], Flow[Message, Message, *]] =
      wireWith(AkkaHttpBackendFactory.create(_))
    lazy val authService: AuthService = wire[LiveAuthService]
    lazy val serviceRoutes = wire[AuthServiceRoutes]
    lazy val healthRoutes = wire[HealthRoutes]
    lazy val statsd: StatsdReporter = wire[StatsdReporter]
    lazy val routes: RoutingModule = wire[RoutingModule]
    lazy val server: Server = wire[Server]

    server.run()

In the final-tagless approach, we pick a concrete F. In our case, we're going to use IO. This is the only place that IO makes an appearance, all of our business logic is in some generic F with capabilities defined by typeclasses:

  implicit val logger = Slf4jLogger.getLogger[IO]

  override def run(args: List[String]): IO[ExitCode] =
    AsyncHttpClientCatsBackend.resource[IO]().use { clients =>
      implicit val statsD = new StatsdIO("SomeConfig")
      for {
        services <- LiveAuthService.make[IO](clients)
        events = NoOpEventService.make[IO]
        programs = AuthProgram.make[IO](services, events)
        api <- RoutingModule.make[IO](programs)
        _ <- BlazeServerBuilder[IO]
          .bindHttp()
          .withHttpApp(api.httpApp)
          .serve
          .compile
          .drain
      } yield ExitCode.Success
    }

Gone is the DI framework. We have something that is kind of like dependency injection in that we have injected IO as our F and the entire program snaps into place from the abstract F to the concrete IO type. There are some other quirks.

There is some additional boilerplate. Our programs/algebras need some way to be constructed into a concrete F. We put a make on the companion object of the algebra to do this. E.g.

trait AuthService[F[_]] {
  def authorize(req: ValidAuthorizeRequest): F[AuthorizeResponse]
  def authenticate(req: ValidAuthenticateRequest): F[AuthenticateResponse]
  def getManageableSocialProfiles(accessToken: AuthAccessToken): F[SocialProfilesResponse]
}

object LiveAuthService {
  def make[F[_]: Sync](
    client: SttpBackend[F, Nothing, WebSocketHandler]
  ): F[AuthService[F]] = { 
       implicit val sttp = client
       Sync[F].delay(new LiveAuthService())
}

class LiveAuthService[F[_]: MonadError[*[_], Throwable]](
  private implicit val client: SttpBackend[F, Nothing, WebSocketHandler]
) extends AuthService[F] {
  ...
}

Using the make we get back our LiveAuthService[F] but we are still trapped inside an F. Since LiveAuthService requires a SttpBackend, which is treated as a resource, then we also need to make our AuthService a resource itself. Once you see this pattern (F[Thing[F]]), you see it all over the place in libraries. You can read more about it in Practical FP in scala

So what happened?

Hopefully that wasn't that bad to go from something Future based to something a little more functional. In fact, it almost looked the same but has some interesting benefits to how we put together the program. Final Tagless really pushes you down the path of defining interface boundaries. You should be doing this anyways but we all know people love to make mega classes (this is software dev 101). The entire final tagless approach only works if you actually make these interface boundaries. My gut tells me is that people will be more successful at defining interface boundaries with this approach but who knows in practice, but I haven't used this approach with a team yet.

The other benefit is more concrete: we can chose to interpret the description of our program (our business logic) however we want. This shouldn't really feel different than DI to provide a different description (think mocking). E.g. some LiveInMemoryDb[F] vs LiveRedisDB[F]. This is just well, regular DI and regular benefits of composition over inheritance. What is more interesting is that we can change to a different F in testing. Critically, we don't change our business logic when doing this and we can do it while still maintaining all the benefits of referential transparency.

Part of why you are probably thinking so what? is that our example is too simple. Our service is too small to really show off the benefits. So let's make our example a little more complicated. Let's pretend we have some AuthProgram wrapping our AuthService algebra that:

The AuthProgram

final class AuthProgram[F[_]: Logger: MonadError: Timer: Statsd](
  private val authService: AuthService[F],
  private val eventService: EventService[F]
) {
  val retryPolicy: RetryPolicy[F] =
    limitRetries[F](3) |+| exponentialBackoff[F](10.milliseconds)
 
  def authorize(req: ValidAuthorizeRequest): F[AuthorizeResponse] =
    for {
      _    <- Logger[F].info("started authorize request")
      resp <- withRetry(authService.authorize(req), "authenticate")
      _    <- eventService.emit(Event("SomeAuditEvent"))
      _    <- Statsd[F].emit("SomeKey", 10)
      _    <- Logger[F].info("finished authorize request")
    } yield resp
  ....
}

Now we have a bunch more going on. To steal some terminology from practical FP in scala, a program is anything that uses algebras to describe business logic. In our case, we have a program that describes making some requests, logging some info, and emitting some events. What has changed?

We use cats-retry for retrying. It has an excellent API for building up retry logic.

Remember the con to the final tagless approach. Our algebra and typeclasses share the same encoding. Logger is not a typeclass. Statsd is not a typelcass. MonadError is typeclass. The other con is what is a capability of F and what gets passed in to the constructor. You could argue that statsd should be some StatsdService that gets passed around manually. The rough rule of thumb I've seen floating around is if it has managed resources, pass it in manually. It really boils down on whether you want explicit or implicit dependencies. I'm still figuring out this decision myself.

Testing tricks

Since we can manipulate our F, aka our environment, we can do interesting things. For example, we can build an environment that can record log messages. This is powerful! Much of my day job revolves around diagnosing problems with third party APIs. I often have more logging code than actual business logic but it's kind of a pain to test. Let's look at testing that the log messages I wanted were crated:

  spec("Should track logging") {
    Ref.of[IO, Log](Log.empty).flatMap { logs =>
      implicit val logger = com.mf.util.test.TestLogger.acc(logs)
      new AuthProgram[IO](successfulAuthService, NoOpEventService.make[IO])
        .authenticate(validAuthenticateRequest)
        .attempt
        .flatMap {
          case Right(_) =>
            logs.get.map(
              log =>
                (log.info should contain)
                  .theSameElementsAs(List("started authorize request", "finished authorize request"))
            )
          case Left(_) => fail("expected AuthenticateResponse")
        }
    }
  }

We use another cats-effect IO primitive called Ref which is a concurrent mutable reference. I provide a version of the Logger algebra that holds onto this Ref that holds onto a case class that can hold info / warn / error messages. I create the Ref and then interpret my program inside of this.

I'm handwaving over lots of here --you really have to play with this stuff yourself. But what we've done here is extremely cool: I'm asserting on the correct log messages being produced.

In fact, we can use the same trick to record more relevant F[Unit] type behaviour. For example, writing to a DB, putting an event on a SQS queue, and many more modern business requirements will end up as something happening in a Future[Unit]. They are all over the place and often not well tested, even though you could always make a testing implementation of an interface w/ some mutable fields to keep track of this stuff if you didn't use this approach.

Say we have some EventService algebra that puts some event onto a queue via some emit method that is F[Unit]. I'd love to be able to test that the correct event gets emitted in a test and we can use this same recording environment trick!

  spec("Should record that event XXX is emitted") {

    val refEventService: Ref[IO, List[Event]] => EventService[IO] = ref =>
      new EventService[IO] {
        override def emit(event: Event): IO[Unit] =
          ref.update(events => event +: events)
      }

    Ref.of[IO, List[Event]](List()).flatMap { events =>
      val eventService = refEventService(events)
      implicit val logger = com.mf.util.test.TestLogger.NoOp
      new AuthProgram[IO](successfulAuthService, eventService)
        .authenticate(validAuthenticateRequest)
        .attempt
        .flatMap {
          case Right(_) =>
            events.get.map(
              e =>
                (e should contain theSameElementsAs (List(Event("SomeAuditEvent"))))
            )
          case Left(_) => fail("expected AuthenticateResponse")
        }
    }
  }

Great! We've manipulated our environment to test a hard piece of business logic without having to change that business logic. For me, this is powerful stuff that I can use all over the place in my codebase. What's more, we can do this same trick further up at the route/controller level to do more integration-type testing. Powerful stuff.

Downsides

That being said, this is still a great way to program with lots of upsides.

ZIO - An alternative

ZIO can be seen as an alternative to the final-tagless approach. You lose the ability to narrowly scope F to just a few capabilities; however, you gain a whole lot of ergonomics especially around the environment. It's still not a silver bullet.

ZIO

ZIO is a datatype: ZIO[R, E, A]:

Conceptually you can kind of think of it like some:

Environment => Either[E,A]

even though that is not what is exactly happening

This is really a combination of IO and Async (just like cats-effect IO) but it also mixes in ReaderT, which if you remember is how you do functional dependency injection. This is basically the RIO pattern from Haskell, nicely ported to Scala w/ a whole lot of ergonomic considerations.

ZIO can be in a few modes. We would like to use the R environment parameter but maybe you don't have to, or you want to get ZIO into your codebase without making too many changes --e.g. to move from Future to ZIO, or Task to ZIO:

UIO[A]    == ZIO[Any, Nothing  , A] // no requirements, can't fail, succeeds with A 
TASK[A]   == ZIO[Any, Throwable, A] // may fail w/ throwable or succeed with A
RIO[R, A] == ZIO[R  , Throwable, A] // often what we want! 
IO[E, A]  == ZIO[Any, E        , A] // like Task but to some non-Throwable error

Porting AuthService to ZIO

AuthService looks suspiciously the same thanks to sttp having a ZIO backend:

// remember Task = ZIO[Any, Throwable, A] here 
class LiveAuthService(
  private implicit val client: SttpBackend[Task, Nothing, WebSocketHandler]
) extends AuthService[Task] {
 
  override def authorize(req: ValidAuthorizeRequest): Task[AuthorizeResponse] = {
    val authorizeBody: Map[String, String] = Map(
            ... post params ... 
    )
 
    basicRequest
      .body(authorizeBody)
      .post(AuthService.authorizeBase)
      .headers(Header.contentType(MediaType.ApplicationXWwwFormUrlencoded))
      .response(asJson[AuthAccessToken])
      .send()
      .flatMap(
        response =>
          response.body.fold(
            error => Task.fail(GraphApiError.fromResponse(response.code.code, error)),
            token => Task.succeed(AuthorizeResponse(AuthService.oauthVersion, token.token.toString, None, None))
        )
      )
  }

Looking into AuthProgram we need to figure out how to make one of our custom capabilities like say statsd. ZIO uses scala modules system which takes a bit to wrap your head around:

trait Statsd { def statsd: Statsd.Service }
object Statsd {
  trait Service {
    def emit(key: String, value: Int): UIO[Unit]
  }

  trait NoOp extends Statsd.Service {
    def emit(key: String, value: Int): UIO[Unit] = UIO.unit
  }

  object NoOp extends NoOp
}

We can access an environment via ZIO.accessM[Statsd](_.statsd(emit(..)) which is verbose and annoying. You would usually construct a helper method to do this for us:

def emit(key: String, value: Int): ZIO[Statsd, Nothing, Unit] =
    ZIO.accessM(_.statsd.emit(key, value))

They have been working on improving this and there is this ZLayer type for constructing environments now but it's hot off the presses and there doesn't really seem to be any documentation around it yet. The bits I've seen floating around twitter look nice though

ZIO AuthProgram

Also not that different:

final class AuthProgram(private val authService: AuthService[Task]) {
  type Env = Clock with Logging[String] with Statsd with Scheduler

  val retryPolicy = Schedule.recurs(3) && Schedule.exponential(Duration.fromScala(10.milliseconds))

  def authenticate(req: ValidAuthenticateRequest): RIO[Env, AuthenticateResponse] =
    for {
      _    <- logger.info("started authorize request")
      resp <- authService.authenticate(req).retry(retryPolicy)
              // same as ZIO.accessM(_.statsd.emit("someKey", 10))
      _    <- statsd.emit("someKey", 10))
      _    <- logger.info("finished authorize request")
    } yield resp
}

The same idea as F[_] but note that it's all in the return type! In this case we have some Env type providing all the capabilities:

type Env = Clock with Logging[String] with Statsd with Scheduler

Note: we can do the same testing trick as before, as ZIO also has a world of concurrency primitives available to us. We just change what Logging gets injected into our environment.

Remember ZIO[R, E, A] is just some description of a program that is kind of an R => Either[E, A]. It's not this exactly but conceptually if you give it the environment R, you will get back either a failure or what you wanted.

So how would we use this thing? Well all the work is in constructing the environment. 

val env: Env = new Logging[String] with Statsd with Clock.Live {
  override val logging = NoOpLogger.logging
  override val statsd = Statsd.NoOp
}

val testAuthService = new AuthProgram(successfulAuthService)

// I'm still a ZIO[R, E, A] here, just a description
val authorizeResponse = testAuthService.authenticat(validAuthenticateRequest)

// provide my env, leaving me with an IO[E, A]
// this is like being stuck in a Future[A]
val still_a_description = authorizeResponse.provide(env)

// actually interpret my structure in a runtime
val runtime = new DefaultRuntime {} // this would be in your/edge of your program  
val res = runtime.unsafeRunAsync(still_a_description)

// result is of type Exit[E, A] => ValidAuthorizeResponse 
// there is also unsafeRunToFuture for interop

Final Tagless vs ZIO big differences

Cats-effect and ZIO have the same underlying principles: all programs are values, composed with pure functions. The rest is differences in style, ergonomics, etc.

They are more alike than different though:

No silver bullets

For instance:

def authorize(req: ValidAuthorizeRequest): F[AuthorizeResponse] =
  for {
    _    <- Logger[F].info("started authorize request")
    resp <- withRetry(authService.authorize(req), "authenticate")
    _    =  println("arbitrary untracked capability") // oops, scala let's us do
                                                      // arbitrary effects
    _    =  Thread.sleep(..) // oops, compiler let's us mess w/ the runtime
    _    <- eventService.emit(Event("SomeAuditEvent"))
  } yield resp

tldr; use your brain and still practice good code.

Resources

I've barely touched on the FP concurrency runtimes which are also awesome, so check those out:

Bonus Monad Transformers are awful in Scala

At the beginning of this post I mentioned Monad transformer stacks. This is very much a haskell approach to the problem. It works in scala but suffers from:

So let's use our final-tagless skeleton and instead of providing IO for our concrete type for F well use a transformer. Just a single one. It gets worse if we have more (e.g. a Reader to also be injecting an environment). Let's say we have some EventService algebra that emits events and we want to record them. Well we can define an interpretter using our concrete stack of WriterT[IO, List[Event], A] which means I have some IO environment that can also write a List[Events] while producing some value A

type Stack[A] = WriterT[IO, List[Event], A]
val recordingEventService = new EventService[Stack] {
  def emit(event: Event): Stack[Unit] =
    WriterT.liftF[IO, List[Event], Unit](IO.delay(())).tell(List(testEvent))
}

We have some other service that talks to some Social Media auth. It also gets stuck with the Stack type even though it's not writing any events: 

val authServiceStack = new AuthService[Stack] {
  def authenticate(req: ValidAuthenticateRequest): Stack[AuthenticateResponse] =
    WriterT.liftF[IO, List[Event], AuthenticateResponse](
      IO.delay(AuthenticateResponse("http://www.success.com/success", None))
    )
}

Now we have some program that uses both of these services (which is why the types had to be same even though we only care about the Writing part in one):

 new AuthProgramWithoutLogger[WriterT[IO, List[Event], *]](authServiceStack, recordingEventService)
    .authenticate(validAuthenticateRequest)
    .run // runs the Writer leaving an IO[(List[Event], ValidAuthenticateResponse)]
    .attempt
    .flatMap {
      case Right((events, _)) => IO.delay(events should contain theSameElementsAs(List(testEvent)))
      case Left(_) => fail("expected AuthenticateResponse")
    }

As you can see this is much worse than using final-tagless w/ cats-effect or using ZIO. It does work! We've manipulated our environment. We just changed the effects that were running and we can do that w/o changing AuthProgram. the syntax is awful. This is why ZIO and other approaches came to be. I would consider it practically impossible to get buy in on the above monad transformer approach --expect serious push back trying to teach this. It's hella noisy (all those liftFs) even for a trivial example.

Lots of pitfalls. Never use the Writer type for this kind of thing --if an exception occurs you will not get anything written. It's also not thread safe.

Fin

That's it! Just a taste of more advanced FP. Hopefully this series sparked interest. You have the tools to comprehend IO and ZIO programs now. Hopefully you see why they can be powerful: readable, maintainable, explicit, with great testing. Enjoy the journey.