Implicit parameters when to use them? Part 2

In my previous blog post, I presented the way that implicit parameters work, as well as one of their design patterns: the environment pattern.

Today, I will present a second use case for implicit parameters: the typeclasses.

Let’s say that we are implementing a web framework. One of the basic use cases for our API is the ability to generate an HTTP response with a variety of JSON payloads. For example, we would like to define a method Ok which behaves like that:

val user   : User         = User(UserId(“1234”), "John Doe")
val userIds: List[UserId] = List(UserId(“1234”), UserId("9464"), UserId("0582"))

Ok(user)
// res: HttpResponse(200, {"userId" : "1234", "name" : "John Doe"})

Ok(userIds)
// res: HttpResponse(200, ["1234", "9464", "0582"])

Our first attempt to implement Ok will probably look like this:

def Ok[A](value: A): HttpResponse =
  HttpResponse(200, value.toJson)

The problem with this implementation is that everything is not serialisable to JSON. For example, there is no meaningful way to serialise a function.

Ok((x: Int) => x + 1) // ???

A more realistic counter example would be an application with internal domain objects and external Data Transfer Objects (DTO). In this scenario, we want to serialise the DTOs to JSON but not the domain objects and if by mistake we call Ok with a domain object, we would like to receive a compile-time error. One way to achieve this result is to define a generic interface JsonEncoder and modify the Ok function to require a JsonEncoder instance for the corresponding payload.

trait JsonEncoder[A] {
  def encode(value: A): Json
}

def Ok[A](value: A, encoder: JsonEncoder[A]): HttpResponse =
  HttpResponse(200, encoder.encode(value))

Then, we need to define JsonEncoder instances for User and for List[UserId] in order to use Ok.

val userEncoder       : JsonEncoder[User]         = ...
val listUserIdsEncoder: JsonEncoder[List[UserId]] = ...

Ok(user, userEncoder)
// res: HttpResponse(200, {"userId" : "1234", "name" : "John Doe"})

Ok(userIds, listUserIdsEncoder)
// res: HttpResponse(200, ["1234", "9464", "0582"])

This solution works but it is annoying to manually pass a JsonEncoder every time we want to produce an HTTP response because in most cases, there is only one JsonEncoder per type. Instead, we could make the JsonEncoder parameter implicit.

def Ok[A](value: A)(implicit encoder: JsonEncoder[A]): HttpResponse =
  HttpResponse(200, encoder.encode(value))

Thanks to the implicit parameter we regained our desired syntax.

implicit val userEncoder       : JsonEncoder[User]         = ...
implicit val listUserIdsEncoder: JsonEncoder[List[UserId]] = ...

Ok(user)
// res: HttpResponse(200, {"userId" : "1234", "name" : "John Doe"})

Ok(userIds)
// res: HttpResponse(200, ["1234", "9464", "0582"])

Now, the problem is that we need to define the implicit JSON encoders in the same scope where we use the method Ok. A simple improvement would be to package all JsonEncoder instances in an object and then import this object whenever we use Ok.

object JsonSerializer {
  implicit val userEncoder       : JsonEncoder[User]         = ...
  implicit val listUserIdsEncoder: JsonEncoder[List[UserId]] = ...
}

import JsonSerializer._

Ok(user)
// res: HttpResponse(200, {"userId" : "1234", "name" : "John Doe"})

It is better but it is not really modular to define all instances into a single object. Additionally, we need to remember the location of this object so that we can import it and personally, I always forget where it is defined.

To solve this issue, the Scala programming language added a special rule to search for implicits.

When the compiler searches for an implicit JsonEncoder[User], it will also inspect the companion of User.

Or more generally,

When the compiler searches for an implicit F[X], it will also inspect the companion of X.

So if we define the implicit JsonEncoder instance in the companion object of User, we can use Ok(user) without any import! It is almost as if we had defined the JsonEncoder[User] instance at the global level.

case class User(userId: UserId, email: String)

object User {
  implicit val encoder: JsonEncoder[User] = ...
}

// without import

Ok(user)
// res: HttpResponse(200, {"userId" : "1234", "name" : "John Doe"})

However, we can’t use the same trick for Ok(userIds: List[UserId]). This is because List is a class defined inside the standard library and there is no JsonEncoder in its companion object. That’s why the Scala programming language implemented a second special rule to search for implicits.

When the compiler searches for an implicit JsonEncoder[User], it will also inspect the companion of JsonEncoder.

Or more generally,

When the compiler searches for an implicit F[X], it will also inspect the companion of F.

This is the perfect solution for types we don’t control within our applications such as String, Int, LocalDate or List.

trait JsonEncoder[A] {
  def encode(value: A): Json
}

object JsonEncoder {
  implicit val string: JsonEncoder[String]    = ...
  implicit val int   : JsonEncoder[Int]       = ...
  implicit val date  : JsonEncoder[LocalDate] = ...

  implicit val list  : JsonEncoder[List[???]] = ...
}

However, we still have a problem because List takes a type parameter and we don’t want to implement a List encoder for each element type! So, maybe we could make JsonEncoder.list generic:

object JsonEncoder {
  implicit def list[A]: JsonEncoder[ListA]] = ...
}

But this would mean that we can encode List of any elements to Json and we have seen that some types are NOT serialisable to JSON. For example, a list of functions or a list of model objects should not be serialisable to JSON! So, we would like to express the following condition: let’s create a JsonEncoder for List of all elements as long as the elements themselves are serialisable to JSON. To do this in Scala, we need to add an implicit parameter to the instance definition.

object JsonEncoder {
  implicit def list[A](implicit valueEncoder: JsonEncoder[A]): JsonEncoder[ListA]] = ...
}

Now, this starts to be quite complex and confusing. This is because the implicit keyword is used twice for two reasons:

The implicit def defines a new implicit value.
The implicit valueEncoder introduces a requirement: the instance only exists for value types which have an implicit JsonEncoder.

Scala 3 clarifies this situation by splitting the implicit keywords into two new keywords:

given to define a new implicit value.
using to require/use an implicit value.

For example in Scala 3, the companion object of JsonEncoder would look like this:

object JsonEncoder {
  given string: JsonEncoder[String]  = ...

  given list[A](using valueEncoder: JsonEncoder[A]) : JsonEncoder[List[A]] = ...
}

As we rarely need to use an implicit value by name, Scala 3 made it optional to name them. So, we could have also defined JsonEncoder companion object like this:

object JsonEncoder {
  given JsonEncoder[String]  = …

  given[A](using JsonEncoder[A]): JsonEncoder[List[A]] = …
}

Let’s review how the scala compiler finds an implicit value of type JsonEncoder[List[UserId]] in the following code sample:

Ok(userIds)
// res: HttpResponse(200, ["1234", "9464", "0582"])

The compiler searches in the current scope. It doesn’t exist. The compiler searches in the companion object of List. It doesn’t exist. The compiler searches in the companion object of JsonEncoder. There it finds that there might be an implicit JsonEncoder[List[UserId]] if there is an implicit JsonEncoder[UserId] defined. The compiler searches in the companion object of UserId and it finds the required implicit!

This process of creating implicit instances using other implicit instances is called typeclass derivation. If you have used Scala serialization libraries such as circe or Play Json [links], you have most likely already used this feature. For example,:

import io.circe.generic.semiauto._

case class User(userId: UserId, email: String)

object User {
  implicit val encoder: Encoder[User] = deriveEncoder[User]
}

deriveEncoder is a macro which creates an Encoder instance for User by recursively searching for Encoder instances for each field of User. In other words, deriveEncoder can create an Encoder for any case classes where all fields are themselves serialisable to JSON.

In Scala 2, we need to use macros or libraries such as shapeless or magnolia [links] to create complex typeclass derivation but in Scala 3, this feature is built in to the language. In this article, I will only show you how to derive a typeclass instance in Scala 3. We only need to add a derives keyword after the class definition!

case class User(userId: UserId, email: String) derives JsonEncoder

Typeclass Summary

Typeclasses allow us to describe a common interface for types that are unrelated to each other (no inheritance).

Typeclasses allow us to write generic logic which makes this feature particularly useful for libraries and frameworks.

A typeclass is an interface with a type parameter. This interface contains one-or-more methods which either consume or produce value of the type parameter.

trait JsonEncoder[A] {
  def encode(value: A): Json
}

trait JsonDecoder[A] {
  def decode(value: Json): Either[String, A]
}

Instances of a typeclass should be defined and consumed implicitly.

Typeclass instances should be defined in the companion objects of the class we control (e.g. User, UserId) or in the companion object of the typeclass for types defined elsewhere (e.g. Int, String, LocalDate). This way, instances are globally accessible and we don’t need any import at the call site.

case class User(userId: UserId, email: String)

object User {
  implicit val encoder: JsonEncoder[User] = ...
}

trait JsonEncoder[A] {
  def encode(value: A): Json
}

object JsonEncoder {
  implicit val string: JsonEncoder[String]    = ...
  implicit val int   : JsonEncoder[Int]       = ...
  implicit val date  : JsonEncoder[LocalDate] = ...

  implicit val list  : JsonEncoder[List[???]] = ...
}

Typeclass instances can be automatically implemented using a feature called typeclass derivation.

When not to use typeclasses

Here we need to follow the golden rule of implicits:

Passing values implicitly should only be used when the values are obvious.

This means there shouldn’t be any doubts about the typeclass instance that is selected. If there are several good options available and you need to check your imports or run a debugger to identify the instance that was selected, then you shouldn’t use implicit. Instead, you should pass the values explicitly.

For example, the standard library has a typeclass called Ordering which is used for sorting collections. In my opinion, Ordering shouldn’t have been defined as a typeclass. Let me show you why.

Ordering’s main method is compare which tells us if two values are equal or which one is lower/greater than the other.

trait Ordering[A] {
  def compare(left: A, right: A): Int
}

Ordering is used throughout the Scala standard library for sorting or finding the min/max value of a collection. For example,

trait List[A] {
  def sorted(implicit ordering: Ordering[A]): List[A] = ...
}

As you can see Ordering is a typeclass because:

It is an interface with a type parameter.
Its method consumes values of the type parameter.
Ordering instances are consumed implicitly.

Now, the problem with Ordering is that the “default” order is not obvious. For example, what do you think would be the result of:

List(8, 1, 9).sorted

Will it return List(1, 8, 9) by sorting in ascending order or List(9, 8, 1) by sorting in descending order – or something else? It turns out that Scala sorts in ascending order for numbers by default, but this isn’t obvious at all!

Additionally, say that we need to sort a list in descending order, to do this we would need to define a custom implicit in scope.

implicit val descendingIntOrdering: Ordering[Int] = Ordering.Int.reverse

List(8,1,9).sorted
// res: List[Int] = List(9, 8, 1)

But there might be many lines of code between the implicit definition and the call to sorted. This means it isn’t straightforward to identify the implicit value that is selected by the compiler. Furthermore, we may want to sort two data structures in the same scope using separate Ordering, which means we either need to create artificial scopes to separate them, or pass the Orderings values explicitly.

In my opinion, Ordering shouldn’t have been a typeclass. Ordering values should always be passed explicitly to avoid any confusion. Here is my preferred version of Ordering:

trait Ordering[A] {
  def compare(left: A, right: A): Int
}

object Ordering {
  object Int {
    val ascending: Ordering[Int] = …
    val descending: Ordering[Int] = …
  }
}

trait List[A] {
  def sorted(ordering: Ordering[A]): List[A] = ...
}

Then, we call sorted we would have to pass the Ordering instance explicitly. This makes the code more verbose but everyone will know what the code does which is much more important than saving a few keystrokes.

List(8,1,9).sorted(Ordering.Int.ascending)
// res: List[Int] = List(1, 8, 9)

List(8,1,9).sorted(Ordering.Int.descending)
// res: List[Int] = List(9, 8, 1)

Footnotes: This article isn’t meant to be a complete guide to typeclasses. I have made some approximations and I didn’t mention some advanced features such as implicit prioritisation. My goal is to present the main idea and pitfalls behind typeclasses.

Implicit parameters when to use them? Part 2

Typeclass Summary

When not to use typeclasses

Subscribe to receive the latest Scala jobs in your inbox