A Handy-Dandy Typesafe Config Implicit Wrapper in base-api

In an earlier post I released base-api, a template project for developing Scala API servers. In this post I will describe the convenient configuration system included in that project.

The Typesafe Config project is easily the best, most convenient, easiest to use, easiest to understand extensible configuration management system I've ever had the pleasure of working with. It has no dependencies, it reads JSON and HOCON (a JSON superset with substitutions, etc.), has excellent merge strategies, accepts command line runtime overrides automatically, and is of course very well tested (coming from Typesafe after all). Ok great, so what does that look like? Here's an example REST API reference.conf section:

rest {
  protocol = http
  host = "0.0.0.0"
  port = 8080
  timeout = 2 seconds
}

Which might be paired with a production.conf that overlays these values:

rest {
  protocol = https
  port = 443
  timeout = 5 seconds
}

We have 4 values (addressable as rest.protocol, etc.) that have 3 distinct data types - string, string, int, FiniteDuration. The code that requires these values is probably a case class or other typed input:

  case class RestService(
    protocol: String, 
    host: String, 
    port: Int, 
    timeout: FiniteDuration
  )

And to obtain them in at runtime we might have some code that looks like this:

  import scala.concurrent.duration._ // provides implicit millis->FiniteDuration cnvsn
  
  val defaultConf = ConfigFactory.defaultReference()
  val conf = ConfigFactory.load().withFallback(defaultConf)
  
  val secondsFormatter = new PeriodFormatterBuilder()
    .appendSeconds().appendSuffix(" seconds").toFormatter
  val timeoutPeriod = secondsFormatter.parsePeriod(conf.getString("rest.timeout"))  
  
  val restService = RestService(
    conf.getString("rest.protocol"),
    conf.getString("rest.host"),
    conf.getInt("rest.port"),
    timeoutPeriod.toStandardDuration.getMillis.millis
  )

Doesn't look horrible, but could become a bit nasty once we have hundreds of configurable values - a typical fate for any actively developed API. Now let's see what configuring the RestService would look like in base-api where we have our fancy wrapper:

  import base.common.config.BaseConfig._ // provides HOCON implicit cnvsns 

  val REST = "rest"
  val restService = RestService(
    Keys(REST, "protocol"),
    Keys(REST, "host"),
    Keys(REST, "port"),
    Keys(REST, "timeout")
  )

Awesome! We are no longer responsible for specifying the types of data to be retrieved, and particularly in the case of the FiniteDuration there's some real magic going on - somewhere, some code is figuring out how to get from a string in the conf 5 seconds to a strongly typed duration (and it would work equally well if we have put 5 hours or 1 day).

The something, somewhere is the BaseConfig which provides a chain of implicits that will figure out how to populate config values for just about anything HOCON supports, and are easily extensible to custom data types. The primary built-in custom data type is Period, which cycles through a number of formatters attempting to find one that properly parse the provided config value (e.g. "milli", "millis", "millisecond", "milliseconds").

Easy stuff so far. Let's make it interesting.

Ok let's say we need something more complicated. Assume we have this conf:

rest {
  protocol = http
  // ...
  endpoints = [
    { path = foo, methods = [get]       },
    { path = bar, methods = [get, post] }
  ]
}

To configure these values:

  object Methods extends Enumeration {
    type Method = Value
    val GET = "get"
    val POST = "post"
    // ...
  }
  case class Endpoint(path: String, methods: Set[Method])
  case class RestService(
    protocol: String, 
    // ... host, port, etc ...
    endpoints: List[Endpoint]
  )

Unfortunately we don't have any implicits smart enough to figure out how to populate a Set[Method] data type let alone a List[Endpoint] data type. But with just a wee bit of extra work we can get these running as smoothly as the simpler types:

  implicit def string2Method(s: String): Method = Methods.withName(s)

  val REST = "rest"
  val restService = RestService(
    Keys(REST, "protocol"),
    getConfigList(Keys(REST, "endpoints")).map { endpointConfig =>
      implicit val config = new BaseConfig(endpointConfig)
      Endpoint(
        Keys("path"), 
        Keys("methods")
      )
    }
  )

All we had to do was sprinkle a little more implicit magic on it and bingo, we get this rather complex strong type hierarchy built for us out of our config DSL. What's happening in the above snippet is that we are saying the endpoint config value is itself a list of Typesafe Configs. This allows us access to the full power of the implicit chain but scoped down to the contents of each index of this value's list of properties. Neat huh?

This is cool, but isn't implicit magic bad?

Some people feel pretty strongly that implicits should be used with extreme caution and that they can quickly run away in a large multi-developer codebase to an indecipherable unmaintainable mess. To that I say: I agree. You have to be really careful with them, and frankly I think using them in core business logic is a mistake that comes back to haunt people frequently - though it's pretty hard to get around their usage in core libs like JSON DSLs. For their usage here, I think it will be OK even in medium to larger size projects, as long as they remain constrained to the configuration system and only deal with common primitives. On the other hand, I wouldn't fault a dev team for saying "no way Jose, not in my codebase" ;)