Copying fields using Shapeless - Part 1

Sometimes we need to copy fields from one case class to another. Often the two case classes are quite similar, but we still need to copy each field manually. Shapeless can eliminate this boilerplate for us.

First let’s think why we may want to do this. We often face the scenario of collecting information in multiple steps, validating it along the way and performing the final operation when we have everything we need. Wizard style forms are a good example of this (booking a rail or air ticket).The ubiquitous case class is the usual data type employed but a single case class is not ideal:

1. If we use a single case class we have to accept that some fields will be Null until we reach the final page of the wizard (ouch!)

2. Alternatively we can make the fields optional but this introduces it’s own problems. A case class made up of entirely optional fields is pretty useless

3. The best solution is usually to use one case class per “page” then process them all together at the end.

Option 3 works well but at some point we probably want to aggregate the data from the various case classes together into a single object or otherwise transform/reshape it. Either way we end up copying data from one case class to another.

The scenario

I’m going to walk through a simplified car hire example in which our hypothetical user chooses her pickup and dropoff location, dates etc, chooses the type of car she wants and finally enters her own data. The three step flow can be modelled using these case classes:

import java.time.LocalDate

case class Location(pickup: String, dropOff: String, from: LocalDate, to: LocalDate)
case class Vehicle(vehicleCategory: String, automatic: Boolean, numDoors: Int)
case class Driver(driverAge: Int, nationality: String)

Our final reservation object will look like this:

import java.time.LocalDate

case class Reservation(
  pickup: String,
  dropOff: String,
  from: LocalDate,
  to: LocalDate,
  vehicleCategory: String,
  automatic: Boolean,
  numDoors: Int,
  driverAge: Int,
  nationality: String
)

For the purposes of this post I’m going to focus on how we can copy data from the three individual objects. I’ll do this without copying the fields manually

Getting started with Shapeless

As usual we add Shapeless as an SBT dependency:

"com.chuusai" %% "shapeless" % "2.3.3"

HLists

The HList is at the core of Shapeless, it’s like a tuple on steroids. Like a tuple an HList has an fixed number of elements which can be of different types. We can model our data using HLists:

import shapeless.{::, HNil}

type LocationH = String :: String :: LocalDate :: LocalDate :: HNil
type VehicleH = String :: Boolean :: Int :: HNil
type DriverH = Int :: String :: HNil

Note how the syntax is very similar to a normal list. We can model our final reservation using an HList also:

type ReservationH = String :: String :: LocalDate :: LocalDate :: String :: Boolean :: Int :: Int :: String :: HNil

We can create instances of our HLists using a very similar syntax:

val locationH: LocationH = "Malaga Airport" :: "Malaga Airport" :: LocalDate.of(2018,8,1) :: LocalDate.of(2018,8,10) :: HNil
val vehicleH: VehicleH = "Economy" :: false :: 4 :: HNil
val driverH: DriverH = 35 :: "British" :: HNil

Unlike tuples, HLists can be concatenated and this is where things get interesting. We can concatenate the three HLists together and we get a ReservationH type:

val reservationH: ReservationH = locationH ++ vehicleH ++ driverH

We’ve found a way to concatenate our three pieces of data together but the resulting type is pretty messy. Like tuples if we want to extract a field the best we can do is to pattern match:

reservationH match {
  case
    pickup ::
    dropOff ::
    from ::
    to ::
    vehicleCategory ::
    automatic ::
    numDoors ::
    driverAge ::
    nationality ::
    HNil => println(s"pickup location: $pickup")
}

From case classes to Hlists and back again

Fortunately Shapeless allows us to convert case classes to Hlists and vice versa:

import shapeless.{::, HNil, Generic}

val location = Location(pickup = "Malaga Airport", dropOff = "Malaga Airport", from = LocalDate.of(2018,8,1), to = LocalDate.of(2018,8,10))
val vehicle = Vehicle(vehicleCategory = "Economy", automatic = false, numDoors = 4)
val driver = Driver(driverAge = 35, nationality = "British")

val locationH: LocationH = Generic[Location].to(location)
val vehicleH: VehicleH = Generic[Vehicle].to(vehicle)
val driverH: DriverH = Generic[Driver].to(driver)

val reservationH: ReservationH = locationH ++ vehicleH ++ driverH

The Generic trait allows us to map from a case class (actually any Product or Coproduct type) to an HList. Generic[Location].to(location) means “transform the location case class instance to an Hlist”. Of course we can also go back again:

val reservation: Reservation = Generic[Reservation].from(reservationH)

We can actually get rid of our custom HList types (the compiler will infer them for us) and greatly simplify the code. By convention we typically use the term Repr for HList representations of case classes:

import shapeless.Generic

val location = Location(pickup = "Malaga Airport", dropOff = "Malaga Airport", from = LocalDate.of(2018,8,1), to = LocalDate.of(2018,8,10))
val vehicle = Vehicle(vehicleCategory = "Economy", automatic = false, numDoors = 4)
val driver = Driver(driverAge = 35, nationality = "British")

val locationRepr = Generic[Location].to(location)
val vehicleRepr = Generic[Vehicle].to(vehicle)
val driverRepr = Generic[Driver].to(driver)

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr
val reservation: Reservation = Generic[Reservation].from(reservationRepr)

Improving the design

Things are not quite as good as they seem. Lets modify our Location case class and swap the from and to dates around. We’ll leave the rest of the code unchanged:

case class Location(pickup: String, dropOff: String, to: LocalDate, from: LocalDate)

Everything compiles but look what happens when we get the reservation’s from date:

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr
val reservation: Reservation = Generic[Reservation].from(reservationRepr)
println(s"from date: ${reservation.from}")

2018-8-10 // Actually this is the to date

This weird behaviour happens because like tuples, HLists operate on the type and position of the fields, not the field names. As far as Shapeless is concerned we’ve just extracted two dates from the location and we pass two dates into the Reservation. However there is a fix for this …

LabelledGeneric

Shapeless includes a utility called LabelledGeneric which creates custom types on the fly based on the field type and the field name. I won’t go into the details but the end result is that the resulting Location HList looks something like this (conceptually):

PickupString :: DropOffString :: ToLocalDate :: FromLocalDate :: HNil

The relevant part of the Reservation HList would look something like

PickupString :: DropOffString :: FromLocalDate :: ToLocalDate :: HNil

In this scenario the types don’t align and therefore the code won’t compile:

import shapeless.LabelledGeneric

val LocationGen = LabelledGeneric[Location]
val VehicleGen = LabelledGeneric[Vehicle]
val DriverGen = LabelledGeneric[Driver]
val ReservationGen = LabelledGeneric[Reservation]

val locationRepr = LocationGen.to(location)
val vehicleRepr = VehicleGen.to(vehicle)
val driverRepr = DriverGen.to(driver)

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr
// This won't compile
val reservation: Reservation = ReservationGen.from(reservationRepr)

However if we swap the Location’s dates around to align with the Reservation:

case class Location(pickup: String, dropOff: String, from: LocalDate, to: LocalDate)

It will now compile. Checkout the 5-working-labelled-generic tag or fix it yourself ;)

Summary

Shapeless provides two useful utilities for copying data between arbitrary case classes. HLists are a generic representation of strongly typed data, like tuples on steroids. LabelledGeneric lets of map case classes to and from HLists. Used together we get behaviour similar to BeanUtils copyProperties but with full type safety!

What next

In my second post I will demonstrate how Shapeless can add and remove fields and reorder them