In Part 1 and Part 2 , we have presented Json combinators for Reads[T], Writes[T] and Format[T]. So now you should know how to validate JSON and convert into any structure you can write in Scala and back to JSON. But as soon as I’ve begun to use those combinators to write web applications, I almost immediately encountered a case : read JSON from network, validate it and convert it into… JSON.

Introducing JSON coast-to-coast design

Are we doomed to convert JSON to OO?

For a few years now, in almost all web frameworks (except recent JS serverside stuff maybe in which JSON is the default data structure), we have been used to get JSON from network and convert JSON (or even POST/GET data) into OO structures such as classes (or case classes in Scala). Why?

• for a good reason : OO structures are “language-native” and allows manipulating data with respect to your business logic in a seamless way while ensuring isolation of business logic from web layers.
• for a more questionable reason : ORM frameworks talk to DB only with OO structures and we have (kind of) convinced ourselves that it was impossible to do else… with the well-known good & bad features of ORMs… (not here to criticize those stuff)

Is OO conversion really the default usecase?

In many cases, you don’t really need to perform any real business logic with data but validating/transforming before storing or after extracting.

Let’s take the CRUD case:

• You just get the data from the network, validate them a bit and insert/update into DB.
• In the other way, you just retrieve data from DB and send them outside.

So, generally, for CRUD ops, you convert JSON into a OO structure just because the frameworks are only able to speak OO.

I don’t say or pretend you shouldn’t use JSON to OO conversion but maybe this is not the most common case and we should keep conversion to OO only when we have real business logic to fulfill.

New tech players change the way of manipulating JSON

Besides this fact, we have some new DB types such as Mongo (or CouchDB) accepting document structured data looking almost like JSON trees (isn’t BSON, Binary JSON?).
With these DB types, we also have new great tools such as ReactiveMongo which provides reactive environment to stream data to and from Mongo in a very natural way.
I’ve been working with Stephane Godbillon to integrate ReactiveMongo with Play2.1 while writing the Play2-ReactiveMongo module. Besides Mongo facilities for Play2.1, this module provides Json To/From BSON conversion typeclasses.

So it means you can manipulate JSON flows to and from DB directly without even converting into OO.

JSON coast-to-coast design

Taking this into account, we can easily imagine the following:

• receive JSON,
• validate JSON,
• transform JSON to fit expected DB document structure,
• directly send JSON to DB (or somewhere else)

This is exactly the same case when serving data from DB:

• extract some data from DB as JSON directly,
• filter/transform this JSON to send only mandatory data in the format expected by the client (for ex, you don’t want some secure info to go out),
• directly send JSON to the client

In this context, we can easily imagine manipulating a flow of JSON data from client to DB and back without any (explicit) transformation in anything else than JSON.
Naturally, when you plug this transformation flow on reactive infrastructure provided by Play2.1, it suddenly opens new horizons.

This is the so-called (by me) JSON coast-to-coast design:

• Don’t consider JSON data chunk by chunk but as a continuous flow of data from client to DB (or else) through server,
• Treat the JSON flow like a pipe that you connect to others pipes while applying modifications, transformations alongside,
• Treat the flow in a fully asynchronous/non-blocking way.

This is also one of the reason of being of Play2.1 reactive architecture…
I believe considering your app through the prism of flows of data changes drastically the way you design your web apps in general. It may also open new functional scopes that fit today’s webapps requirements quite better than classic architecture. Anyway, this is not the subject here ;)

So, as you have deduced by yourself, to be able to manipulate Json flows based on validation and transformation directly, we needed some new tools. JSON combinators were good candidates but they are a bit too generic.
That’s why we have created some specialized combinators and API called JSON transformers to do that.

JSON transformers are Reads[T <: JsValue]

You may tell JSON transformers are just f:JSON => JSON.
So a JSON transformer could be simply a Writes[A <: JsValue].
But, a JSON transformer is not only a function: as we said, we also want to validate JSON while transforming it.
As a consequence, a JSON transformer is a Reads[A <: Jsvalue].

Keep in mind that a Reads[A <: JsValue] is able to transform and not only to read/validate

Recent Play2 JSON syntax evolutions

As you know, Json API for Play2.1 was still draft and has evolved since I began writing article part 1/2.
We have changed a few things since (nothing conceptual, just cosmetics).

Syntax clarification

Reads[A] andThen Reads[B] has been renamed Reads[A] andKeep Reads[B] (keep the right side result)

Reads[A] provided Reads[B] has been renamed Reads[A] keepAnd Reads[B] (keep the left side result and is symmetric with andKeep)

Remarkable new Reads[A] features

Reads[A <: JsValue] andThen Reads[B]

andThen has the classic Scala semantic of function composition : it applies Reads[A <: JsValue] on JSON retrieving a JsValue and then applies Reads[B] on this JsValue.

Reads[A <: JsValue].map(f: A => B): Reads[B]

map is the classic and always very useful Scala map function.

Reads[A <: JsValue].flatMap(f: A => Reads[B]): Reads[B]

flatMap is the classic Scala flatMap function.

JSON transformers case by case

A few reminders

JSON new syntax

In code samples, we’ll use the following JSON.

1
2
3
4
5
6
7
8
9
10
11
12
13

{
  "key1" : "value1",
  "key2" : {
    "key21" : 123,
    "key22" : true,
    "key23" : [ "alpha", "beta", "gamma"]
    "key24" : {
      "key241" : 234.123,
      "key242" : "value242"
    }
  },
  "key3" : 234
}

Remind that, in Play2, you can write this JSON as following.

1
2
3
4
5
6
7
8
9
10
11
12
13

val json = Json.obj(
  "key1" -> "value1",
  "key2" -> Json.obj(
    "key21" -> 123,
    "key22" -> true,
    "key23" -> Json.arr("alpha", "beta", "gamma"),
    "key24" -> Json.obj(
      "key241" -> 234.123,
      "key242" -> "value242"
    )
  ),
  "key3" -> 234
)

Defining Play2 JSON action controller

Here is how you would write a Play2.1 action controller to receive and manipulate/validate JSON.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

object YourController {
  …

  // not implicit because I want to control which transformer I use
  val jsonTransformer: Reads[JsValue] = …

  def doit = Action(parse.json) { request =>
      request.body.validate(jsonTransformer).map { js =>
          // do something
          Ok(...)  
      }.recover { error =>
          BadRequest(…)
          }
  }
  
  …
}

Please note the JsResult.map and JsResult.recover functions allowing to compose result and deal with errors.

Now let’s describe JSON transformers with samples

Case 1: Pick JSON value in JsPath

Pick value as JsValue

1
2
3
4
5
6
7
8
9
10

import play.api.libs.json._

val jsonTransformer = (__ \ 'key2 \ 'key23).json.pick

scala> json.validate(jsonTransformer)
res9: play.api.libs.json.JsResult[play.api.libs.json.JsValue] =
  JsSuccess(
    ["alpha","beta","gamma"],
    /key2/key23
  )

(__ \ 'key2 \ 'key23).json...

• All JSON transformers are in JsPath.json.

(__ \ 'key2 \ 'key23).json.pick

• pick is a Reads[JsValue] which picks the value IN the given JsPath. Here ["alpha","beta","gamma"]

JsSuccess(["alpha","beta","gamma"],/key2/key23)

• This is a simply successful JsResult
• For info, /key2/key23 represents the JsPath where data were read but don’t care about it, it’s mainly used by Play API to compose JsResult(s))
• ["alpha","beta","gamma"] is just due to the fact that we have overriden toString

To Remember jsPath.json.pick gets ONLY the value inside the JsPath

Pick value as Type

1
2
3
4
5
6
7
8
9
10

import play.api.libs.json._

val jsonTransformer = (__ \ 'key2 \ 'key23).json.pick[JsArray]

scala> json.validate(jsonTransformer)
res10: play.api.libs.json.JsResult[play.api.libs.json.JsArray] =
  JsSuccess(
    ["alpha","beta","gamma"],
    /key2/key23
  )

(__ \ 'key2 \ 'key23).json.pick[JsArray]

• pick[T] is a Reads[T <: JsValue] which picks the value (as a JsArray in our case) IN the given JsPath

To Remember: jsPath.json.pick[T <: JsValue] extracts ONLY the typed value inside the JsPath

Case 2: Pick branch following JsPath

Pick branch as JsValue

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import play.api.libs.json._

val jsonTransformer = (__ \ 'key2 \ 'key24 \ 'key241).json.pickBranch

scala> json.validate(jsonTransformer)
res11: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
  {
    "key2": {
      "key24":{
        "key241":234.123
      }
    }
  },
  /key2/key24/key241
  )

(__ \ 'key2 \ 'key23).json.pickBranch

• pickBranch is a Reads[JsValue] which picks the branch from root to given JsPath

{"key2":{"key24":{"key242":"value242"}}}

• The result is the branch from root to given JsPath including the JsValue in JsPath

To Remember: jsPath.json.pickBranch extracts the single branch down to JsPath + the value inside JsPath

Case 3: Copy a value from input JsPath into a new JsPath

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import play.api.libs.json._

val jsonTransformer = (__ \ 'key25 \ 'key251).json.copyFrom( (__ \ 'key2 \ 'key21).json.pick )

scala> json.validate(jsonTransformer)
res12: play.api.libs.json.JsResult[play.api.libs.json.JsObject]
  JsSuccess(
    {
      "key25":{
        "key251":123
      }
    },
    /key2/key21
  )

(__ \ 'key25 \ 'key251).json.copyFrom( reads: Reads[A <: JsValue] )

• copyFrom is a Reads[JsValue]
• copyFrom reads the JsValue from input JSON using provided Reads[A]
• copyFrom copies this extracted JsValue as the leaf of a new branch corresponding to given JsPath

{"key25":{"key251":123}}

• copyFrom reads value 123
• copyFrom copies this value into new branch (__ \ 'key25 \ 'key251)

To Remember: jsPath.json.copyFrom(Reads[A <: JsValue]) reads value from input JSON and creates a new branch with result as leaf

Case 4: Copy full input Json & update a branch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import play.api.libs.json._

val jsonTransformer = (__ \ 'key2 \ 'key24).json.update(
  __.read[JsObject].map{ o => o ++ Json.obj( "field243" -> "coucou" ) }
)

scala> json.validate(jsonTransformer)
res13: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
    {
      "key1":"value1",
      "key2":{
        "key21":123,
        "key22":true,
        "key23":["alpha","beta","gamma"],
        "key24":{
          "key241":234.123,
          "key242":"value242",
          "field243":"coucou"
        }
      },
      "key3":234
    },
  )

(__ \ 'key2).json.update(reads: Reads[A < JsValue])

• is a Reads[JsObject]

(__ \ 'key2 \ 'key24).json.update(reads) does 3 things:

• extracts value from input JSON at JsPath (__ \ 'key2 \ 'key24)
• applies reads on this relative value and re-creates a branch (__ \ 'key2 \ 'key24) adding result of reads as leaf
• merges this branch with full input JSON replacing existing branch (so it works only with input JsObject and not other type of JsValue)

JsSuccess({…},)

• Just for info, there is no JsPath as 2nd parameter there because the JSON manipulation was done from Root JsPath

To Remember: jsPath.json.update(Reads[A <: JsValue]) only works for JsObject, copies full input JsObject and updates jsPath with provided Reads[A <: JsValue]

Case 5: Put a given value in a new branch

1
2
3
4
5
6
7
8
9
10
11
12
13

import play.api.libs.json._

val jsonTransformer = (__ \ 'key24 \ 'key241).json.put(JsNumber(456))

scala> json.validate(jsonTransformer)
res14: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
    {
      "key24":{
        "key241":456
      }
    },
  )

(__ \ 'key24 \ 'key241).json.put( a: => JsValue )

• is a Reads[JsObject]

(__ \ 'key24 \ 'key241).json.put( a: => JsValue )

• creates a new branch (__ \ 'key24 \ 'key241)
• puts a as leaf of this branch.

jsPath.json.put( a: => JsValue )

• takes a JsValue argument passed by name allowing to pass even a closure to it.

jsPath.json.put

• doesn’t care at all about input JSON
• simply replace input JSON by given value

To Remember: jsPath.json.put( a: => Jsvalue ) creates a new branch with a given value without taking into account input JSON

Case 6: Prune a branch from input JSON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import play.api.libs.json._

val jsonTransformer = (__ \ 'key2 \ 'key22).json.prune

scala> json.validate(jsonTransformer)
res15: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
    {
      "key1":"value1",
      "key3":234,
      "key2":{
        "key21":123,
        "key23":["alpha","beta","gamma"],
        "key24":{
          "key241":234.123,
          "key242":"value242"
        }
      }
    },
    /key2/key22/key22
  )

(__ \ 'key2 \ 'key22).json.prune

• is a Reads[JsObject] that works only with JsObject

(__ \ 'key2 \ 'key22).json.prune

• removes given JsPath from input JSON (key22 has disappeared under key2)

Please note the resulting JsObject hasn’t same keys order as input JsObject. This is due to the implementation of JsObject and to the merge mechanism. But this is not important since we have overriden JsObject.equals method to take this into account.

To Remember: jsPath.json.prune only works with JsObject and removes given JsPath form input JSON) Please note that:

• prune doesn’t work for recursive JsPath for the time being
• if prune doesn’t find any branch to delete, it doesn’t generate any error and returns unchanged JSON.

More complicated cases

Case 7: Pick a branch and update its content in 2 places

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import play.api.libs.json._
import play.api.libs.json.Reads._

val jsonTransformer = (__ \ 'key2).json.pickBranch(
  (__ \ 'key21).json.update(
    of[JsNumber].map{ case JsNumber(nb) => JsNumber(nb + 10) }
  ) andThen
  (__ \ 'key23).json.update(
    of[JsArray].map{ case JsArray(arr) => JsArray(arr :+ JsString("delta")) }
  )
)

scala> json.validate(jsonTransformer)
res16: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
    {
      "key2":{
        "key21":133,
        "key22":true,
        "key23":["alpha","beta","gamma","delta"],
        "key24":{
          "key241":234.123,
          "key242":"value242"
        }
      }
    },
    /key2
  )

(__ \ 'key2).json.pickBranch(reads: Reads[A <: JsValue])

• extracts branch __ \ 'key2 from input JSON and applies reads to the relative leaf of this branch (only to the content)

(__ \ 'key21).json.update(reads: Reads[A <: JsValue])

• updates (__ \ 'key21) branch

of[JsNumber]

• is just a Reads[JsNumber]
• extracts a JsNumber from (__ \ 'key21)

of[JsNumber].map{ case JsNumber(nb) => JsNumber(nb + 10) }

• reads a JsNumber (value 123 in __ \ 'key21)
• uses Reads[A].map to increase it by 10 (in immutable way naturally)

andThen

• is just the composition of 2 Reads[A]
• first reads is applied and then result is piped to second reads

of[JsArray].map{ case JsArray(arr) => JsArray(arr :+ JsString("delta")

• reads a JsArray (value [alpha, beta, gamma] in __ \ 'key23)
• uses Reads[A].map to append JsString("delta") to it

Please note the result is just the __ \ 'key2 branch since we picked only this branch

Case 8: Pick a branch and prune a sub-branch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import play.api.libs.json._
import play.api.libs.json.Reads._

val jsonTransformer = (__ \ 'key2).json.pickBranch(
  (__ \ 'key23).json.prune
)

scala> json.validate(jsonTransformer)
res18: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
    {
      "key2":{
        "key21":123,
        "key22":true,
        "key24":{
          "key241":234.123,
          "key242":"value242"
        }
      }
    },
    /key2/key23
  )

(__ \ 'key2).json.pickBranch(reads: Reads[A <: JsValue])

• extracts branch __ \ 'key2 from input JSON and applies reads to the relative leaf of this branch (only to the content)

(__ \ 'key23).json.prune

• removes branch __ \ 'key23 from relative JSON

Please remark the result is just the __ \ 'key2 branch without key23 field.

What about combinators?

I stop there before it becomes boring (if not yet)…

Just keep in mind that you have now a huge toolkit to create generic JSON transformers.
You can compose, map, flatmap transformers together into other transformers. So possibilities are almost infinite.

But there is a final point to treat: mixing those great new JSON transformers with previously presented Reads combinators. This is quite trivial as JSON transformers are just Reads[A <: JsValue]

Let’s demonstrate by writing a Gizmo to Gremlin JSON transformer.

Here is Gizmo:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

val gizmo = Json.obj(
  "name" -> "gizmo",
  "description" -> Json.obj(
      "features" -> Json.arr( "hairy", "cute", "gentle"),
      "size" -> 10,
      "sex" -> "undefined",
      "life_expectancy" -> "very old",
      "danger" -> Json.obj(
          "wet" -> "multiplies",
          "feed after midnight" -> "becomes gremlin"
      )
  ),
  "loves" -> "all"
)

Here is Gremlin:

1
2
3
4
5
6
7
8
9
10
11

val gremlin = Json.obj(
  "name" -> "gremlin",
  "description" -> Json.obj(
      "features" -> Json.arr("skinny", "ugly", "evil"),
      "size" -> 30,
      "sex" -> "undefined",
      "life_expectancy" -> "very old",
      "danger" -> "always"
  ),
  "hates" -> "all"
)

Ok let’s write a JSON transformer to do this transformation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.json.util._

val gizmo2gremlin = (
  (__ \ 'name).json.put(JsString("gremlin")) and
  (__ \ 'description).json.pickBranch(
      (__ \ 'size).json.update( of[JsNumber].map{ case JsNumber(size) => JsNumber(size * 3) } ) and
      (__ \ 'features).json.put( Json.arr("skinny", "ugly", "evil") ) and
      (__ \ 'danger).json.put(JsString("always"))
      reduce
  ) and
  (__ \ 'hates).json.copyFrom( (__ \ 'loves).json.pick )
) reduce

scala> gizmo.validate(gizmo2gremlin)
res22: play.api.libs.json.JsResult[play.api.libs.json.JsObject] =
  JsSuccess(
    {
      "name":"gremlin",
      "description":{
        "features":["skinny","ugly","evil"],
        "size":30,
        "sex":"undefined",
        "life_expectancy":
        "very old","danger":"always"
      },
      "hates":"all"
    },
  )

Here we are ;)
I’m not going to explain all of this because you should be able to understand now.
Just remark:

(__ \ 'features).json.put(…) is after (__ \ 'size).json.update so that it overwrites original (__ \ 'features)

(Reads[JsObject] and Reads[JsObject]) reduce

• It merges results of both Reads[JsObject] (JsObject ++ JsObject)
• It also applies the same JSON to both Reads[JsObject] unlike andThen which injects the result of the first reads into second one.

Conclusion

After 3 long articles, I think we have done a full 360° around new features brought by Play2.1 JSON API.
I hope you glimpse the whole new world of possiblities it can bring to us.
Personnally, I’ve begun using it in projects and I don’t yet see the limits of it and how I’ll end using it.

The only thing I can say is that it has changed my way of manipulating JSON data flows in general.

Next article coming soon: an applied example of a webapp following Json coast-to-coast design with ReactiveMongo

Have fun ;););)

In Part 1 article, we gave a quick overview of new Play2.1 JSON API features around JsPath and Reads combinators. It’s funny to be able to read JSON to Scala structures but it’s better to be able to write Scala structures to JSON (Writes[T]) and it’s even better to mix Reads and Writes (Format[T]) sometimes.

Now let’s focus on Writes and Format in the details ;)

Writes[T] hasn’t changed (except combinators)

Writes in Play2.0.x

Do you remember how you had to write a Json Writes[T] in Play2.0.x ?
You had to override the writes function.

1
2
3
4
5
6
7

trait Writes[-A] {
  self =>
  /**
   * Convert the object into a JsValue
   */
  def writes(o: A): JsValue
}

Take the same simple case class we used in Part 1:

1
2
3
4
5

case class Creature(
  name: String,
  isDead: Boolean,
  weight: Float
)

In Play2.0.x, you would write your Writes[Creature] as following (using new Json syntax to re-show it even if it didn’t exist in Play2.0.x ;) ):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import play.api.libs.json._

implicit val creatureWrites = new Writes[Creature] {
  def writes(c: Creature): JsValue = {
    Json.obj(
      "name" -> c.name,
      "isDead" -> c.isDead,
      "weight" -> c.weight
    )
  }
}

scala> val gizmo = Creature("gremlins", false, 1.0F)
scala> val gizmojs = Json.toJson(gizmo)
gizmojs: play.api.libs.json.JsValue = {"name":"gremlins","isDead":false,"weight":1.0}

Writes in Play2.1.x

No suspense to be kept: in Play2.1, you write Writes exactly in the same way :D

So what’s the difference?
As presented in Part 1, Reads could be combined using simple logical operators.
Using functional Scala power, we were able to provide combinators for Writes[T].

If you want more theoretical aspects about the way it was implemented based on generic functional structures adapted to our needs, you can read this post “Applicatives are too restrictive, breaking Applicatives and introducing Functional Builders” written by @sadache

Writes main change: combinators

Once again, code first: re-writing previous Writes[T] using combinators.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

// IMPORTANT import this to have the required tools in your scope
import play.api.libs.json._
// imports required functional generic structures
import play.api.libs.json.util._
// imports implicit structure for Writes only (always try to import only what you need)
import play.api.libs.json.Writes._

implicit val creatureWrites = (
  (__ \ "name").write[String] and
  (__ \ "isDead").write[Boolean] and
  (__ \ "weight").write[Float]
)(unlift(Creature.unapply))

// or using the operators inspired by Scala parser combinators for those who know them
implicit val creatureWrites = (
  (__ \ "name").write[String] ~
  (__ \ "isDead").write[Boolean] ~
  (__ \ "weight").write[Float]
)(unlift(Creature.unapply))

scala> val c = Creature("gremlins", false, 1.0F)
scala> val js = Json.toJson(c)
js: play.api.libs.json.JsValue = {"name":"gremlins","isDead":false,"weight":1.0}

It looks exactly like Reads[T] except a few things, isn’t it?
Let’s explain a bit (by copying Reads article changing just a few things… I’m lazy ;)):

import play.api.libs.json.Writes._

It imports only the required stuff for Writes[T] without interfering with other imports.

(__ \ "name").write[String]

You apply write[String] on this JsPath (exactly the same as Reads)

and is just an operator meaning Writes[A] and Writes[B] => Builder[Writes[A ~ B]]

• A ~ B just means Combine A and B but it doesn’t suppose the way it is combined (can be a tuple, an object, whatever…)
• Builder is not a real type but I introduce it just to tell that the operator and doesn’t create directly a Writes[A ~ B] but an intermediate structure that is able to build a Writes[A ~ B] or to combine with another Writes[C]

(…)(unlift(Creature.unapply)) builds a Writes[Creature]

• (__ \ "name").write[String] and (__ \ "isDead").write[Boolean] and (__ \ "weight").write[Float] builds a Builder[Writes[String ~ Boolean ~ Float])] but you want a Writes[Creature].
• So you apply the Builder[Writes[String ~ Boolean ~ String])] to a function Creature => (String, Boolean, Float) to finally obtain a Writes[Creature]. Please note that it may seem a bit strange to provide Creature => (String, Boolean, Float) to obtain a Writes[Creature] from a Builder[Writes[String ~ Boolean ~ String])] but it’s due to the contravariant nature of Writes[-T].
• We have Creature.unapply but its signature is Creature => Option[(String, Boolean, Float)] so we unlift it to obtain Creature => (String, Boolean, Float).

The only thing you have to keep in mind is this unlift call which might not be natural at first sight!

As you can deduce by yourself, the Writes[T] is far easier than the Reads[T] case because when writing, it doesn’t try to validate so there is no error management at all.

Moreover, due to this, you have to keep in mind that operators provided for Writes[T] are not as rich as for Reads[T]. Do you remind keepAnd and andKeep operators? They don’t have any meaning for Writes[T]. When writing A~B, you write A and B but not only A or only B. So and is the only operators provided for Writes[T].

Complexifying the case

Let’s go back to our more complex sample used in end of Part1. Remember that we had imagined that our creature was modelled as following:

1
2
3
4
5
6
7
8
9

case class Creature(
  name: String,
  isDead: Boolean,
  weight: Float,
  email: String, // email format and minLength(5)
  favorites: (String, Int), // the stupid favorites
  friends: List[Creature] = Nil, // yes by default it has no friend
  social: Option[String] = None // by default, it's not social
)

Let’s write corresponding Writes[Creature]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

// IMPORTANT import this to have the required tools in your scope
import play.api.libs.json._
// imports required functional generic structures
import play.api.libs.json.util._
// imports implicit structure for Writes only (always try to import only what you need)
import play.api.libs.json.Writes._

implicit val creatureWrites: Writes[Creature] = (
  (__ \ "name").write[String] and
  (__ \ "isDead").write[Boolean] and
  (__ \ "weight").write[Float] and
  (__ \ "email").write[String] and
  (__ \ "favorites").write(
      (__ \ "string").write[String] and
      (__ \ "number").write[Int]
      tupled
  ) and
  (__ \ "friends").lazyWrite(Writes.traversableWrites[Creature](creatureWrites)) and
  (__ \ "social").write[Option[String]]
)(unlift(Creature.unapply))

val gizmo = Creature("gremlins", false, 1.0F, "gizmo@midnight.com", ("alpha", 85), List(), Some("@gizmo"))
val gizmojs = Json.toJson(gizmo)
gizmojs: play.api.libs.json.JsValue = {"name":"gremlins","isDead":false,"weight":1.0,"email":"gizmo@midnight.com","favorites":{"string":"alpha","number":85},"friends":[],"social":"@gizmo"}

val zombie = Creature("zombie", true, 100.0F, "shaun@dead.com", ("ain", 2), List(gizmo), None)
val zombiejs = Json.toJson(zombie)
zombiejs: play.api.libs.json.JsValue = {"name":"zombie","isDead":true,"weight":100.0,"email":"shaun@dead.com","favorites":{"string":"ain","number":2},"friends":[{"name":"gremlins","isDead":false,"weight":1.0,"email":"gizmo@midnight.com","favorites":{"string":"alpha","number":85},"friends":[],"social":"@gizmo"}],"social":null

You can see that it’s quite straightforward. it’s far easier than Reads[T] as there are no special operator. Here are the few things to explain:

(__ \ "favorites").write(…)

(__ \ "string").write[String] and
(__ \ "number").write[Int]
tupled

• Remember that (__ \ "string").write[String](…) and (__ \ "number").write[Int](…) => Builder[Writes[String ~ Int]]
• What means tupled ? as for Reads[T], it “tuplizes” your Builder: Builder[Writes[A ~ B]].tupled => Writes[(A, B)]

(__ \ "friend").lazyWrite(Writes.traversableWrites[Creature](creatureWrites))

It’s the symmetric code for lazyRead to treat recursive field on Creature class itself:

• Writes.traversableWrites[Creature](creatureWrites) creates a Writes[Traversable[Creature]] passing the Writes[Creature] itself for recursion (please note that a list[Creature]should appear very soon ;))
• (__ \ "friends").lazyWrite[A](r: => Writes[A])) : lazyWrite expects a Writes[A] value passed by name to allow the type recursive construction. This is the only refinement that you must keep in mind in this very special recursive case.

FYI, you may wonder why Writes.traversableWrites[Creature]: Writes[Traversable[Creature]] can replace Writes[List[Creature]]?
This is because Writes[-T] is contravariant meaning: if you can write a Traversable[Creature], you can write a List[Creature] as List inherits Traversable (relation of inheritance is reverted by contravariance).

What about combinators for Format?

Remember in Play2.1, there was a feature called Format[T] extends Reads[T] with Writes[T].
It mixed Reads[T] and Writes[T] together to provide serialization/deserialization at the same place.

Play2.1 provide combinators for Reads[T] and Writes[T]. What about combinators for Format[T] ?

Let’s go back to our very simple sample:

1
2
3
4
5

case class Creature(
  name: String,
  isDead: Boolean,
  weight: Float
)

Here is how you write the Reads[Creature]:

1
2
3
4
5
6
7
8

import play.api.libs.json.util._
import play.api.libs.json.Reads._

val creatureReads = (
  (__ \ "name").read[String] and
  (__ \ "isDead").read[Boolean] and
  (__ \ "weight").read[Float]
)(Creature)

Please remark that I didn’t use implicit so that there is no implicit Reads[Creature] in the context when I’ll define Format[T]

Here is how you write the Writes[Creature]:

1
2
3
4
5
6
7
8

import play.api.libs.json.util._
import play.api.libs.json.Writes._

val creatureWrites = (
  (__ \ "name").write[String] and
  (__ \ "isDead").write[Boolean] and
  (__ \ "weight").write[Float]
)(unlift(Creature.unapply))

How to gather both Reads/Writes to create a Format[Creature]?

1st way = create from existing reads/writes

You can reuse existing Reads[T] and Writes[T] to create a Format[T] as following:

1
2
3
4
5
6
7
8
9
10
11
12

implicit val creatureFormat = Format(creatureReads, creatureWrites)

val gizmojs = Json.obj(
  "name" -> "gremlins",
  "isDead" -> false,
  "weight" -> 1.0F
)

val gizmo = Creature("gremlins", false, 1.0F)

assert(Json.fromJson[Creature](gizmojs).get == gizmo)
assert(Json.toJson(gizmo) == gizmojs)

2nd way = create using combinators

We have Reads and Writes combinators, isn’t it?
Play2.1 also provides Format Combinators due to the magic of functional programming (actually it’s not magic, it’s just pure functional programming;) )

As usual, code 1st:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import play.api.libs.json.util._
import play.api.libs.json.Format._

implicit val creatureFormat = (
  (__ \ "name").format[String] and
  (__ \ "isDead").format[Boolean] and
  (__ \ "weight").format[Float]
)(Creature.apply, unlift(Creature.unapply))

val gizmojs = Json.obj(
  "name" -> "gremlins",
  "isDead" -> false,
  "weight" -> 1.0F
)

val gizmo = Creature("gremlins", false, 1.0F)

assert(Json.fromJson[Creature](gizmojs).get == gizmo)
assert(Json.toJson(gizmo) == gizmojs)

Nothing too strange…

(__ \ "name").format[String]

It creates a format[String] reading/writing at the given JsPath

( )(Creature.apply, unlift(Creature.unapply))

To map to a Scala structure:

• Reads[Creature] requires a function (String, Boolean, Float) => Creature
• Writes[Creature] requires a function Creature => (String, Boolean, Float)

So as Format[Creature] extends Reads[Creature] with Writes[Creature] we provide Creature.apply and unlift(Creature.unapply) and that’s all folks…

More complex case

The previous sample is a bit dumb because the structure is really simple and because reading/writing is symmetric. We have:

1

Json.fromJson[Creature](Json.toJson(creature)) == creature

In this case, you read what you write and vis versa. So you can use the very simple JsPath.format[T] functions which build both Reads[T] and Writes[T] together.

But if we take our usual more complicated case class, how to write the Format[T]?

Remind the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

import play.api.libs.json._
import play.api.libs.json.util._

// The case class
case class Creature(
  name: String,
  isDead: Boolean,
  weight: Float,
  email: String, // email format and minLength(5)
  favorites: (String, Int), // the stupid favorites
  friends: List[Creature] = Nil, // yes by default it has no friend
  social: Option[String] = None // by default, it's not social
)

import play.api.data.validation.ValidationError
import play.api.libs.json.Reads._

// defines a custom reads to be reused
// a reads that verifies your value is not equal to a give value
def notEqualReads[T](v: T)(implicit r: Reads[T]): Reads[T] = Reads.filterNot(ValidationError("validate.error.unexpected.value", v))( _ == v )

def skipReads(implicit r: Reads[String]): Reads[String] = r.map( _.substring(2) )

val creatureReads: Reads[Creature] = (
  (__ \ "name").read[String] and
  (__ \ "isDead").read[Boolean] and
  (__ \ "weight").read[Float] and
  (__ \ "email").read(email keepAnd minLength[String](5)) and
  (__ \ "favorites").read(
      (__ \ "string").read[String]( notEqualReads("ni") andKeep skipReads ) and
      (__ \ "number").read[Int]( max(86) or min(875) )
      tupled
  ) and
  (__ \ "friends").lazyRead( list[Creature](creatureReads) ) and
  (__ \ "social").read(optional[String])
)(Creature)

import play.api.libs.json.Writes._

val creatureWrites: Writes[Creature] = (
  (__ \ "name").write[String] and
  (__ \ "isDead").write[Boolean] and
  (__ \ "weight").write[Float] and
  (__ \ "email").write[String] and
  (__ \ "favorites").write(
      (__ \ "string").write[String] and
      (__ \ "number").write[Int]
      tupled
  ) and
  (__ \ "friends").lazyWrite(Writes.traversableWrites[Creature](creatureWrites)) and
  (__ \ "social").write[Option[String]]
)(unlift(Creature.unapply))

As you can see, creatureReads and creatureWrites are not exactly symmetric and couldn’t be merged in one single Format[Creature] as done previously.

1

Json.fromJson[Creature](Json.toJson(creature)) != creature

Hopefully, as done previously, we can build a Format[T] from a Reads[T] and a Writes[T].

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

implicit val creatureFormat: Format[Creature] = Format(creatureReads, creatureWrites)

// Testing Serialization of Creature to Json
val gizmo = Creature("gremlins", false, 1.0F, "gizmo@midnight.com", ("alpha", 85), List(), Some("@gizmo"))
val zombie = Creature("zombie", true, 100.0F, "shaun@dead.com", ("ain", 2), List(gizmo), None)

val zombiejs = Json.obj(
  "name" -> "zombie",
  "isDead" -> true,
  "weight" -> 100.0,
  "email" -> "shaun@dead.com",
  "favorites" -> Json.obj(
      "string" -> "ain",
      "number" -> 2
  ),
  "friends" -> Json.arr(
      Json.obj(
          "name" -> "gremlins",
          "isDead" -> false,
          "weight" -> 1.0,
          "email" -> "gizmo@midnight.com",
          "favorites" -> Json.obj(
              "string" -> "alpha",
              "number" -> 85
          ),
          "friends" -> Json.arr(),
          "social" -> "@gizmo"
      )
  ),
  "social" -> JsNull
)

assert(Json.toJson(zombie) == zombiejs)

// Testing Deserialization of JSON to Creature (note the dissymetric reading)
val gizmo2 = Creature("gremlins", false, 1.0F, "gizmo@midnight.com", ("pha", 85), List(), Some("@gizmo"))
val zombie2 = Creature("zombie", true, 100.0F, "shaun@dead.com", ("n", 2), List(gizmo2), None)

assert(Json.fromJson[Creature](zombiejs).get == zombie2)

Conclusion & much more…

So here is the end of this 2nd part.

Json combinator is cool!
Now what about transforming Json into Json???
Why would you need this?
Hmmm: why not receive JSON, validate/transform it, send it to a managing Json (Mongo for ex or others), read it again from the DB, modify it a bit and send it out…
Json “coast to coast” without any model class…

Let’s tease a bit:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

val jsonTransform = (
  (__ \ "key1").json.pickBranch and
  (__ \ "key2").json.pickBranch(
    (__ \ "key22").json.put( (__ \ "key22").json.pick.map( js => js \ "key221" ) ) and
    (__ \ "key23").json.put( (__ \ "key23").json.pick.map{ case JsString(s) => JsString(s + "123") } )
    reduce
  )
) reduce

val js = Json.obj(
  "key1" -> "alpha",
  "key2" -> Json.obj("key22" -> Json.obj("key221" -> "beta"), "key23" -> "gamma"),
)

assert(js.validate(jsonTransform), JsSuccess( Json.obj("key1" -> "alpha", "key2" -> Json.obj( "key22" -> "beta", "key23" -> "gamma123" ) ) ) )

Next parts about Json generators/transformers ;)

Have fun…

Addendum: recent API refactoring (modified in the articled)

• Reads[A] provided Reads[B] has been renamed to Reads[A] keepAnd Reads[B]
• Reads[A] andThen Reads[B] has been renamed to Reads[A] andKeep Reads[B]

In incoming Play2.1 version, a huge re-thinking has been done about JSON API provided by Play2.0.x which provides some great features but is clearly just the tip of the iceberg…
Here is a first presentation of those evolutions aimed at unleashing your JSON usage in Play2 and revealing new forms of manipulation of web dataflows from/to external data systems.
A usecase of this is manipulating DB structures directly using Json without any class models for document oriented structures such as MongoDB

BTW Don’t forget the recent release of new MongoDB async/non-blocking driver ReactiveMongo ;-)

Now let’s go in the details ;)

Very Quick Json syntax

Concerning the new Json syntax, I won’t spend time on this, it’s quite explicit and you can try it very easily by yourself.

1
2
3
4
5
6
7
8
9

val js = Json.obj(
  "key1" -> "value1",
  "key2" -> 234,
  "key3" -> Json.obj(
    "key31" -> true,
    "key32" -> Json.arr("alpha", "beta", 234.13),
    "key33" -> Json.obj("key1" -> "value2", "key34" -> "value34")
  )
)

Quick JsPath

You certainly know XMLPath in which you can access a node of an XML AST using a simple syntax based on path.
JSON is already an AST and we can apply the same kind of syntax to it and logically we called it JsPath.

All following examples use JSON defined in previous paragraph.

Building JsPath

1
2
3
4
5
6
7
8
9
10
11

// Simple path
JsPath \ "key1"

// 2-levels path
JsPath \ "key3" \ "key33"

// indexed path
(JsPath \ "key3" \ "key32")(2) // 2nd element in a JsArray

// multiple paths
JsPath \\ "key1"

Alternative syntax

JsPath is quite cool but we found this syntax could be made even clearer to highlight Reads[T] combinators in the code.
That’s why we provide an alias for JsPath: __ (2 underscores).
You can use it or not. This is just a visual facility because with it, you immediately find your JsPath in the code…

You can write:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

// Simple path
__ \ "key1"

// 2-levels path
__ \ "key3" \ "key33"

// indexed path
(__ \ "key3" \ "key32")(2) // 2nd element in a JsArray

// multiple paths
__ \\ "key1"

// a sample on Reads[T] combinators to show the difference with both syntax
// DON'T TRY TO UNDERSTAND THIS CODE RIGHT NOW… It's explained in next paragraphs
val customReads =
  (JsPath \ "key1").read(
    (JsPath \ "key11").read[String] and
    (JsPath \ "key11").read[String] and
    (JsPath \ "key11").read[String]
    tupled
  ) and
  (JsPath \ "key2").read[Float](min(45)) and
  (JsPath \ "key3").read(
    (JsPath \ "key31").read[String] and
    (JsPath \ "key32").read[String] and
    (JsPath \ "key33").read[String]
    tupled
  )
  tupled
  
// with __
val customReads =
  (__ \ "key1").read(
    (__ \ "key11").read[String] and
    (__ \ "key11").read[String] and
    (__ \ "key11").read[String]
    tupled
  ) and
  (__ \ "key2").read[Float](min(45)) and
  (__ \ "key3").read[List[String]] (
    (__ \ "key31").read[String] and
    (__ \ "key32").read[String] and
    (__ \ "key33").read[String]
    tupled
  )
  tupled
  
// You can immediately see the structure of the JSON tree

Accessing value of JsPath

The important function to retrieve the value at a given JsPath in a JsValue is the following:

1
2
3
4

sealed trait PathNode {
  def apply(json: JsValue): List[JsValue]
  …
}

As you can see, this function retrieves a List[JsValue]

You can simply use it like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// build a JsPath
scala> (__ \ "key1")(js)
res12: List[play.api.libs.json.JsValue] = List("value1")  // actually this is JsString("value1")

// 2-levels path
scala> (__ \ "key3" \ "key33")(js)
res13: List[play.api.libs.json.JsValue] = List({"key":"value2","key34":"value34"})

// indexed path
scala> (__ \ "key3" \ "key32")(2)(js)
res14: List[play.api.libs.json.JsValue] = List(234.13)

// multiple paths
scala> (__ \\ "key1")(js)
res17: List[play.api.libs.json.JsValue] = List("value1", "value2")

Reads[T] is now a validator

Reads in Play2.0.x

Do you remember how you had to write a Json Reads[T] in Play2.0.x ?
You had to override the reads function.

1
2
3
4
5
6
7

trait Reads[A] {
  self =>
  /**
   * Convert the JsValue into a A
   */
  def reads(json: JsValue): A
}

Take the following simple case class that you want to map on JSON structure:

1
2
3
4
5

case class Creature(
  name: String,
  isDead: Boolean,
  weight: Float
)

In Play2.0.x, you would write your reader as following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import play.api.libs.json._

implicit val creatureReads = new Reads[Creature] {
  def reads(js: JsValue): Creature = {
    Creature(
      (js \ "name").as[String],
      (js \ "isDead").as[Boolean],
      (js \ "weight").as[Float]
    )
  }
}

scala> val js = Json.obj( "name" -> "gremlins", "isDead" -> false, "weight" -> 1.0F)
scala> val c = js.as[Creature]
c: Creature("gremlins", false, 1.0F)

Easy isn’t it ? So what’s the problem if it’s so easy?

Imagine, you pass the following JSON with a missing field:

1

val js = Json.obj( "name" -> "gremlins", "weight" -> 1.0F)

What happens?

1
2
3
4
5

java.lang.RuntimeException: Boolean expected
  at play.api.libs.json.DefaultReads$BooleanReads$.reads(Reads.scala:98)
  at play.api.libs.json.DefaultReads$BooleanReads$.reads(Reads.scala:95)
  at play.api.libs.json.JsValue$class.as(JsValue.scala:56)
  at play.api.libs.json.JsUndefined.as(JsValue.scala:70)

WHATTTTTTT???? Yes ugly RuntimeException (not even subtyped) but you can work around it using JsValue.asOpt[T] :)

1
2

scala> val c: Option[Creature] = js.asOpt[Creature]
c: None

Cool but you only know that the deserialization Json => Creature failed but not where or on which field(s)?

Reads in Play2.1

We couldn’t keep this imperfect API as is and in Play2.1, the Reads[T] API has changed into :

1
2
3
4
5
6
7

trait Reads[A] {
  self =>
  /**
   * Convert the JsValue into a A
   */
  def reads(json: JsValue): JsResult[A]
}

Yes you have to refactor all your existing custom Reads but you’ll see you’ll get lots of new interesting features…

So you remark immediately JsResult[A] which is a very simple structure looking a bit like an Either applied to our specific problem.

JsResult[A] can be of 2 types:

• JsSuccess[A] when readssucceeds

1
2
3
4
5
6
7

case class JsSuccess[A](
  value: T, // the value retrieved when deserialization JsValue => A worked
  path: JsPath = JsPath() // the root JsPath where this A was read in the JsValue (by default, it's the root of the JsValue)
) extends JsResult[T]

// To create a JsSuccess from a value, simply do:
val success = JsSuccess(Creature("gremlins", false, 1.0))

• JsError[A] when reads fails

Please note the greatest advantage of JsError is that it’s a cumulative error which can store several errors discovered in the Json at different JsPath

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

case class JsError(
  errors: Seq[(JsPath, Seq[ValidationError])]
  // the errors is a sequence of JsPath locating the path 
  // where there was an error in the JsValue and validation 
  // errors for this path
) extends JsResult[Nothing]

// ValidationError is a simple message with arguments (which can be mapped on localized messages)
case class ValidationError(message: String, args: Any*)

// To create a JsError, there are a few helpers and for ex:
val errors1 = JsError( __ \ 'isDead, ValidationError("validate.error.missing", "isDead") )
val errors2 = JsError( __ \ 'name, ValidationError("validate.error.missing", "name") )

// Errors are cumulative which is really interesting
scala> val errors = errors1 ++ errors2
errors: JsError(List((/isDead,List(ValidationError(validate.error.missing,WrappedArray(isDead)))), (/name,List(ValidationError(validate.error.missing,WrappedArray(name))))))

So what’s interesting there is that JsResult[A] is a monadic structure and can be used with classic functions of such structures:

• flatMap[X](f: A => JsResult[X]): JsResult[X]
• fold[X](invalid: Seq[(JsPath, Seq[ValidationError])] => X, valid: A => X)
• map[X](f: A => X): JsResult[X]
• filter(p: A => Boolean)
• collect[B](otherwise:ValidationError)(p:PartialFunction[A,B]): JsResult[B]
• get: A

And some sugar such :

• asOpt
• asEither
• …

Reads[A] has become a validator

As you may understand, using the new Reads[A], you don’t only deserialize a JsValue into another structure but you really validate the JsValue and retrieve all the validation errors.
BTW, in JsValue, a new function called validate has appeared:

1
2
3
4
5
6
7
8
9
10
11
12
13

trait JsValue {
…
  def validate[T](implicit _reads: Reads[T]): JsResult[T] = _reads.reads(this)

  // same behavior but it throws a specific RuntimeException JsResultException now
  def as[T](implicit fjs: Reads[T]): T
  // exactly the same behavior has before
  def asOpt[T](implicit fjs: Reads[T]): Option[T]
…
}

// You can now write if you got the right implicit in your scope
val res: JsResult[Creature] = js.validate[Creature])

Manipulating a JsResult[A]

So when manipulating a JsResult, you don’t access the value directly and it’s preferable to use map/flatmap/fold to modify the value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

val res: JsResult[Creature] = js.validate[Creature]

// managing the success/error and potentially return something
res.fold(
  valid = { c => println( c ); c.name },
  invalid = { e => println( e ); e }
)

// getting the name directly (using the get can throw a NoSuchElementException if it's a JsError)
val name: JsSuccess[String] = res.map( creature => creature.name ).get

// filtering the result
val name: JsSuccess[String] = res.filter( creature => creature.name == "gremlins" ).get

// a classic Play action
def getNameOnly = Action(parse.json) { request =>
  val json = request.body
  json.validate[Creature].fold(
    valid = ( res => Ok(res.name) ),
    invalid = ( e => BadRequest(e.toString) )
  )
}

Rewriting the Reads[T] with JsResult[A]

The Reads[A] API returning a JsResult, you can’t write your Reads[A] as before as you must return a JsResult gathering all found errors.
You could imagine simply compose Reads[T] with flatMap :

1
2
3
4
5
6
7
8
9
10
11
12

// DO NOT USE, WRONG CODE
implicit val creatureReads = new Reads[Creature] {
  def reads(js: JsValue): JsResult[Creature] = {
    (js \ "name").validate[String].flatMap{ name =>
      (js \ "isDead").validate[Boolean].flatMap { isDead =>
      (js \ "weight").validate[Float].map { weight =>
        Creature(name, isDead, weight)
      }
    }
  }
  }
}

Remember the main purpose of JsResult is to gather all found errors while validating the JsValue.

JsResult.flatMap is pure monadic function (if you don’t know what it is, don’t care about it, you can understand without it) implying that the function that you pass to flatMap() is called only if the result is a JsSuccess else it just returns the JsError.
This means the previous code won’t aggregate all errors found during validation and will stop at first error which is exactly what we don’t want.

Actually, Monad pattern is not good in our case because we are not just composing Reads but we expect combining them following the schema:

1
2
3

Reads[String] AND Reads[Boolean] AND Reads[Float]
  => Reads[(String, Boolean, Float)]
     => Reads[Creature]

So we need something else to be able to combine our Reads and this is the greatest new feature that Play2.1 brings for JSON :
THE READS combinators with JsPath

If you want more theoretical aspects about the way it was implemented based on generic functional structures adapted to our needs, you can read this post “Applicatives are too restrictive, breaking Applicatives and introducing Functional Builders” written by @sadache

Rewriting the Reads[T] with combinators

Go directly to the example as practice is often the best :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// IMPORTANT import this to have the required tools in your scope
import play.api.libs.json.util._

implicit val creatureReads = (
  (__ \ "name").read[String] and
  (__ \ "isDead").read[Boolean] and
  (__ \ "weight").read[Float]
)(Creature.apply _)

// or in a simpler way as case class has a companion object with an apply function
implicit val creatureReads = (
  (__ \ "name").read[String] and
  (__ \ "isDead").read[Boolean] and
  (__ \ "weight").read[Float]
)(Creature)

// or using the operators inspired by Scala parser combinators for those who know them
implicit val creatureReads = (
  (__ \ "name").read[String] ~
  (__ \ "isDead").read[Boolean] ~
  (__ \ "weight").read[Float]
)(Creature)

So there is nothing quite complicated, isn’t it?

(__ \ "name") is the JsPath where you gonna apply read[String]

and is just an operator meaning Reads[A] and Reads[B] => Builder[Reads[A ~ B]]

• A ~ B just means Combine A and B but it doesn’t suppose the way it is combined (can be a tuple, an object, whatever…)
• Builder is not a real type but I introduce it just to tell that the operator and doesn’t create directly a Reads[A ~ B] but an intermediate structure that is able to build a Reads[A ~ B] or to combine with another Reads[C]

(…)(Creature) builds a Reads[Creature]

• (__ \ "name").read[String] and (__ \ "isDead").read[Boolean] and (__ \ "weight").read[Float] builds a Builder[Reads[String ~ Boolean ~ Float])] but you want a Reads[Creature].
• So you apply the Builder[Reads[String ~ Boolean ~ String])] to the function Creature.apply = (String, Boolean, Float) => Creature constructor to finally obtain a Reads[Creature]

Try it:

1
2
3

scala> val js = Json.obj( "name" -> "gremlins", "isDead" -> false, "weight" -> 1.0F)
scala> js.validate[Creature]
res1: play.api.libs.json.JsResult[Creature] = JsSuccess(Creature(gremlins,false,1.0),) // nothing after last comma because the JsPath is ROOT by default

Now what happens if you have an error now?

1
2
3

scala> val js = Json.obj( "name" -> "gremlins", "weight" -> 1.0F)
scala> js.validate[Creature]
res2: play.api.libs.json.JsResult[Creature] = JsError(List((/isDead,List(ValidationError(validate.error.missing-path,WrappedArray())))))

Explicit, isn’t it?

Complexifying the case

Ok, I see what you think : what about more complex cases where you have several constraints on a field and embedded Json in Json and recursive classes and whatever…

Let’s imagine our creature:

• is a relatively modern creature having an email and hating email addresses having less than 5 characters for a reason only known by the creature itself.
• may have 2 favorites data:
  • 1 String (called “string” in JSON) which shall not be “ni” (because it loves Monty Python too much to accept this) and then to skip the first 2 chars
  • 1 Int (called “number” in JSON) which can be less than 86 or more than 875 (don’t ask why, this is creature with a different logic than ours)
• may have friend creatures
• may have an optional social account because many creatures are not very social so this is quite mandatory

Now the class looks like:

1
2
3
4
5
6
7
8
9

case class Creature(
  name: String,
  isDead: Boolean,
  weight: Float,
  email: String, // email format and minLength(5)
  favorites: (String, Int), // the stupid favorites
  friends: List[Creature] = Nil, // yes by default it has no friend
  social: Option[String] = None // by default, it's not social
)

Play2.1 provide lots of generic Reads helpers:

• JsPath.read[A](implicit reads:Reads[A]) can be passed a custom Reads[A] which is applied to the JSON content at this JsPath. So with this property, you can compose hierarchically Reads[T] which corresponds to JSON tree structure.
• JsPath.readOpt allows Reads[Option[T]]
• Reads.email which validates the String has email format
• Reads.minLength(nb) validates the minimum length of a String
• Reads[A] or Reads[A] => Reads[A] operator is a classic OR logic operator
• Reads[A] keepAnd Reads[B] => Reads[A] is an operator that tries Reads[A] and Reads[B] but only keeps the result of Reads[A] (For those who know Scala parser combinators keepAnd == <~ )
• Reads[A] andKeep Reads[B] => Reads[B] is an operator that tries Reads[A] and Reads[B] but only keeps the result of Reads[B] (For those who know Scala parser combinators andKeep == ~> )

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

// import just Reads helpers in scope
import play.api.libs.json._
import play.api.libs.json.util._
import play.api.libs.json.Reads._
import play.api.data.validation.ValidationError

// defines a custom reads to be reused
// a reads that verifies your value is not equal to a give value
def notEqualReads[T](v: T)(implicit r: Reads[T]): Reads[T] = Reads.filterNot(ValidationError("validate.error.unexpected.value", v))( _ == v )

def skipReads(implicit r: Reads[String]): Reads[String] = r.map( _.substring(2) )

implicit val creatureReads: Reads[Creature] = (
  (__ \ "name").read[String] and
  (__ \ "isDead").read[Boolean] and
  (__ \ "weight").read[Float] and
  (__ \ "email").read(email keepAnd minLength[String](5)) and
  (__ \ "favorites").read(
      (__ \ "string").read[String]( notEqualReads("ni") andKeep skipReads ) and
      (__ \ "number").read[Int]( max(86) or min(875) )
      tupled
  ) and
  (__ \ "friends").lazyRead( list[Creature](creatureReads) ) and
  (__ \ "social").readOpt[String]
)(Creature)

Many things above can be understood very logically but let’s explain a bit:

(__ \ "email").read(email keepAnd minLength[String](5))

• As explained previously, (__ \ "email").read(…) applies the Reads[T] passed in parameter to function read at the given JsPath (__ \ "email")
• email keepAnd minLength[String](5) => Reads[String] is a Js validator that verifies JsValue:
  1. is a String : email: Reads[String] so no need to specify type here
  2. has email format
  3. has min length of 5 (we precise the type here because minLength is a generic Reads[T])
• Why not email and minLength[String](5)? because, as explained previously, it would generate a Builder[Reads[(String, String)]] whereas you expect a Reads[String]. The keepAnd operator (aka <~) does what we expect: it validates both sides but if succeeded, it keeps only the result on left side.

notEqualReads("ni") andKeep skipReads

• No need to write notEqualReads[String]("ni") because String type is inferred from (__ \ "knight").read[String] (the power of Scala typing engine)
• skipReads is a customReads that skips the first 2 chars
• andKeep operator (aka ~>) is simple to undestand : it validates the left and right side and if both succeeds, only keeps the result on right side. In our case, only the result of skipReads is kept and not the result of notEqualReads.

max(86) or min(875)

• Nothing to explain there, isn’t it ? or is the classic ORlogic operator, nothing else

(__ \ "favorites").read(…)

(__ \ "string").read[String]( notEqualReads("ni") andKeep notEqualReads("swallow") ) and
(__ \ "number").read[Int]( max(86) or min(875) )
tupled

• Remember that (__ \ "string").read[String](…) and (__ \ "number").read[Int](…) => Builder[Reads[(String, Int)]]
• What means tupled ? Builder[Reads[(String, Int)]] can be used with a case class apply function to build the Reads[Creature] for ex. But it provides also tupled which is quite easy to understand : it “tuplizes” your Builder: Builder[Reads[(A, B)]].tupled => Reads[(A, B)]
• Finally (__ \ "favorites").read(Reads[(String, Int)] will validate a (String, Int) which is the expected type for field favorites

(__ \ "friend").lazyRead( list[Creature](creatureReads) )

This is the most complicated line in this code. But you can understand why: the friend field is recursive on the Creature class itself so it requires a special treatment.

• list[Creature](…) creates a Reads[List[Creature]]
• list[Creature](creatureReads) passes explicitly creatureReads as an argument because it’s recursive and Scala requires that to resolve it. Nothing too complicated…
• (__ \ "friend").lazyRead[A](r: => Reads[A])) : lazyRead expects a Reads[A] value passed by name to allow the type recursive construction. This is the only refinement that you must keep in mind in this very special recursive case.

(__ \ "social").readOpt[String]

Nothing quite complicated to understand: we need to read an option and readOpt helps in doing this.

Now we can use this Reads[Creature]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

val gizmojs = Json.obj(
  "name" -> "gremlins",
  "isDead" -> false,
  "weight" -> 1.0F,
  "email" -> "gizmo@midnight.com",
  "favorites" -> Json.obj("string" -> "alpha", "number" -> 85),
  "friends" -> Json.arr(),
  "social" -> "@gizmo"
)

scala> val gizmo = gizmojs.validate[Creature]
gizmo: play.api.libs.json.JsResult[Creature] = JsSuccess(Creature(gremlins,false,1.0,gizmo@midnight.com,(pha,85),List(),Some(@gizmo)),)

val shaunjs = Json.obj(
  "name" -> "zombie",
  "isDead" -> true,
  "weight" -> 100.0F,
  "email" -> "shaun@dead.com",
  "favorites" -> Json.obj("string" -> "brain", "number" -> 2),
  "friends" -> Json.arr( gizmojs))

scala> val shaun = shaunjs.validate[Creature]
shaun: play.api.libs.json.JsResult[Creature] = JsSuccess(Creature(zombie,true,100.0,shaun@dead.com,(ain,2),List(Creature(gremlins,false,1.0,gizmo@midnight.com,(alpha,85),List(),Some(@gizmo))),None),)

val errorjs = Json.obj(
  "name" -> "gremlins",
  "isDead" -> false,
  "weight" -> 1.0F,
  "email" -> "rrhh",
  "favorites" -> Json.obj("string" -> "ni", "number" -> 500),
  "friends" -> Json.arr()
)

scala> errorjs.validate[Creature]
res0: play.api.libs.json.JsResult[Creature] = JsError(List((/favorites/string,List(ValidationError(validate.error.unexpected.value,WrappedArray(ni)))), (/email,List(ValidationError(validate.error.email,WrappedArray()), ValidationError(validate.error.minlength,WrappedArray(5)))), (/favorites/number,List(ValidationError(validate.error.max,WrappedArray(86)), ValidationError(validate.error.min,WrappedArray(875))))))