Addendum: recent API refactoring (modified in the articled)
Reads[A] provided Reads[B]
has been renamed toReads[A] keepAnd Reads[B]
Reads[A] andThen Reads[B]
has been renamed toReads[A] andKeep Reads[B]
In incoming Play2.1
version, a huge re-thinking has been done about JSON API
provided by Play2.0.x
which provides some great features but is clearly just the tip of the iceberg…
Here is a first presentation of those evolutions aimed at unleashing your JSON usage in Play2 and revealing new forms of manipulation of web dataflows from/to external data systems.
A usecase of this is manipulating DB structures directly using Json without any class models for document oriented structures such as MongoDB
BTW Don’t forget the recent release of new MongoDB async/non-blocking driver ReactiveMongo ;-)
Summary of main new features added in Play2.1
- Simplified Json Objects/Arrays syntax
1 2 3 4 5
Json.obj( "key" -> "value", "key2" -> 123, "key3" -> Json.arr("alpha", 143.55, false) )
JsPath
for manipulating JSON inXmlPath
-style1
(JsPath \ "key" \ "key2" \\ "key3")(3)
Reads[T]
/Writes[T]
/Format[T]
combinators based on JsPath and simple logic operators1 2 3 4 5
val customReads: Reads[(String, Float, List[String])] = (JsPath \ "key1").read[String](email keepAnd minLength(5)) and (JsPath \ "key2").read[Float](min(45)) and (JsPath \ "key3").read[List[String]] tupled
Reads[T]
API now validates JSON by returning a monadicJsResult[T]
being aJsSuccess[T]
or aJsError
aggregating all validation errors1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
val js = Json.obj( "key1" -> "alpha", "key2" -> 123.345F, "key3" -> Json.arr("alpha", "beta") ) scala> customReads.reads(js) res5: JsSuccess(("alpha", 123.345F, List("alpha", "beta"))) customReads.reads(js).fold( valid = { res => val (s, f, l): (String, Float, List[String]) = res ... }, invalid = { errors => ... } )
Now let’s go in the details ;)
Very Quick Json syntax
Concerning the new Json syntax, I won’t spend time on this, it’s quite explicit and you can try it very easily by yourself.
1 2 3 4 5 6 7 8 9 |
|
Quick JsPath
You certainly know XMLPath
in which you can access a node of an XML AST using a simple syntax based on path.
JSON is already an AST and we can apply the same kind of syntax to it and logically we called it JsPath
.
All following examples use JSON defined in previous paragraph.
Building JsPath
1 2 3 4 5 6 7 8 9 10 11 |
|
Alternative syntax
JsPath
is quite cool but we found this syntax could be made even clearer to highlight Reads[T]
combinators in the code.
That’s why we provide an alias for JsPath
: __
(2 underscores).
You can use it or not. This is just a visual facility because with it, you immediately find your JsPath in the code…
You can write:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
|
Accessing value of JsPath
The important function to retrieve the value at a given JsPath in a JsValue is the following:
1 2 3 4 |
|
As you can see, this function retrieves a List[JsValue]
You can simply use it like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
Reads[T] is now a validator
Reads in Play2.0.x
Do you remember how you had to write a Json Reads[T]
in Play2.0.x
?
You had to override the reads
function.
1 2 3 4 5 6 7 |
|
Take the following simple case class that you want to map on JSON structure:
1 2 3 4 5 |
|
In Play2.0.x
, you would write your reader as following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
Easy isn’t it ? So what’s the problem if it’s so easy?
Imagine, you pass the following JSON with a missing field:
1
|
|
What happens?
1 2 3 4 5 |
|
WHATTTTTTT????
Yes ugly RuntimeException (not even subtyped) but you can work around it using JsValue.asOpt[T]
:)
1 2 |
|
Cool but you only know that the deserialization Json => Creature
failed but not where or on which field(s)?
Reads in Play2.1
We couldn’t keep this imperfect API as is and in Play2.1
, the Reads[T]
API has changed into :
1 2 3 4 5 6 7 |
|
Yes you have to refactor all your existing custom Reads but you’ll see you’ll get lots of new interesting features…
So you remark immediately JsResult[A]
which is a very simple structure looking a bit like an Either applied to our specific problem.
JsResult[A]
can be of 2 types:
JsSuccess[A]
whenreads
succeeds
1 2 3 4 5 6 7 |
|
JsError[A]
whenreads
fails
Please note the greatest advantage of JsError is that it’s a cumulative error which can store several errors discovered in the Json at different JsPath
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
So what’s interesting there is that JsResult[A]
is a monadic structure and can be used with classic functions of such structures:
flatMap[X](f: A => JsResult[X]): JsResult[X]
fold[X](invalid: Seq[(JsPath, Seq[ValidationError])] => X, valid: A => X)
map[X](f: A => X): JsResult[X]
filter(p: A => Boolean)
collect[B](otherwise:ValidationError)(p:PartialFunction[A,B]): JsResult[B]
get: A
And some sugar such :
asOpt
asEither
- …
Reads[A] has become a validator
As you may understand, using the new Reads[A]
, you don’t only deserialize a JsValue into another structure but you really validate the JsValue and retrieve all the validation errors.
BTW, in JsValue
, a new function called validate
has appeared:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
Manipulating a JsResult[A]
So when manipulating a JsResult
, you don’t access the value directly and it’s preferable to use map/flatmap/fold
to modify the value.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
Rewriting the Reads[T] with JsResult[A]
The Reads[A]
API returning a JsResult, you can’t write your Reads[A]
as before as you must return a JsResult gathering all found errors.
You could imagine simply compose Reads[T] with flatMap :
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Remember the main purpose of
JsResult
is to gather all found errors while validating the JsValue.
JsResult.flatMap
is pure monadic function (if you don’t know what it is, don’t care about it, you can understand without it) implying that the function that you pass to flatMap()
is called only if the result is a JsSuccess
else it just returns the JsError
.
This means the previous code won’t aggregate all errors found during validation and will stop at first error which is exactly what we don’t want.
Actually, Monad pattern is not good in our case because we are not just composing Reads but we expect combining them following the schema:
1 2 3 |
|
So we need something else to be able to combine our Reads and this is the greatest new feature that
Play2.1
brings for JSON :
THE READS combinators with JsPath
If you want more theoretical aspects about the way it was implemented based on generic functional structures adapted to our needs, you can read this post “Applicatives are too restrictive, breaking Applicatives and introducing Functional Builders” written by @sadache
Rewriting the Reads[T] with combinators
Go directly to the example as practice is often the best :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
So there is nothing quite complicated, isn’t it?
(__ \ "name")
is the JsPath
where you gonna apply read[String]
and
is just an operator meaning Reads[A] and Reads[B] => Builder[Reads[A ~ B]]
A ~ B
just meansCombine A and B
but it doesn’t suppose the way it is combined (can be a tuple, an object, whatever…)Builder
is not a real type but I introduce it just to tell that the operatorand
doesn’t create directly aReads[A ~ B]
but an intermediate structure that is able to build aReads[A ~ B]
or to combine with anotherReads[C]
(…)(Creature)
builds a Reads[Creature]
(__ \ "name").read[String] and (__ \ "isDead").read[Boolean] and (__ \ "weight").read[Float]
builds aBuilder[Reads[String ~ Boolean ~ Float])]
but you want aReads[Creature]
.- So you apply the
Builder[Reads[String ~ Boolean ~ String])]
to the functionCreature.apply = (String, Boolean, Float) => Creature
constructor to finally obtain aReads[Creature]
Try it:
1 2 3 |
|
Now what happens if you have an error now?
1 2 3 |
|
Explicit, isn’t it?
Complexifying the case
Ok, I see what you think : what about more complex cases where you have several constraints on a field and embedded Json in Json and recursive classes and whatever…
Let’s imagine our creature:
- is a relatively modern creature having an email and hating email addresses having less than 5 characters for a reason only known by the creature itself.
- may have 2 favorites data:
- 1 String (called “string” in JSON) which shall not be “ni” (because it loves Monty Python too much to accept this) and then to skip the first 2 chars
- 1 Int (called “number” in JSON) which can be less than 86 or more than 875 (don’t ask why, this is creature with a different logic than ours)
- may have friend creatures
- may have an optional social account because many creatures are not very social so this is quite mandatory
Now the class looks like:
1 2 3 4 5 6 7 8 9 |
|
Play2.1
provide lots of generic Reads helpers:
JsPath.read[A](implicit reads:Reads[A])
can be passed a customReads[A]
which is applied to the JSON content at this JsPath. So with this property, you can compose hierarchicallyReads[T]
which corresponds to JSON tree structure.JsPath.readOpt
allowsReads[Option[T]]
Reads.email
which validates the String has email formatReads.minLength(nb)
validates the minimum length of a StringReads[A] or Reads[A] => Reads[A]
operator is a classicOR
logic operatorReads[A] keepAnd Reads[B] => Reads[A]
is an operator that triesReads[A]
andReads[B]
but only keeps the result ofReads[A]
(For those who know Scala parser combinatorskeepAnd == <~
)Reads[A] andKeep Reads[B] => Reads[B]
is an operator that triesReads[A]
andReads[B]
but only keeps the result ofReads[B]
(For those who know Scala parser combinatorsandKeep == ~>
)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
Many things above can be understood very logically but let’s explain a bit:
(__ \ "email").read(email keepAnd minLength[String](5))
- As explained previously,
(__ \ "email").read(…)
applies theReads[T]
passed in parameter to functionread
at the given JsPath(__ \ "email")
email keepAnd minLength[String](5) => Reads[String]
is a Js validator that verifies JsValue:- is a String :
email: Reads[String]
so no need to specify type here - has email format
- has min length of 5 (we precise the type here because minLength is a generic
Reads[T]
)
- is a String :
- Why not
email and minLength[String](5)
? because, as explained previously, it would generate aBuilder[Reads[(String, String)]]
whereas you expect aReads[String]
. ThekeepAnd
operator (aka<~
) does what we expect: it validates both sides but if succeeded, it keeps only the result on left side.
notEqualReads("ni") andKeep skipReads
- No need to write
notEqualReads[String]("ni")
becauseString
type is inferred from(__ \ "knight").read[String]
(the power of Scala typing engine) skipReads
is a customReads that skips the first 2 charsandKeep
operator (aka~>
) is simple to undestand : it validates the left and right side and if both succeeds, only keeps the result on right side. In our case, only the result ofskipReads
is kept and not the result ofnotEqualReads
.
max(86) or min(875)
- Nothing to explain there, isn’t it ?
or
is the classicOR
logic operator, nothing else
(__ \ "favorites").read(…)
(__ \ "string").read[String]( notEqualReads("ni") andKeep notEqualReads("swallow") ) and
(__ \ "number").read[Int]( max(86) or min(875) )
tupled
- Remember that
(__ \ "string").read[String](…) and (__ \ "number").read[Int](…) => Builder[Reads[(String, Int)]]
- What means
tupled
?Builder[Reads[(String, Int)]]
can be used with a case classapply
function to build theReads[Creature]
for ex. But it provides alsotupled
which is quite easy to understand : it “tuplizes” your Builder:Builder[Reads[(A, B)]].tupled => Reads[(A, B)]
- Finally
(__ \ "favorites").read(Reads[(String, Int)]
will validate a(String, Int)
which is the expected type for fieldfavorites
(__ \ "friend").lazyRead( list[Creature](creatureReads) )
This is the most complicated line in this code. But you can understand why: the friend
field is recursive on the Creature
class itself so it requires a special treatment.
list[Creature](…)
creates aReads[List[Creature]]
list[Creature](creatureReads)
passes explicitlycreatureReads
as an argument because it’s recursive and Scala requires that to resolve it. Nothing too complicated…(__ \ "friend").lazyRead[A](r: => Reads[A]))
:lazyRead
expects aReads[A]
value passed by name to allow the type recursive construction. This is the only refinement that you must keep in mind in this very special recursive case.
(__ \ "social").readOpt[String]
Nothing quite complicated to understand: we need to read an option and readOpt
helps in doing this.
Now we can use this Reads[Creature]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
|
Conclusion & much more…
So here is the end of this 1st part not to make a too long article. But there are so many other features to explain and demonstrate and there will other parts very soon!!!
Reads[T]
combinators are cool but you know what?Writes[T]
&Format[T]
can be combined too!
Let’s tease a bit:
1 2 3 4 5 |
|
Next parts about
Writes[T]
combinators and another intriguing feature that I’ll nameJson Generators
;)
Have fun…