Play AutoSource & Datomisca: Proofing the Concept on Datomic, a Schema DB

| Comments

Now you should use play-autosource 2.0 correcting a few issues & introducing ActionBuilder from play2.2


The code for all autosources & sample apps can be found on Github here

Brand New Autosources

Play AutoSource now have 2 more implementations :

One month ago, I’ve demo’ed the concept of Autosource for Play2/Scala with ReactiveMongo in this article. ReactiveMongo was the perfect target for this idea because it accepts Json structures almost natively for both documents manipulation and queries.

But how does the concept behave when applied on a DB for which data are constrained by a schema and for which queries aren’t Json.


Using Datomisca-Autosource in your Play project

Add following lines to your project/Build.scala

1
2
3
4
5
6
7
8
9
10
11
12
13
14

val mandubianRepo = Seq(
  "Mandubian repository snapshots" at "https://github.com/mandubian/mandubian-mvn/raw/master/snapshots/",
  "Mandubian repository releases" at "https://github.com/mandubian/mandubian-mvn/raw/master/releases/"
)

val appDependencies = Seq()

val main = play.Project(appName, appVersion, appDependencies).settings(
  resolvers ++= mandubianRepo,
  libraryDependencies ++= Seq(
    "play-autosource"   %% "datomisca"       % "1.0",
    ...
  )
)

Create your Model + Schema

With ReactiveMongo Autosource, you could create a pure blob Autosource using JsObject without any supplementary information. But with Datomic, it’s not possible because Datomic forces to use a schema for your data.

We could create a schema and manipulate JsObject directly with Datomic and some Json validators. But I’m going to focus on the static models because this is the way people traditionally interact with a Schema-constrained DB.

Let’s create our model and schema.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// The Model (with characters pointing on Datomic named entities)
case class Person(name: String, age: Long, characters: Set[DRef])

// The Schema written with Datomisca
object Person {
  // Namespaces
  val person = new Namespace("person") {
    val characters = Namespace("person.characters")
  }

  // Attributes
  val name       = Attribute(person / "name",       SchemaType.string, Cardinality.one) .withDoc("Person's name")
  val age        = Attribute(person / "age",        SchemaType.long,   Cardinality.one) .withDoc("Person's age")
  val characters = Attribute(person / "characters", SchemaType.ref,    Cardinality.many).withDoc("Person's characterS")

  // Characters named entities
  val violent = AddIdent(person.characters / "violent")
  val weak    = AddIdent(person.characters / "weak")
  val clever  = AddIdent(person.characters / "clever")
  val dumb    = AddIdent(person.characters / "dumb")
  val stupid  = AddIdent(person.characters / "stupid")

  // Schema
  val schema = Seq(
    name, age, characters,
    violent, weak, clever, dumb, stupid
  )

Create Datomisca Autosource

Now that we have our schema, let’s write the autosource.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import datomisca._
import Datomic._

import play.autosource.datomisca._

import play.modules.datomisca._
import Implicits._

import scala.concurrent.ExecutionContext.Implicits.global
import play.api.Play.current

import models._
import Person._

object Persons extends DatomiscaAutoSourceController[Person] {
  // gets the Datomic URI from application.conf
  val uri = DatomicPlugin.uri("mem")

  // ugly DB initialization ONLY for test purpose
  Datomic.createDatabase(uri)

  // Datomic connection is required
  override implicit val conn = Datomic.connect(uri)
  // Datomic partition in which you store your entities
  override val partition = Partition.USER

  // more than ugly schema provisioning, ONLY for test purpose
  Await.result(
    Datomic.transact(Person.schema),
    Duration("10 seconds")
  )

}

Implementing Json <-> Person <-> Datomic transformers

If you compile previous code, you should have following error:

1

could not find implicit value for parameter datomicReader: datomisca.EntityReader[models.Person]

Actually, Datomisca Autosource requires 4 elements to work:

  • Json.Format[Person] to convert Person instances from/to Json (network interface)
  • EntityReader[Person] to convert Person instances from Datomic entities (Datomic interface)
  • PartialAddEntityWriter[Person] to convert Person instances to Datomic entities (Datomic interface)
  • Reads[PartialAddEntity] to convert Json to PartialAddEntity which is actually a simple map of fields/values to partially update an existing entity (one single field for ex).

It might seem more complicated than in ReactiveMongo but there is nothing different. The autosource converts Person from/to Json and then converts Person from/to Datomic structure ie PartialAddEntity. In ReactiveMongo, the only difference is that it understands Json so well that static model becomes unnecessary sometimes ;)…

Let’s define those elements in Person companion object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

object Person {
...
  // Classic Play2 Json Reads/Writes
  implicit val personFormat = Json.format[Person]

  // Partial entity update : Json to PartialAddEntity Reads
  implicit val partialUpdate: Reads[PartialAddEntity] = (
    ((__ \ 'name).read(readAttr[String](Person.name)) orElse Reads.pure(PartialAddEntity(Map.empty))) and
    ((__ \ 'age) .read(readAttr[Long](Person.age)) orElse Reads.pure(PartialAddEntity(Map.empty)))  and
    // need to specify type because a ref/many can be a list of dref or entities so need to tell it explicitly
    (__ \ 'characters).read( readAttr[Set[DRef]](Person.characters) )
    reduce
  )

  // Entity Reads (looks like Json combinators but it's Datomisca combinators)
  implicit val entity2Person: EntityReader[Person] = (
    name      .read[String]   and
    age       .read[Long]     and
    characters.read[Set[DRef]]
  )(Person.apply _)

  // Entity Writes (looks like Json combinators but it's Datomisca combinators)
  implicit val person2Entity: PartialAddEntityWriter[Person] = (
    name      .write[String]   and
    age       .write[Long]     and
    characters.write[Set[DRef]]
  )(DatomicMapping.unlift(Person.unapply))

...
}

Now we have everything to work except a few configurations.

Add AutoSource routes at beginning conf/routes

1

->      /person                     controllers.Persons

Create conf/play.plugins to initialize Datomisca Plugin

1

400:play.modules.datomisca.DatomicPlugin

Append to conf/application.conf to initialize MongoDB connection

1

datomisca.uri.mem="datomic:mem://mem"

Insert your first 2 persons with Curl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

>curl -X POST -d '{ "name":"bob", "age":25, "characters": ["person.characters/stupid", "person.characters/violent"] }' --header "Content-Type:application/json" http://localhost:9000/persons --include

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 21

{"id":17592186045423} -> oh a Datomic ID

>curl -X POST -d '{ "name":"john", "age":43, "characters": ["person.characters/clever", "person.characters/weak"] }' --header "Content-Type:application/json" http://localhost:9000/persons --include

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 21

{"id":17592186045425}

Querying is the biggest difference in Datomic

In Datomic, you can’t do a getAll without providing a Datomic Query.

But what is a Datomic query? It’s inspired by Datalog which uses predicates to express the constraints on the searched entities. You can combine predicates together.

With Datomisca Autosource, you can directly send datalog queries in the query parameter q for GET or in body for POST with one restriction: your query can’t accept input parameters and must return only the entity ID. For ex:

[ :find ?e :where [ ?e :person/name "john"] ] --> OK

[ :find ?e ?name :where [ ?e :person/name ?name] ] --> KO

Let’s use it by finding all persons.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

>curl -X POST --header "Content-Type:text/plain" -d '[:find ?e :where [?e :person/name]]' 'http://localhost:9000/persons/find' --include

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 231

[
    {
        "name": "bob",
        "age": 25,
        "characters": [
            ":person.characters/violent",
            ":person.characters/stupid"
        ],
        "id": 17592186045423
    },
    {
        "name": "john",
        "age": 43,
        "characters": [
            ":person.characters/clever",
            ":person.characters/weak"
        ],
        "id": 17592186045425
    }
]

Please note the use of POST here instead of GET because Curl doesn’t like [] in URL even using -g option

Now you can use all other routes provided by Autosource

Autosource Standard Routes

Get / Find / Stream

  • GET /persons?… -> Find by query
  • GET /persons/ID -> Find by ID
  • GET /persons/stream -> Find by query & stream result by page

Insert / Batch / Find

  • POST /persons + BODY -> Insert
  • POST /persons/find + BODY -> find by query (when query is too complex to be in a GET)
  • POST /persons/batch + BODY -> batch insert (multiple)

Update / batch

  • PUT /persons/ID + BODY -> Update by ID
  • PUT /persons/ID/partial + BODY -> Update partially by ID
  • PUT /persons/batch -> batch update (multiple)

Delete / Batch

  • DELETE /persons/ID -> delete by ID
  • DELETE /persons/batch + BODY -> batch delete (multiple)


Conclusion

Play-Autosource’s ambition was to be DB agnostic (as much as possible) and showing that the concept can be applied to schemaless DB (ReactiveMongo & CouchDB) and schema DB (Datomic) is a good sign it can work. Naturally, there are a few more elements to provide for Datomic than in ReactiveMongo but it’s useful anyway.

Thank to @TrevorReznik for his contribution of CouchBase Autosource.

I hope to see soon one for Slick and a few more ;)

Have Autofun!