28Jul2012

Planning a Scala based WCS framework

With this post I am starting to collect ideas about developing an Open Source Scala based development framework for WCS, tentatively named ScalaWCS. I am making public my intention in order to collect suggestions, critiques and, why not?, collaborators.

The rationale behind this idea is my desire (and I do not think I am alone) to  modernise a bit WCS development .

I do all my  work outside Fatwire/WCS in Scala. I love Scala a lot  and I want to use it for Fatwire coding as well. Here I  collected my ideas so far for this framework. I would really like to hear from you what you think.

Warning: the rest of this article will be hard to read for anyone who does not know Scala. Apologies for this. Maybe the number of developers is really small but Scala is really an exciting and powerful language and it is worth every minute you spent on it.

Move Away From JSP

The main goal of this framework is moving away from JSP.  I would like to to do all the coding in Scala instead.

JSP are really meant to be used only as a presentation layer, but in Fatwire you end up to do much more than this in JSP, and it is not very nice.

Of course JSP are actually part of Fatwire, there is no direct replacement for them you cannot get rid of them. But you can use them as a stub to call your Scala code.

The initial, rough idea is that you will keep Templates and CSElement, but the body of a template will be simply something like this

<% wcs.Dispatcher.dispatch(ics); %>

(automatically generated configuring the default template ). You are not expected to code anything else.

So, what will replace JSP?

Scala based templates

The real code will be written in Scala and will be something like this:

package article

object Layout extends Template {
    def apply(ics: ICS, c: String, cid: Long) =
      <h1>{ "title"! }</h1>
      <p>{ "body"! }</p>
 }

Instead of writing JSPs you use for rendering a method written in Scala.  For template coding,  I am leveraging the embedded xml literals available in Scala.

Furthermore, I am going to leverage the powerful Scala syntax, that will allow me to write "title"!  instead of ics.GetVar("title").

I understand there are some problems with this approach, mostly on  development and deployment. More on this later.

Nice wrapper on Fatwire API

Using Scala, I plan to wrap all the existing Fatwire tags with Scala calls. Something like

<asset:load name="thePage" type="Page" objectid="<%=ics.GetVar("cid")&>"/>
 <asset:get name="thePage" field="id" output="id"/>

will be available as a Scala call (using named parameters) like:

asset.load(name="thePage", type="Page", objectid= ("cid"!))
asset.get(name="thePage", field="id", output= "id")

This is however only the first step. The API will leverage some common elements in the API to be able to do

val thePage = asset.load("Page", "cid"!)
val id = thePage("id")

Each element will be functional and will return objects with appropriate methods allowing to extract fields using the apply syntax. A lot of work will be done on making  the API as nice as possible

It would be nice also to be able to keep the tag based syntax in the XML literals.

This should be possible, however the xml literals are not compiled in Scala, while they are in JSP.

A simpler implementation is possible just interpreting them when generating the  output. A more advanced implementation could be generating actual Scala code. This is more difficult to implement since it needs to leverage either a compiler plugin or a Scala macro.  Advantages and disadvantages will have to be carefully considered.

Real Unit testing

Once we moved away from JSP, the biggest advantage of using this approach is the possibility to perform real unit testing. Any unit test framework can be used, either ScalaTest or Specs2.

Because real code will live outside of WCS it will be possible to test it before deploying. However the framework will have to provide mocks in order to simulate the WCS environment.

This is an important feature.

Flexible URL Handling

 I plan to introduce a flexible and general url assembler allowing for arbitrary "routing".
The idea is roughly to introduce a Router object that will perform translations.
If the request is for a url like /article/London2012?page=1 (well, actually /cs/Satellite/article/London?page=1)
the url will be first split  in a list of components (List("article", "London2012")) and a map or parameters (Map( "page" -> 1)), then passed to a a router object,  something like this:
Router {
   def name2id(name: String) = /* map a name in an id */
   def apply(components: List, param: Map ) components match {
       case Nil => Map( "c" -> "Page", "cid" -> name2id("Home"))+params
       case cid :: Nil => Map( "c" -> "Page", "cid" -> name2id(cid))+param
       case c :: cid :: Nil => Map("c" => c, cid -> name2id(cid)) + params
   }
}
The sample router before will return a map where:
c/cid corresponds to the home page for  an url "/",
c/cid refers to a Page named "Hello" for url "/Hello",
c/cid/page refers to an article named "London2012" with an additional parameter  =1 for the input "/article/London2012?page=1".
The idea however is that the URL assembler will be  easy to change,  allowing for different url strategies that can be dynamically changed.
An interesting feature to consider is using  the router with an "unapply" function that will perform the reverse function, mapping an asset to an url.
This feature should be integrated in the link generation making pretty easy to control the url generation strategy of the URLs.

Agile Development

I plan to use SBT (simple build tool) for building the application.

The framework is mostly compatible with the existing frameworks, but it will move away from existing standards because it uses external code (not jsp) to do most of the real work. This poses a serious problem: hot reloading and distributed development.

While JSP have many limits, they have an important advantage, they are recompiled on the fly and the result is immediately visible without restarting the application server. In the case of WCS, it is pretty problematic to perform those restarts, not only because it is time consuming but also because the system can be remote, sometimes even located in another continent, and shared  with other developers.

While I think that working with a local jump start is a better options, I cannot exclude the need to use a shared development environment. Even with  a local development environment there is still the problem that restarting the application development is time consuming, and using non JSP code would require to do for each change.

To solve this problem (admittedly a major one) the fast track is to use the (non free) tool JRebel. The tool is actually free for Scala developers so it is not a cost issue. In the first place I think I will just use JRebel

However it is still a problem when the development server is remote. To solve this problem I am considering to write a custom class loader for a jar file coupled with a "file deployer", integrating the procedure with a SBT tool.

Conclusion

This is my first collection of ideas. I plan to launch a poll on Linkedin asking your opinion and I recommend to send me an email or comment this post about what you like or dislike of this idea.

 

Filed under: News, ScalaWCS No Comments