22Sep2012
Almost in all the projects I worked this year, I was plagued with legacy Fatwire versions that are still around. An outdated version of the software was often a major problem on all the development and the issues I was called to solve. And the worse is that customers often did not feel any need to upgrade.
I discussed so many times the reasons to upgrade at least to 7.6 that... I decide to write this blog post, both to share the wisdom with other, and both as a reminder for myself to shorten future discussions on this subject .
Fatwire 7.6 is still largely compatible with 7.x and 6.x: users won't notice any significant difference, so (unless you have a heavily customised UI) upgrading to 7.6 is mostly painless (except for CAS, see below). However you get a number of significant improvements pretty useful for new development . You can see here and here the release notes document for 7.6 and 7.5, but in this post I am trying to better explain what they really means.
Upgrading to 11g probably is the better choice, as I discussed in this blog post, however it somewhat changes the editorial team experience. In this post instead I explain the technical reasons for upgrading to at least to 7.6.
CSDT
Number one reason to switch to 7.6 is CSDT. While I was critical of CSDT in the past (because I do not like the eclipse plugin - I still prefer either Fatclipse or my homegrown alternative fatproject) there is a feature in CSDT (that is similar to my own FatProject code) that is actually working well and it is a much complete solution than FatProject : the import/export feature.
CSDT lets you export an entire web site in XML format, including configurations and content model, and then reimport it in another instance.
Anyone working with Fatwire knows the pain having of multiple developers working on the same server, using ContentServer Explorer to edit pages, and how limited as a version control system revision tracking is. Modern agile development prescribes a continuos integration server. Furthermore, publishing is a tool for deploying content, but it is not great to deploy code.
Just to make a simple example, code has relations (a piece of code to work may require a specific version of another piece of code), but publishing just doesn't capture this relation, while version control systems does. So publishing template by template is very error prone and lead to inconsistencies between environments. Furthermore code is usually dependent on a specific content model, so it must be kept together.
CSDT gives finally developers freedom from those constraints. Since you can import and export a whole site, you can export it from a shared development environment then import in a JumpStart and use it for development. Furthermore, you can revision control the csdt export , sharing and merging it with other developers. And finally deploy it to the final, production server.
So definitely, CSDT (Content Server Development Tool) is the definitive tool to make Fatwire Development acceptable for a professional developers, used to work in their machine for development, accustomed to perform merging of others work, revision control the code as a whole, creating automated builds and running tests against the builds.
AssetCache
Asset Cache has been introduced in 7.6, it is only vaguely mentioned in the release notes but it addresses a very important problem: improve performances of uncacheable templates caching the assets instead.
Let's define the problem: your site is dynamic enough that many templates cannot be cached. Before version 7.5, Fatwire cache was based on the idea of caching the pagelets, the snippets of html generated by template execution.
This works well as long as the snippets you generate are not too many. But think to a search page, for example: you may generate way too many different renderings to cache them, so unless you want to fill the cache without any performance gain, you need to leave the search page uncached. And this can be pretty slow.
What it is really needed then it is the ability to cache the Java "pojos", the underlying objects that are created when you render a templates. To better explain what this means, consider the standard call <asset:load name="asset" ...> used to load an asset in memory for extracting the content.
The "traditional" Fatwire way is to load the asset in a pojo in memory, use it to generate a snippet of html (the pagelet), then throw it away and cache the pagelet. This works well for cacheable template, but uncached templates were a "don't do it" before Asset Cache...
Now, using AssetCache the underlying pojo is no more thrown away: instead it is cached, as it happens in many other java applications (like those based on Hibernate for example).
This feature lets developers to write uncacheable templates acceptable .... and save their butt when the performance of the site are unacceptable because templates are all uncached (somethings that happens too much often unfortunately) and cannot be cached for many reasons (complex logic, too many parameters, messy design and so on).
So if your site is now unacceptably slow, and you have just discovered that the templates cannot be cached because you passed 20 different arguments including the browser agent and the current date and time, you can still keep your job just recommending to upgrade to Fatwire 7.6 and then enable Asset Cache (and InCache, see below)... Actually AssetCache requires InCache, let's focus on it.
Incache
Incache was introduced with Fatwire 7.5. What is InCache? It is basically just ehcache, the underlying cache used with Hibernate and many other project around. It improves the traditional fatwire cache offering a number of interesting features. From the release notes, InCache, as ehcache, is:
- Distributed
- Scalable and High performance
- No single node needs a complete view of cache
- Communication via dependencies
- No shared disk
It is basically a better cache that works distributing the load within different nodes. More nodes can be added when required so it is not limited by the speed of a shared disk and can be scaled up adding more servers.
You can learn more here.
Rest API
Fatwire 7.5 introduces also a new API that replaces the obsolete SOAP bases web services API. This means... a lot. This Rest API is pretty verbose, but is it finally allows interaction with the server in XML and JSON, and not only through Form POST.. Using the rest API your web site can be written not only as a static site, but also as an HTML5 fronted calling backend restful webservices, a design becoming increasingly common. I would say that is the current state of the art since all the new sites includes (and uses) jQuery...
So, if you plan to advance your web site to implement some ajax call using jQuery or similar, of better you want to have code that can also update some assets and not only read them, definitely having the Rest API deployed is a must.
Single Sign On (CAS)
Well, I am not sure if the Single Sign On component (that is actually the open source project
CAS) is an advantage... or a liability. It was introduced together with the RestAPI both to protect restful resources and allows multiple applications to be plugged as extension to Fatwire (like Community Server and Gadget Server).
It is required since the Rest API is based on it, and since CSDT uses the RestAPI you cannot get rid of it if you want CSDT. You can skip the installation but you will lose RestAPI... and CSDT. Not recommended. However, it IS a major source of pain for upgrader, since it works using a double redirect.
When you access to a protected resource, or just log into Fatwire, CAS redirects to its server, generate a ticket and then redirect back. This requires a that DNS is properly configured (and a broken DNS is not uncommon in many intranets). This requires also that all the proxy servers in the middle can proxy the fixed URL and not a generic ip based URL.
CAS can make also impossible to log into ContentServer when you try to access using a tunnel and you are trying to do an emergency intervention to a customer that cannot quickly open you the firewall, so you trying to log into the the server through multiple ssh to the host server...
It is also a source of problems in publishing. Since it is used also by the publishing system, a misconfiguration can cause the publishing to start to fail after the upgrade... when it was previously working perfectly.
Last but not least
I am aware at least of one important security hole still present in 7.5 that has been fixed in 7.6. I prefer not to tell the details in a public blog post, but I strongly suggest to check release notes of 7.6... or just upgrade...
Conclusion
If you want
- develop in a healthy, isolated, version controlled development environment
- need caching options for complex uncached templates
- need to scale your cache up to the infinity
- leverage ajax using jquery
- fix some security bugs
- AND you are NOT SCARED of fixing DSN and PROXY configuration because of the introduction of CAS
then definitely go and upgrade .
28Jul2012
With this post I am starting to collect ideas about developing an Open Source Scala based development framework for WCS, tentatively named ScalaWCS. I am making public my intention in order to collect suggestions, critiques and, why not?, collaborators.
The rationale behind this idea is my desire (and I do not think I am alone) to modernise a bit WCS development .
I do all my work outside Fatwire/WCS in Scala. I love Scala a lot and I want to use it for Fatwire coding as well. Here I collected my ideas so far for this framework. I would really like to hear from you what you think.
Warning: the rest of this article will be hard to read for anyone who does not know Scala. Apologies for this. Maybe the number of developers is really small but Scala is really an exciting and powerful language and it is worth every minute you spent on it.
Move Away From JSP
The main goal of this framework is moving away from JSP. I would like to to do all the coding in Scala instead.
JSP are really meant to be used only as a presentation layer, but in Fatwire you end up to do much more than this in JSP, and it is not very nice.
Of course JSP are actually part of Fatwire, there is no direct replacement for them you cannot get rid of them. But you can use them as a stub to call your Scala code.
The initial, rough idea is that you will keep Templates and CSElement, but the body of a template will be simply something like this
<% wcs.Dispatcher.dispatch(ics); %>
(automatically generated configuring the default template ). You are not expected to code anything else.
So, what will replace JSP?
Scala based templates
The real code will be written in Scala and will be something like this:
package article
object Layout extends Template {
def apply(ics: ICS, c: String, cid: Long) =
<h1>{ "title"! }</h1>
<p>{ "body"! }</p>
}
Instead of writing JSPs you use for rendering a method written in Scala. For template coding, I am leveraging the embedded xml literals available in Scala.
Furthermore, I am going to leverage the powerful Scala syntax, that will allow me to write "title"! instead of ics.GetVar("title").
I understand there are some problems with this approach, mostly on development and deployment. More on this later.
Nice wrapper on Fatwire API
Using Scala, I plan to wrap all the existing Fatwire tags with Scala calls. Something like
<asset:load name="thePage" type="Page" objectid="<%=ics.GetVar("cid")&>"/>
<asset:get name="thePage" field="id" output="id"/>
will be available as a Scala call (using named parameters) like:
asset.load(name="thePage", type="Page", objectid= ("cid"!))
asset.get(name="thePage", field="id", output= "id")
This is however only the first step. The API will leverage some common elements in the API to be able to do
val thePage = asset.load("Page", "cid"!)
val id = thePage("id")
Each element will be functional and will return objects with appropriate methods allowing to extract fields using the apply syntax. A lot of work will be done on making the API as nice as possible
It would be nice also to be able to keep the tag based syntax in the XML literals.
This should be possible, however the xml literals are not compiled in Scala, while they are in JSP.
A simpler implementation is possible just interpreting them when generating the output. A more advanced implementation could be generating actual Scala code. This is more difficult to implement since it needs to leverage either a compiler plugin or a Scala macro. Advantages and disadvantages will have to be carefully considered.
Real Unit testing
Once we moved away from JSP, the biggest advantage of using this approach is the possibility to perform real unit testing. Any unit test framework can be used, either ScalaTest or Specs2.
Because real code will live outside of WCS it will be possible to test it before deploying. However the framework will have to provide mocks in order to simulate the WCS environment.
This is an important feature.
Flexible URL Handling
I plan to introduce a flexible and general url assembler allowing for arbitrary "routing".
The idea is roughly to introduce a Router object that will perform translations.
If the request is for a url like /article/London2012?page=1 (well, actually /cs/Satellite/article/London?page=1)
the url will be first split in a list of components (List("article", "London2012")) and a map or parameters (Map( "page" -> 1)), then passed to a a router object, something like this:
Router {
def name2id(name: String) = /* map a name in an id */
def apply(components: List, param: Map ) components match {
case Nil => Map( "c" -> "Page", "cid" -> name2id("Home"))+params
case cid :: Nil => Map( "c" -> "Page", "cid" -> name2id(cid))+param
case c :: cid :: Nil => Map("c" => c, cid -> name2id(cid)) + params
}
}
The sample router before will return a map where:
c/cid corresponds to the home page for an url "/",
c/cid refers to a Page named "Hello" for url "/Hello",
c/cid/page refers to an article named "London2012" with an additional parameter =1 for the input "/article/London2012?page=1".
The idea however is that the URL assembler will be easy to change, allowing for different url strategies that can be dynamically changed.
An interesting feature to consider is using the router with an "unapply" function that will perform the reverse function, mapping an asset to an url.
This feature should be integrated in the link generation making pretty easy to control the url generation strategy of the URLs.
Agile Development
I plan to use SBT (simple build tool) for building the application.
The framework is mostly compatible with the existing frameworks, but it will move away from existing standards because it uses external code (not jsp) to do most of the real work. This poses a serious problem: hot reloading and distributed development.
While JSP have many limits, they have an important advantage, they are recompiled on the fly and the result is immediately visible without restarting the application server. In the case of WCS, it is pretty problematic to perform those restarts, not only because it is time consuming but also because the system can be remote, sometimes even located in another continent, and shared with other developers.
While I think that working with a local jump start is a better options, I cannot exclude the need to use a shared development environment. Even with a local development environment there is still the problem that restarting the application development is time consuming, and using non JSP code would require to do for each change.
To solve this problem (admittedly a major one) the fast track is to use the (non free) tool JRebel. The tool is actually free for Scala developers so it is not a cost issue. In the first place I think I will just use JRebel
However it is still a problem when the development server is remote. To solve this problem I am considering to write a custom class loader for a jar file coupled with a "file deployer", integrating the procedure with a SBT tool.
Conclusion
This is my first collection of ideas. I plan to launch a poll on Linkedin asking your opinion and I recommend to send me an email or comment this post about what you like or dislike of this idea.
20Feb2012
Hi all, I have been silent for a while, although from access logs I see there are many visitors, and the number is actually growing.
Thank you to all the visitors, we are not dead, yet. Actually, I has been very busy so I had not even the time to think topics to blog about.
Now I am moving the blog and all its content to Fatw... ops WebCenter Sites . I got a good working relationship with some really great guys at Oracle , business is growing fast and it is time to build a proper company web site using WCS itself and not just WordPress.
Stay tuned.
19Jan2011
I received a number of emails about installing FatPhone, and it turned out that the Fatwire Eclipse Plugin allows to commit only content that has been checked out from the same content server. So basically the suggested way (use the Eclipse Plugin to deploy the code) does not really work as expected...
I found a rather tricky way to overcome this limitation, but also someone complained that having to install eclipse just to deploy the code is not... ehm.. "nice".
So I decided instead to find another way to publish the code.
After some experimentation, I am currently convinced that the best way to release FatPhone is writing some scripts that publish content assets using XMLPost. I am currently working on it.
I have some working code already, but since I have also some "real world" work to do, the process is taking a bit more that I hoped.
However I am still committed to release FatPhone (an improved and simplified version actually) , hopefully in a form that there is minimal work to set it up and running.
As usual, if you are interested, stay tuned.
16Jan2011
In current code there is just the layout!
We have already written the code for an iPhone website, but releasing it on the public is a different matter.
The code needs to be updated, cleaned up and documented. So we are basically rebuilding the old web site, collecting the code documenting and releasing it, step by step.
The first code drop is now on GitHub, here. IT IS ONLY THE LAYOUT! Not editable yet.
How do I deploy it?
To deploy it you have to follow the provided directions, admittedly not very detailed. It will improve...
You have to:
- manually create the site,
- deploy the css and javascript in your webserver,
- download and install an eclipse plugin (and eclipse, if you don't have it, too!).
After this, you can:
- checkout the code inside Eclipse
- then use Eclipse to publish it.
Definitely, not a task for the faint of hearth.
Ok, but what I will get?
Not too much, yet. Basically, a page like this:
Don't be fooled! It is just static html. There is only the layout, for now, and I put some sample html in it just to see the rendering.
Cannot be edited with FatWire, yet.
But it will became a standalone editabile web site in the foreseeable future. Stay tuned.
