In this post I am interviewing David Henriksen who used AgileSites for the implementation of a Telco website and also developed an extension of the AgileSItes framework.
Q. Hello David, please to meet you. Would you like to introduce yourself?
I m a developer, and have 15 years of experience which includes Professional and open source
development in different kind of languages, such as C/C++, Java, Php, ASP.
Q. How much experience you have with Java development in general and with Fatwire in particular?
A. I have 8 years experience in Java, and 2 years with fatwire. That includes both backend enterprise
applications, and more frontend oriented work with Java and fatwire as a platform.
Q. So what you think of the normal Fatwire/Sites development cycle?
A. I think the normal cycle requires a lot of manual work. You have to manually create your attributes, definitions, etc. When it comes to templates, it is also very manual and therefore also very hard to debug, since it's pretty much trial on error, if things doesn't work. You really have to have a good insight knowledge to fatwire, for maximum use.
Q. Why did you decide to change it and try something different?
I was introduced to this framework on a workshop, and must admit I already saw the advantages within the first couple of hours, since I come from a Java world, and is a developer.
We have had a lot of overhead time with doing things more manually, since it requires a lot of know how, when it comes to debugging, but it can also be a trivial task, just to do more complex modeling - and when it comes to version controlling your model and templates, with several development branches, it becomes near impossible.
Q. What changed using AgileSites?
A. With AgileSites everything has become more rapid. It is super flexible, everything is source controlled, and supports therefor branching when it comes to modeling, and templating. Since the whole model is done in Java code, it is easy to restore broken elements, or even do debug stepping when a template is failing - if your unit test doesn't catch the error; this too is very useful. To sum it up, AgileSites has made development more rapid, more transparent and more flexible. It basically gives the developer the possibility to make best use of the time for developing, instead of interface clicking.
Before AgileSites, it was very hard to be an agile development team. Now with the flexibility that AgileSites offers, agility has become an opportunity and natural choice, when developing for Oracle Webcenter Sites.
Q. What is your work currently?
A. Currently I work in a smaller team, which is developing a new responsive website for a Danish telco brand, using AgileSites together with webcenter sites. Furthermore I'm using the ZKoss
framework for developing the self service part.
Q. You worked on an extension for AgileSites. Can you tell me more about this extension you did?
A. I wanted an easy way to model asset definitions in AgileSites, which could help with supporting
rapid development, but also give a certain amount of transparency of the data model which is a
given, when you design bigger sites with complex models. To solve this, I developed an annotation driven modeling method - inspired from JPA and
Hibernate. All you have to do, is to make a simple POJO class, which defines the asset definition and
attributes. It of course supports various properties, such as parent links, multiples, editors, asset
types and so on.
Q. What are your plans moving forward?
A. The next natural step, would be extending the model so it can be used as a data container, in the
templates. By doing so, it will add safety to the template attribute references.
After 2 projects done by my team with AgileSites and a lot of feedback from everywhere in the world, we are finally releasing AgileSites 1.8-RC1.
This is the first release coming from the London-based Sciabarra.com ltd officially supporting it commercially, and I am the CTO. The project is still Open Source and Apache Licensed, and it will be stay free for all users forever, including commercial ones. However since some larger organizations were looking for a commercial entity supporting it, here is it.
The biggest thing missing for a full release is the improved documentation, that I am writing. However the current documentation is still largely valid, and this blog post summarized the new things, so for a few days I hope you can live with.
So without any further ado, here is what is new.
Ready for Sites 11.8
The latest release of WebCenter Sites is 11.1.1.8.0, and AgileSites is ready for it
We introduced support for Web Center Sites 11.1.1.8.0 and retained 11.1.1.6.1 .
We now supports multi-devices and multi-web root with the same api as before. A new mechanism to write Elements that render different for each device has been introduced.
Improved TOOLS
We introduced an automatic configurator for quick installation. You are no more required to edit the configuration file, a GUI will pop up and detect automatically the correct parameters.
The Eclipse plugin previously generated 3 different projects and required Scala-IDE (althoug the code was all Java). This is no more: there is only one project for your app, and a java-only IDE is all that is needed. Basically any flavour of Eclispse is enough. IntelliJ idea can be used also, and only one project needs to be imported (the one where you do your development work).
CSDT interface now supports multiple workspaces. In the command line you can use this sintax:
wcs-dt import #MySite-11.8
to import from the "MySite-11.8" workspace.
Improved API
The API has been reorganized and extended, supporting easily also dependencies and recommendation (you don't need any more to use tags for this). Part of the API is now in the core, defining a set of "key" interfaces, while the implementation is an "add-on". This distinction because the API will have a new pluggable implementation, accessing to the content with REST services.
A very nice addition to the Picker templating engine is a "moustache" syntax for extracting attributes. So you do not need to do any more this for a simple attribute:
html.replace("#id", e.getString("Title"))"
but just write:
<div>{{Title}}</div>
and the Picker will extract directly the attribute from the current asset. This is part of the newly introduced Content interface. The Asset, the Env and a custom Model are all contents that the Picker can use.
We removed some oddities from the naming convention, so everything now it is uniform. In the code now the name of the element you call has the same name as the underlying class. You are recommended to give a consistent prefix to all types and subtypes (and ideally that should be the site name), separated by "_". Any prefix is removed from your code.
Under the hood
Lot of work has been done under the hood.
The most important is that we revamped the classloader. Now it is multi-jar (it was single jar previously). The big advantage is that you can now drop multiple jar, having them hot reloaded.
This opens a lot of possibilities, including full multi-project and plugin support (that we are going to leverage in the next release). Also it is more efficient because the poll is done only once in a while (currently once every second) allowing to use it even if your site had thousands of calls per second.
We worked hard to refine details to clean many stupid warning and errors and . For example, now in the core there is no more an "agilesites-core.jar" but an "agilesites-core_11.1.1.8.0_1.8.0.jar" with the full version number (just in case it is not clear).
Another stupid but not easy task was to remove warnings when you started catalog mover (now integrated in the shell too).
Java Server Pages are at the hearth of Fatwire and Sites development. The core activity in CMS development is extracting content created within the system to render it in a web page. Normally the web designer provides the HTML mockup, then the developer write code to replace key parts of the mockup with the actual content stored in the CMS.
Writing JSP templates is the main task of the programmers in charge of implementing the mockup in the CMS. Sites programmers may also choose to code XML templates, using the less powerful XML based proprietary language of Fatwire/Sites.
There is another option today: pure Java controllers with pure HTML views, using the AgileSites framework; it can significantly simplify the work of implementing the mockup, as described below. However, let us discuss the available options in order.
You may have significant problems with JSPs, because it is a technology not designed to implement business logic. Indeed, JSP is only a presentation layer, the View Tier of the ubiquitous MVC pattern. The pattern actually assume a controller layer. Unfortunately, a controller is not available out of the box in Sites, only the view. Furthermore, standard tags actually are verbose and requires the developer to perform all the steps to extract the model inside the view layer (a job that the programmer should delegate to a controller layer).
So in Sites often we end up with business logic embedded in the presentation layer (the view). Sometimes, it is simple business logic (just read the content). However, it can became pretty complex especially when you need to perform integrations. Long and detailed JSP, containing much more java code than HTML , are frequent.
Why you should not use JSP for complex logic
Writing java-filled JSPs are problematic because:
They are not easy to understand since they mix different levels of logic
The Java usable in JSP is limited (for example in a JSP you cannot define a class ) so you end up coding with a crippled Java
Sites API creates a global environment not really (containing only of strings and lists), that is not a very powerful environment to express complex business logic.
As a result of the considerations before, I consider the JSP only template technology of Sites a big drawback that hinder productivity and in many cases actually put projects at a risks.
It is not an exaggeration. Why?
Because developers working under pressure rarely have the discipline of follow healthy coding styles. Many projects ends up in a collection of large, hard to read and understand, complex spaghetti JSPs. Those JSPs use crippled Java so they are repetitive and often filled of copy-and-paste code. Finally they implements business logic that should really go in a controller layer.
So what we can do?
Proper separation of Presentation from Content
What we need in my opinion is the following:
A way to really separate the controller logic from the rendering logic (MVC)
The ability to keep the mockup in his original, HTML only format.
A nice, easy way to place content in the original HTML mockup
A way to cut a snippet of the complete mockup to use it in a template that renders a pagelet.
You can do it all with AgileSites. The framework (that is completely open source, free and usable with no license charges in commercial projects) satisfies those requirements by design.
Let's see how.
First, AgileSites lets you to code in Java everything. The details of its magic are in this post:
The simplest possible implementation of a template is then a Java class like this:
publicclassHelloextends Element {
@Override public String apply(Env e){
return "<h1>Hello, world</h1>";
}
}
Considering only this sample, AgileSites provided a solution for writing business logic using full Java (you can create classes, use inheritance, share objects and so on)
However, it looks like we created a problem that JSP were supposed to solve: producing HTML in pure java is cumbersome.
Actually, this java class is only part of what the framework provide: indeed, it provides out of the box the Picker, that addresses the HTML generation problem in a good way.
In AgileSites, the programmer uses the HTML mockup provided by the Web Designer, leaving it in its original form. Here is the code you can use to render the mockup with the picker:
@Override public String apply(Env e){
Picker html = Picker.load("/blueprint/template.html");
return html.outerHtml();
}
This code will load your mockup and emit it in output. We reached goal 2 (keep the original html), and we are close to reach the goal number 1 (separation of html from code), in an MVC way.
Of course, that is interesting but it is not enough. For example we may want a template that is only a snippet of the mockup. What we can do? Usually in JSP template development, we do exactly this: we cut snippets of the mockup and place it in the JSP. A practice that makes updating of the template a real pain. In AgileSites you do not need to do this. Instead, you can use this code:
Picker html = Picker.load("/blueprint/template.html","#content") ;
return html.html();
It is enough to cut the snippet we need. The Picker will “pick” (hence its name) the snippet identified by the id “content”. The syntax for selecting the snippet we want, is the same used for CSS and the ubiquitous jQuery library. The CSS selection syntax is so well know that should not create any problem to Java developer of any level with some exposure to the web. Note that we used the “html” method, that should probably be named “innerHtml()” since it returns the inner html of the block selected with “#content”. The the outerHtml() we used before returns the html including the node you selected.
Now things should became easier to understand. We also reached goal number 4, cutting snippets from the original HTML. However, we still need to place the actual content in the HTML. Here is where the real power of the Picker lies: it lets you to replace parts of the original mockup with the actual content.
The picker, combined with the API to access attributes described in the previous post, creates a very powerful tool to implement easily templates keeping the original mockup and let us to reach all the goals we discussed before.
This code:
html.replace("#title", a.editString("Title"));
combined with this html code in the mockup:
<h1 id="title">Welcome</h1>
lets you to replace html in the mockup with the content stored in an attribute.
But wait: there is more
Actually, the picker is much more powerful that just this. You can replace attributes, delete nodes, use complex selectors, append and prepend to existing html and so on.
You can get an idea of what is possible to do checking this example:
Fatwire (now Sites) API was defined around 1995 and was pretty cool... for the hostoric period when it was developed . At that time XML was pretty new and everyone was wondering which purposes it could have. The obvious idea was to use it as an extended programming language syntax.
Originally Fatwire API was based on a executable XML format, and every function is a tag. The whole idea was later moved in the JSP world with custom tag libraries. So what you have today is a set of JSP tags to do everything you need, including loops, conditions and variable management.
The problem is that the power of the tag based programming is very low compared to what you can do in Java, so you end up using a crippled Java managing a pretty primitive and verbose environment.
A great example of this problem is the code needed to retrieve an attribute. You have to do all of this:
Now finally you get the attribute value in the single variable "title" (that you still need to access as ics.GetVar("title"))
What this code does? Simple: it retrieves a java object (what is today called a Pojo) that is a value object holding part of the current asset. Then the code extracts from the Pojo the requested attribute as an IList.
An IList is the internal datastructure used by Fatwire (that is actually a cached result set, and it is more similar to a table than a list). Finally, you can extract the field value from the first row of the IList.
The code is so long and verbose because it uses the tag syntax (definitely not a good programming language syntax) and also exposes details (the IList) that should be encapsulated in some object. Ideally, the Pojo that encapusale the asset should be also have the attributes internally, not to be retrieved separately.
How all of this can be better?
Enter AgileSites, the open source Framework for Sites and Fatwire for Agile development. It offers a way simpler API to do the same operations: extracting an attribute.
Because AgileSites provides a way to code your template in Java (as explained in this post) , while keeping the full ability to invoke the Fatwire API, it was not too hard to implement wrapper cpde that makes this very frequent occurrence much easier.
In AgileSites you write:
Asset a = e.getAsset();
String title = a.getString("Title");
The "e" variable holds the current enviroment (actually it wraps the ICS object with additional features) . Since you can assume that the variable c and cid refers to the current asset, the whole
Now the a variable holds the current asset and provide a number of methods to extract various informations. Since extracting attributes is way the commonest occurrence, a method getString is provided. The equivalent of
How it works? First, all the underlying ILists are created lazily when needed with an unique name to avoid clashes. Then since the lists used to extract attributes uses always a field value, it is not necessary to specify that parameter. As a result, all you need to do is to specify is the name of the attribute and in which data type you want to convert it.
Actually there is much more. The API allows to access multiple fields, create links and even define slots and so on. But I will talk of this in future posts. Stay tuned.
In this post I want to explain the underlying ideas behind AgileSites.
The goal is not only showing what it does but also why and how it performs its magic.
In theory you may also apply the ideas in this article without using AgileSites itself, although I believe the simplest path is just download and use it if you like the approach. There are no strings attached: the framework is Open Source and Apache Licensed (so you can use it in your commercial project without having to release any code).
Simplifying Sites Coding
Everything starts when a Java developer looks at this piece of JSP code (the typical code you have to deal with when you code for Fatwire/Sites)
<assetset:setasset name="as"
type='<%= ics.GetVar("c")%>'
id='<%= ics.GetVar("c")%>'/>
<assetset:getattributevalues
name="as"
listvarname="ls"
attribute="Title"/>
<ics:listget
listname="ls"
fieldname="value"
output="Title">
<%= ics.GetVar("Title") %>
All of this coding is needed just to read an attribute. Code is so verbose because it uses JSP tag libraries. Tag libraries are not really designed to implement a complex business logic, only simple rendering logic in a form readable (and editable) by people who cannot (or does not want) code (for example HTML designers).
As many people point out, XML (and thus HTML) is not a programming language, and if it were, it would be verbose and unreadable. This code is a typical example of a simple task becoming complex because the tool is meant for a different purpose. What this code does, after all, is just retrieving an asset and then extracting an attribute from it.
A Java/OOP developer would immediately say: the intermediate IList used is a technical detail that can be hidden inside some object, and many parameters (like "c" and "cid") can be considered implicit, so they do not need to be expressed.
So if we were using Java, you would define a couple of classes and the code could become as simple as:
e.getAsset().getString("Title")
Indeed this is how attribure retrieving looks in AgileSites, and this simplicity is one of the (many) benefits it provides. The problem here is to understand how it reaches this goal.
Switching back to Java
The first idea to simplify Sites coding is to use a library. In plain Fatwire/Sites you can do something like this:
<ics:callelement name="Lib/ReadAttribute">
<ics:argument name="attribute" value="Title">
<ics:argument name="source" value="as">
</ics:callelement>
Indeed many people uses a library of reusable elements to automate attribute retrieval. The result however is not much more readable than before, only slightly shorter; but elements (because they are not really meant to be an OOP construct) have problems like changing the enviromnent with sometimes unwanted side effects. There are thus drawbacks when using a library of elements.
A plain Java class as a library is much better. So why not use it? Indeed AgileSites defines a couple of classes similar (but more complex than) to those:
class Asset {
String getString(String attribute)
}
class Env {
Asset getAsset()
}
Internally the Env.getAsset() actually performs the assetset:getasset and stores the result in the class instance (code is optimized to load the assetset object only once). The Asset.getString() actually uses the assetset:getattributevalues to retrieve the attribute (code is optimized to load attributes lazily).
All of this is not a great magic for a Java coder. The difficult part is how to write Java code in the JSP-only world of Fatwire/Sites environment, invoking using Java tje API available only as JSP tags...
Java coding in a JSP world
Writing code in Java is not uncommon in the Sites development world. The real problem is that the process is not dynamic and productive enough. In J2EE world if you add a Java class to the webapp, you need to redeploy it to be able to see changes in Java code. Because all of this is pretty time consuming, Fatwire/Sites developers does not do this very often, and instead just rely on JSP (stuffing a lot of java code in it).
AgileSites strives to be agile, so instead of asking the developer to redeploy the application for each change, it does its magic with a core library that can load dinamically a jar built continuosly without restarting the application server.
It is important to note that this ability is not new at all in the Java world. Applets has been working this way for ages: they load in the browser a Java application packages in a jar from the net, and a browser restart is not needed. Indeed, since the ability to load a jar is built into Java from version 1.0, AgileSites uses the Applet class loader to load such a jar.
So your code can be just plain Java. Templating is done with the Picker keeping the view in pure HTM; I will cover this feature in another post. You code your business logic in full, uncrippled, clean Java. Then your code is compiled on the fly and packaged in a jar and it is available for execution putting the shell in continuos compilation mode.
When you invoke a template, the jar is reloaded (only if it is changed) and then your code can be executed. This way your java coding is as dynamic as JSP coding but you do not have to use the crippled java offered by JSP.
The whole trick is explained in this Architecture diagram:
Invoking tags from Java
The next problem is writing code that can actually use JSP tags. Fatwire provides a Java API for asset manipulation but many functions are only provided as a JSP tag. Actually JSP tags in Fatwire/Sites are only a proxy for invoking the internal Sites API (really the one designed for use with the XML tags) exposed through the ICS facade. However invoking a tag with ICS is pretty awkward. This is why AgileSites provides a layer for simplifying access to tags.
When AgileSites is built, it will inspect the Tag Library Definition of Fatwire/Sites to build a wrapper library to access tags from java. Using this wrapper library there is a direct mapping from a tag to a java call, than can be show with a simple example. A tag invocation like:
Using this library for example the retrieval of an attribute can be implemented with this following code (equivalent to the assetset:getattributevalues but encapsulated in a clean java class).
public String getString(String attribute) {
return e.getString(at(attribute), "value");
}
private String at(String attribute) {
String attrList = attribute.toUpperCase();
if (i.GetList(attrList) == null) {
String attrType = e.getConfig()
.getAttributeType(ics.GetVar("c"));
AssetsetTag.getattributevalues().name(as)
.attribute(attribute)
.listvarname(attrList)
.typename(attrType).run(i);
}
return attrList;
}
If you read this sample you should be able now to understand fully how is possible that the complex JSP coding of the first example can became the simpler Java of the second example.
However AgilesSites offers more: for example the pure-html templating . Stay tuned for the next post if you are interested.
