What is wrong in Fatwire/WCS 3: Mixing HTML and Code

It is hard to believe that in 2012 we still we have problems of mixing html and code. Those problems  are supposed to have been solved since many years, but unfortunately they are still around.

In an ideal world, a web designer is also able to code. In a world even better, a web designer is also able to write jsp templates and knows fatwire tags.

In the real world, web designers, while often knowledgeable of  javascript, have usually no clue of JSP nor of specific Fatwire coding, so they produce only a mockup made of client-side HTML, CSS and Javascript. They leave to  "server side" developers the task of using that HTML/CSS/Javascript to create a content manageable website.

The problem with Fatwire/WCS is that to render the mockup code, you have to add Fatwire coding to it. So , because WCS is still a JSP based system,  you have to add  logic to extract the content model and put it in place in the HTML mockup. That would be fine if... it were to be done  only once.

But in reality, HTML code undergoes a number of iterations. Web designer will update the mockup and return it to server side developers. Those developers will  have to use the new HTML to update their templates code, that is now an heavily modified version, with added a lot logic in Java and JSP tags.

This is a big problem, because it is not easy. The usual process is figuring out what changed in the mockup  from the latest version used to build templates,  then go through the code to apply those modifications. Hoping that nothing will break.

When the HTML is heavily modified, starting from scratch is not unusual. In short, the process of updating templates when the HTML mockup changes is a real pain in the ass.

What is really needed.

The principle of separating presentation code (HTML) from logic is almost as old as the web itself. There are millions of solutions arounds for this, yet those solutions did not yet reach WCS in his core. We strongly need to implement for real this separations.

Furthermore, since HTML mockups are going to change ofter, we should be able to leave the mockup in his original form, in order to be easily updatable. Rendering logic should be applicable directly to HTML in his original form, without modifications.

What is wrong in Fatwire/WCS development 2: Deployment Hell

One "feature" of WCS, being a CMS, is deployment of code done in the same way as content. Code is treated as content and managed in the same way. We will see that this fact can create many problems.

A content editor doing is job changes some content, and then he approves it for publishing. WCS is smart enough to detect dependent content and requires the approval of related content to publish it in single publishing session.

This is great for a web site, where you only have to update a single content asset to update all the web pages referring to that content in the web site. Furthermore the publishing process is smart enough to invalidate only parts of the cache affected by the  changed content.

Developers should work in the same way: in the development server, a developer  can change  the template code, then approve it and finally publish it. Code should then go from development servers to staging servers and finally to live servers.

Let's put aside for now that having a single development server for multiple developers is a problem in itself (I will say more of this later), let's give a look at what developers really do and why this way of developing the code does not work as good as it should.

How developers REALLY develop...

There is a great variation in development procedures. The more common, even if now there better tools are available, many are still using the aging ContentServer Explorer  (now Sites Explorer) editing directly JSP code stored in the  ElementCatalog table.

Unfortunately, when you edit a JSP, the associated Template or CSElement is not aware that you changed the code with CSExplorer. So to make sure the "code publishing" mechanism work, you have to manually edit and save the   Template or CSElement corresponding to the JSP you edited, then approve it and finally publish it.

Being a manual process, way too often happens that someone forget either the edit/save or the approval of a changed template.

Also the propagation of the code from staging to live requires a re-approval of the templates. Although you can theoretically could just do a bulk approve, many people are scared of republishing everything. So what usually happens is that all the changed templates are manually approved and published, using manually kept release notes.

Since usually who deployed templates from development to staging is a different person from whom developed them, a floating document with the list of the changed element, or worse a flow of random email is used to propagate those informations.

At some point someone makes a mistake, forget to approve a template, distribute a list with the wrong templates ... and  problems not existing in development starts to appear in staging or in production, randomly.

When different developers are involved, or there is a turnaround in the editorial team, it happens way to often that you no more know what is deployed in which server. I have seen people periodically spending days comparing each template in different server just to figure out what went wrong and the origin of a bug.

But wait... there is more

Actually things can go much worse than this.

Another problem happens  very often when developers are forced to develop on a system disconnected from the staging/deliver chain.

This may happen for many reasons, the more common is some brain-damaged security policy, but there can are other more practical reasons, for example: "the connection from UK to India is too slow and we had to deploy a local development server".

The current solution to this problem is CSDT but to be honest, it is not  yet very widely used. People are very creative in solving the problem of distributing their development work. Some uses catalog mover, but I have seen people distributing their work as a database dump and even manually copying and pasting the code  in the Fatwire Advance Interface.

Needless to say, this is aggravating the deployment hell already described in the previous paragraph.

But the worse situation, that I have seen too, is when developers are developing in their development server, then some other people fix some issues (usually in HTML) editing directly templates in staging, and at the same time some urgent issues are also fixed manually editing templates  directly on the live server. The result as you can imagine is a total unmanageable mess. And unfortunately, even is is an extreme case, it happens.

What you really need

Java has a concept of  deployment unit:  it is called the "jar" file. Fatwire/WCS is one of the few Java places where code is not deployed in jars, but it is instead delivered as separate templates deployed through publishing.

What is really needed is that all the code for a site can be distribuited as single JAR fil, that can be easily deployed, tracked, compared, distribuited and versioned.

All the deployment hell I described would go away if instead of having a bunch of files, you have a jar. The jar can be built by developers, tested separately with bug report and fixed referring to a specific build, delivered to destination and deployed just copying the file and eventually running some schema update procedure.

Jars have a shortcoming, of course. Usually they require the restart of the application server to be recognized. This in a live environment is not usually acceptable. Nonetheless,  it is not always true that deploying a new jar requires the application server restart. There are plenty of hot-reloading  Java systems. Just to mention one, hot reloading of jars in JBoss. So it is possible a system where a site is deployed in jars that are deployed without restarting the application server (and indeed, I already implemented such a system).

I will continue to list WCS/Fatwire development problems in the next few posts before introducing my solution to those problems. Stay tuned.

What is wrong in Fatwire/WCS development 1: JSP are evil

I am close to release a new open source project that  tries to fix  many of the common problems in Fatwire and WCS development.

But before the release, I want to  discuss those problems, since fixing them  is the main motivations underlying my development effort.

Actually, many of the features of the framework I am doing cannot be understood unless you know which real world problems they try to solve.

This is not a rant.  I also have a solution to those problems and it will follow in the next few weeks. 

JSP are evil

The biggest issue in Fatwire/WCS in my view is that you are stuck with JSP (or worse, with the obsolete, very limited and almost undocumented "Fatwire XML").

JSP are designed only to be a quick way of rendering a dynamic HTML page using Java code. Something that is 95% html and only a little bit of code. They are no way  meant in no way to be a complete tool for doing generic Java coding.

They provide the flexibility of immediate availability (since they are recompiled and reloaded on the fly), and they provide the "markup first" approach that is useful when you need to render a lot of HTML and only a small part need to be coded and rendered dynamically.

Indeed this JSP  flexibility comes with a lot of limitation, while the coding in Fatwire development tend to be pretty complex, so complex that you end up writing big JSP so full of code that  may you have difficulties finding the HTML.

Here some of the limitations of JSP. First and before all, because a JSP is  a single method of a single class, you generally are not supposed to define methods and classes. You actually can define classes and methods, using the "<%!" syntax.

Hovewer, since it is not the way  JSP are supposed to be used, you cannot reuse any defined  method in another JSP . This is even worse when you also define a class. You can create classes inside a JSP but you cannot reuse that class in another JSP.

The only way of having code shared between different JSP is to create a separate jar with all the code and deploy in the application server Coding such a jar is relatively awkward, because you have to build it, deploy and restart the application server for every change. So usually this is done rarely.

For this reason, building libraries of code is not normally done in Java (as it should). Instead, the common practice is to create a library of "elements" called by JSP.

The problem is that a CSElement is not really meant to be a library doing complex things, it is meant as a medium to generate repeatable rendering of common elements.

The semantics of calling  CSElements, and using them as a library is, frankly, disgusting. There is no way return value from a CSElement , so you normally use a global environment and  side effects (altering the value of a variable value in a shared environment).

The JSP "language" does not offer any enforcement to using a CSElement as a library call, so everything is left to convention. You need to document clearly which variables are changed to see what is returned. This practice is error prone, hard to read and even harder to maintain.

Also it happen often that a CSElement "pollutes" the calling enviroment, so you need to use pushvar and popvar to preserve the environment. And this makes the whole procedure even more disgusting, unreadable and produces really bad code, where a lot of complexity is there just to move around variables, protect them and read side effects.

Last but not least, the invoking CSElements is really verbose and typing an invocation method is often so long that you copy and paste code. Long code doing little is also very hard to read.

How to solve JSP problems

1. What really is needed is a way to code in full, plain, clean Java, not in JSP. Classes offers all the power needed to build complex programs, and a JSP are a simplified AND limited form of Java classes. Using Java will immediately give you the ability to write methods and classes, keeping them totally reusable.

2. However you need to retain the ability of seeing immediately the result of a change because restarting the application server at each change is usually not really practical:  it is so slow that developers to avoid it will prefer to push tons of code in the JSP. A solution could be "JRebel" but since it is expensive for some teams (most notably indian teams) buying it can be a problem. So that solution should be cheap.

3. Coding in Java you will also want to avoid ton of out.println("...") filled of "\" to escape quotes just to generate HTML. Probably you may not not the "HTML first" approach of JSP  and you may prefer the "code first" of Java, but you still want an easy way of generating HTML.

4. Last but not least, JSP offers some well defined tag libraries for rendering the content model. Since the equivalent Fatwire Java API (the Asset API) is not even close to the quality of tag libraries, you need some  efficient way of invoking tag directly from Java code.

We have not finished, yet.

Actually we have just started, a lot more will come. So please wait, there are more problems to  discuss in the next posts. And the solution will follow, it is a promise. So stay tuned.

The new WCS User Interface

The new  Oracle-branded WecCenter Sites v11g is not a revolutionary product: it is still recognizable as the old, good ContentServer (formerly Fatwire, Divine and OpenMarket) we already know.  Nonetheless, in this new incarnation it looks like a big jump forward.

This release is probably defined more from what has been removed than from what has been added. Most of  the work done  on WCS looks to be a great effort to simplify, make uniform and consistent,  overall improve the user experience of the product. The first and more evident feature gone is the dash interface (but not its functionalities). It is also gone the insite interface,  but again,  insite editing is now even more powerful.

There is no more the dualism of 3 competing user interface (dash, advanced and insite), where you can do more or less the same thing in different incarnations. Instead, there is now one unified UI. It acts like the old Advanced interface when it comes to manage "advanced" content assets (mostly used by developers and webmasters) . The UI also acts like the dash interface to browse and edit content, but seamlessly integrates  insite editing to preview and edit content at the same time.

Let's do a walkthrough with screenshots to  understand better.

The Administrative interface

Let's start from the administrative (the old avanced) interface. It is now probably  going to be the less used but we start from it because it is familiar to the vast majority of current users.

On the top of the screen there is now a toolbar allowing to select the interface (now there are 2, the administrative and the contributor). If we select the administrative interface, as below:

It is now possible to create and search assets, but, important new, ONLY administrative assets: templates, cselements, attributes, definitioins and so on. It is no more possible create and edit content assets from this interface.

Creating an asset is still done in the old way. For example clicking on a start menu link to create a new template:

then the usual form is shown:

nothing new under the sun regarding the administrative assets (except a css restyling). The interesting parts comes now.

 The Contributor interface

When we select the new Contributor Interface things start to became new (and interesting):

Creating a new element now requires you select it from the menu. The new  action  now appears in a pull down menu:

Content can be edited now in a form, so in a sense it is still possible to do form oriented content editing. See below and please note the toolbar of saving, approving , previewing and deleting.

Understanding cache dependencies

WC Sites  (formerly Fatwire ContentServer) is a content oriented CMS, as opposed to other page oriented CMS. The exact meaning of being driven by the content is that you are required only to describe your content without considering how this content will be actually rendered .

This idea has a few consequencies that must be taken in appropriate consideration when you design and code a WCS implementation.

(Well this is not actually totally true, considering insite editing, that is a presentation oriented technology, however it does not change the concept - content and presentation are separated but related).

The relation between presentation and content

To better explain the problem consider a generic content, let's say an article in your site.

On the CMS you want to add it usually just once, but on the site the same article can appear a number of times in different web pages, in different formats.

For example an article can be displayed as a full text in a web page, as a summary in another web page or just as a simple link in many others.

All those occurrencies appears in different web pages, and you have to update  all of them when the underlying content model changes. For this reason WCS stores in the database dependency informations.

The main reason to keep those informations is because they are needed for an easy and efficient updating of the cache when the content changes.  In fact, when the site is rendered, blocks of html  (conventionally called pagelet) are generated from the content model and usually cached.

For each of those  pagelet a dependency is generated and stored. 

When the content model is changed all the stored dependencies are walked through and the dependent pagelets are invalidated and regenerated.

 In practice

You can see all of this in action simply inspecting the table SystemItemCache. This table is somewhat obscure but it keeps one of the more important informations used in publishing: dependencies between assets and pagelets.

This table is calculated while rendering the content model in an actual live site. Many dependencies  are actually calculated by the render:calltemplate. When you call a template, you are also declaring that a new pagelet (the one that will be rendered by the template you are invoking) will depend on the asset specified by c and cid. So a new dependency will be recorded for this asset.

Dependencies can also be recorded explicitly using the render:logdep tag. Indeed, you may notice that when you create a new template, some code involving a logdep is automatically added.

That code should NOT be removed: it is adding a dependency between the template itself (the code generating the pagelet) and the generated pagelet. Obviously, when you change the code generating the pagelet, that pagelet must be regenerated.

In general, a pagelet rendering a given asset (identified by c and cid) depends always on the asset itself (so a c:cid dependency is required) and the template that render it (so a Template:tid dependency is also required).

However things can became much more complex than this. There are other sources of dependencies: for example searchstate calls usually generate some dependencies, and an index gathered by a search can depend on so many assets that... an  "unknown dep" can be generated.

Those special dependencies basically invalidate each pagelet when any asset of a given type changes.

Common Errors

Unfortunately, caching and even worse dependency management are often misunderstood.

Some (untrained) developers are somewhat vaguely aware of the caching, but very often they completely ignore dependencies.

A common error is to create a template that depends on two or more assets (the first one being the c/cid, while the other is specified with extra parameters). Unfortunately, only the dependency specified by c/cid is actually recorded, so when the other asset change, the corresponding pagelet is NOT invalidated.

The net effect is a website that won't be updated when the content change (unless you invalidate all the cache).

Understanding Fatwire Multiple Caches

When it comes discussing the Fatwire cache,  I notices very often some confusion in customer minds. I am not very surprised of this, since  Fatwire has multiple caches and their relation is pretty complex.

In this post I will try to clarify the situation, giving  an overview that explains (almost) all of the them. I am omitting here the BlobServer cache for clarity.

The structure of the available caches is depicted  in the diagram below. Read on for an explanation.

Satellite Pagelet Cache

When you visit a Fatwire website, you normally access to one of the Satellite Servers.

In the diagram there is only one Satellite, but in a normal deployment you should have many satellite servers, ideally geographically distributed in net, and the possibly located close to your browsing location, internet-wise.

Satellite Servers are the ultimate cache in place. They cache the site, splitting web pages in pagelets. A pagelet is a piece of already rendered html. It is important to understand that a pagelet is not a whole webpage, but actually only a part of a page (think to the header, a summary, a generic block).

Pagelets are  assembled  in place by Satellite to build complete web pages. Note that because pagelets are expected to be small enough to be kept in memory, assembling of a page is expected to be very fast (no database is involved,  mostly of the cache is kept in-memory except bigger files are stored on the disk). Actually Satellite spools to disk pagelets larger than a given threshold (configurable in property files).

However Satellite does not have the ability to render templates, so when a pagelet expires or is invalidated, Satellite must grab the updated pagelet from  Content Server.

Satellite cache can be inspected using a specific tool, the Inventory servlet (using the appropriate username and password), and can be flushed using the FlushServer servlet.

Content Server Pagelet Cache

ContentServer generates pagelets on request of Satellite, executing template code (either JSP or XML). It also caches the output of the rendering, for 2 reasons:

• First reason, Satellite is optional, and Content Server can run a  site without Satellite, and must do it efficiently.
• Second reason , multiple Satellites can be  deployed, so you may need to serve the same pagelets more than once. Also a Satellite can crash, or another satellite be added, so serving pagelets must be fast as well.

ContentServer cache is  the second cache in place. You can inspect the Content Server Cache looking at the SystemPageCache database table and its attachments. The servlet to clean the content server cache is CacheServer.

Please note that when you inspect the ContentServer cache you will see the pagelets generated for satellite, however cached pagelet  for Satellite are stored in memory within satellite. So empting ContentServer cache won't clean also Satellite cache. If you want to clean also Satellite cache, that you have to do it manually.

A complete reset of the front-end cache involves both all the Satellite Servers and Content Servers.

InCache Asset Cache

So far we have seen caches that store  pagelets, that are fragments of rendered  html. Those caches works with templates having cache enabled.

However it is common to have some templates that cannot be cached. Typically this happens with certain templates that  always return a different rendering (for example, searches), and caching the output pagelet can is difficult and fundamentally pointless.

The most recent addition to the family of Fatwire caches is InCache, that basically caches the  underlying "pojos" used for rendering.  You may ask what those pojos are.

When you execute template code, you perform calls like  <asset:load  name="theAsset" ...> or <assetset:setasset  name="theAssetSet" ...> to load assets. Please note that "theAsset"  or "theAssetSet" are actually Java Objects (POJO  = Plain Old Java Objects) that will be used to render the content. Loading them is expensive because a lot of database requests may be required (or at least once, that is by far the slower operation).

Enabling InCache, those pojos are cached as well. So this cache helps to speed up the rendering of uncached templates. Indeed, the vast majority of the time is spent querying the database, and reusing already queried POJO helps a lot in speeding up also uncached templates. It also helps saving memory and reducing garbage collection.

ResultSet Cache

Last but not least, there is also the old and good cache of database queries. When you query a database, you store the result in a list in memory (in Fatwire they are generally IList, that caches the output of Java ResultSets). As long as no one accesses any more to the table from where the IList was generated, running the same query will produce the same result, so it may be cached.

Result Set cache is caching results of database queries for each query executed. ResultSet are associated to tables, so  the cached results are valid until the underlying database table changes. The result set cache is invalidated table by table whenever someone write in the database table from where content was retrieved.

This cache is helpful because ultimately all the content must be extracted by the database, but is is useful also when you write custom queries. It can be tuned editing futuretense.ini and flushed using CatalogManager servlet.

The Simplest way to use Eclipse for Fatwire Development

Fatwire development has always been difficult. The recommended way of editing templates has been for a long time... ContentServerExplorer. I dislike ContentServer explorer for many reasons. For example:

1. it is a windows only app, and I use windows only in an emulator from MacOS or Linux
2. the editor embedded is so limited that notepad looks more advanced (for example from notepad you can search...)
3. the process of starting an external editor, then commit at every small change you want to test  is cumbersome

So it is natural that I has  always been looking for a replacement,  possibly using a real IDE like eclipse for development of the JSP. You know, completion, formatting, syntax checking, working syntax highlighting...

There are actually a couple of  Eclipse plugins for Fatwire Development. For a while, Fatclipse looked to be an interesting alternative. The plugin worked well and I was happy with it. Unfortunately, it is no more supported and it has not been updated to work with more recent version of Fatwire (currently it shows Fatwire 7.0.3 as the more recent supported version, and I was unable to run it in recent eclipse versions).

Then the CSDT plugins came out, but I am not happy with it. It shares the same checkout/commit philosophy of ContentServerExplorer, and is not well integrated in Eclipse. Basically it is a Java port of ContentServerExplorer, embeddable in Eclipse.

I wrote a blog post on this subject and I even developed my own alternative, FatStart.  I have now found a better way of doing development using Eclipse, so this post is basically my "adios" to FaStart, replaced  by a simpler and  better method described here.

The simpler and better way to develop with Eclipse is... not using any plugin! I discovered it is not strictly required, and you can do all your coding  in a practical and efficient way just using a recent eclipse (supporting folder linking) and a local install of Fatwire, either the JumpStartKit or your own installation using this technique.

A note of warning

The technique describe here is of course completely not official. Furthermore, in a shared environment it can be dangerous. This technique works if you do your Fatwire development in your own local instance in your machine, then you publish the changed templates in a shared enviroment.

However, if you still want to work in the old fashioned way, where multiple developers access to a shared development instance through http, your option is still ContentServer Explorer or CSDT (or even FatStart, although I do not support it anymore).

UPDATE: according Val Vakar, chief architect of CSDT, CSDT does not support shared development, although I was meaning that multiple developer can access to the same Fatwire instance and, as far as I know, it is possible, but I may be wrong.

How to directly edit the Fatwire JSP

Fatwire keeps templates as assets. So each template has some metadata, stored in the database, but the actual code is still  in a file inside a folder. This folder is located in  the Shared/elements folder. The location of the Shared folder depends on the installation. JumpStart places it under INSTALL-DIR/ContentServer/7.x.y/Shared.  In a custom install you decide where the Shared folder is.

So, if you create a typeless template named "Layout", usually you can find the actual code inside the folder Shared/elements/Typeless/Layout.jsp, while a typed template (for example Body for a Page) will be under Shared/elements/Page/Body.ksp   The actual name of the file is show in the template itself,  in the Element section. Fatwire for versioning purposes can change the name in something like Layout,0.jsp, Body,1.jsp and so on. So definitely you have to consult your template asset to know the actual file name.

So it is a pretty obvious idea trying to edit the JSP directly on the disk... However, if you do this in a default installation, you will have the surprise that... nothing happens (unless you restart the application servers). Your changes are apparently completely ignored by ContentServer.

So I thought that, because of some internal magic, direct editing of JSP is plainly not possibile... until I saw an important guy of the Fatwire team doing exactly this! Direct editing of the JSP under elements, and ContentServer actually immediately reflecting the change!

So I became a sort of detective and looked around to understand how  this is possible. An finally I found it,  actually where I expected: in a property of futuretense.ini: ft.filecheck

If you change this property to yes, you can simply edit JSP on the file system, reload your page and the result will reflect your page! Extremely agile and flexible.

So, now that I know I can actually edit JSP directly on the disk, after enabling this property all I need now is a good JSP editor.  For example eclipse. Here is how to configure Eclipse for Fatwire Editing in 5 simple steps.

Configuring Eclipse for editing templates

You need a recent version of eclipse, at least H(elios) that supports linked resources, and the Web Development Tools (so you can edit the JSP). Previous versions do not support resource linking, and  without linking, configuring the eclipse project became difficult and tricky.

Let's see how to do this step by step.

1. Create a new Dynamic Web Project

First step is to create a  new project, more precisely a Dynamic Web Project. You can create it in a workspace, it does not matter where you place it. You will actually refer to the JSP inside the elements folder through a link. So create this  project, and this is what you should see:

2. Link with the WEB-INF folder

Now, we need jar libraries and tag descriptors from Fatwire to enable JSP editing, tag library recognition, java completion and the like . Basically, all the stuff in the WEB-INF folder in the webapp part of your content server installation.

You do not need to copy them in the project (although you can do it if you like). Instead, as shown in the next image, all you need to do is to drag and drop the WEB-INF folder of the cs webapp in the WebContent folder, then LINK it.  See this image:

Please remember to select LINK  (not COPY) when eclipse asks you what to do.

3. Link with the elements folder

Now you can actually link the elements folder for direct editing. So just select the elements folder under the Shared directory and drag it into the root of the project as shown below.

Again, remember to LINK, not copy. Here you cannot copy, it would not work.

Now you can start to edit the JSP. That is all, except that eclipse will show a few errors. Next steps are here only to fix them, but if you can survive with a few errors showing up you are basically done.

4. Disable XML validation

The first error is related to XML validation. Many xml files in fatwire have still the <?XML prefix with a capital XML, and this is wrong for the Eclipse XML parser (strictly speaking, Eclipse is right, those "XML"s follow a draft xml standard, not the official one). So what you have to do is disabling the XML validation.  See the image below:

Open preferences, go to Validation and then disable XML Validator.

5. Hide a couple of JSP Directories

Some JSP present in Fatwire cannot be validated by Eclipse. Most notably, many contains a reference to the portlet jar, others have umbalanced HTML and so on. All those problems are in the system JSP that I am not generally interested in editing. So  I generally just  hide them from Eclipse eyes (and validation).

The simplest way is to go in Preferences|Resources and add an exclusion rule to hide the 2 folders "elements/OpenMarket" and "elements/fatwire". However, this system worked in Eclipse Indigo in Windows but did not work on Eclipse Helios on a Mac.

In this case all I did is to drag a void directory named fatwire and another void directory OpenMarket under elements. I created 2  new links to void directories  hiding the 2 folders creating the validation errors. .

BE CAREFUL HERE: if you drag a void directory over fatwire or OpenMarket, it once happened all the files inside were also removed! So I suggest you temporarily rename the 2 directory to another name, drag the void directory to create the hiding link and then restore them to the original name.

Final Result and Benefits

This is how  the project looks in my eclipse:

You may notice the 2 links pointing to elements and WEB-INF and the two links to hide the OpenMarket and fatwire folder. Combined with the trick described in the first section of this post, you can now can edit the JSP using the complete power of eclipse (including validation and completion) and immediately see any change reflected in Fatwire .

WCS Tutorial 1.6 – Gator and Flex Assets

While Xcelerate provide the basic functions for creating and editing asset, the more powerful content modeling features are powered by the component called Gator.

To better understand  which features it provides, I note that in the installer code it is still referred as CatalogCentre, while Xcelerate is referred as ContentCentre. So it looks like to be something good to build Catalogs. Actually it is.

Basically, Gator extends Xcelerate providing a rich and flexible Content Modeling feature called "Flex Assets". Apparently,  the original reason because Flex Assets were introduced was to support the implementation of ecommerce sites. It is not a coincidence that the first demo using Gatos is actually an eshop.  In fact, an e-commerce site usually requires lot of different products whose definition change often and unpredictably. Definitely, to support those requirements, Gator and its Flex Assets provide:

• ability to upgrade content model easily often, just publishing it
• a rich taxonomy, since products need to be organized in categories and subcategories
• an user interface to manipulate content definitions, not just content

Flex Assets differences from Basic Assets

We have already seen Basic Assets, so it is easier explaining how Flex Assets improve them.

First and before all, Basic Assets define assets with a descriptor  file, while Flex Assets define assets using other assets.  In fact, we don't talk normally of "flex assets" but of Flex Families, because Flex Assets are really a group (or family) of assets working together to reach a common goal.

Creating a new family will  indeed create 6 different types of assets. It can be at a first sight confusing, but  can be understood thinking that some of those assets are "structural" (define the type of the content) while only 2 of them defines actual content. Basically, structural assets replace the property descriptor file used in Basic Asset.

Second, Basic Assets define content once and for all, while Flex Assets define content in an incremental way. We have seen that for a Basic Asset you have to create a descriptor that will list properties. For Flex Assets you instead have to create attributes, that are assets too, then  you put together the asset using a Content Definition asset. An user interface is available to create attributes and assembling definitions.

This means mostly that while Basic Assets can be upgraded only recreating them from scratch or doing some alterations to the database schema, Flex Assets definition can be actually upgraded just publishing attributes and definitions.

Flex Asset hierarchies

But there is more. Using just Basic Assets you cannot directly create hierarchies (you can  only using some special coding). There are only some built-in hierarchies like the SitePlan. Flex Assets make hierarchies very easy to build using Parents,  that are always part of a new family definition. Furthermore, parents are configurable just like content, creating Parent Definitions.

When you create the content, you can place a content under a parent. Furthermore, a parent can be children of another parent, creating a tree. Using this parent-children relationship you can easily create complex hierarchies, that are very useful when the user needs to organize a large amount of content.

A parent has the special feature that all its  attributes are inherited by  its  children content. This feature is largely used to create custom hierarchies. Actually given the simplicity, is a bit unusual to find a flex family without parents, since they  are very easy to use.

Flex Families Components

So, let's recap what we have said so far. When you create a new Flex Family, you go into the Admin panel in the user interface and   see the following screen:

The Flex Family Maker  creates a number of new different types. Let's explain their meaning and use.

First, you  define a new Attribute type, Content_A in the picture. Attributes are  fields of the content,  the building blocks.

Then, you will have to create definitions to put attributes together. So basically, to define an Article with a title, an author and a body, you  have to create 3 attributes corresponding to the fields, then one Content Definition (a Content_CD asset for the family in the picture) named Article assembling the attributes you have declared. You can create different content definitions, each of them will define a different Content type.

Now finally you can create content, this means you will create an actual Flex Asset of type Content, and you will then associate one content definition, like the Article Content_CD you may have created in the previous step.

You can also create assets useful to organize content in hierarchies. This works in the same way as the content. First you create a parent definition, assembling attributes, then you can use the parent definitions to create parents. The additional step is that you can now associate content  to parent to get a hierarchy. The whole hierarchy can then be displayed in the WCS user interface. See below an example of such a hierarchy:

WCS Tutorial 1.5, AssetMaker and Basic Assets

Basic Assets

Xcelerate provides some built-in asset (an absolute requirement to build a site). However, since WCS  has a rich and flexible type system, you can define your own assets. Well, you actually should, unless you stick using sample assets, but you would lose a good deal of flexibility.

There is a lot to say about all the available possibilities to define your assets. For now, we  limit the exploration  to the features  provided by Xcelerate, namely the Basic Assets. Actually, there is a  more powerful and interesting possibility, Flex Assets, but this feature is part of Gator, that will be  described later. Furthermore,  Basic Assets are still useful although I do not recommend building a site (unless very simple and unlikely to evolve) solely on Basic Asset.

Define a new Basic Asset with AssetMaker

Let's jump start on  Basic Asset  showing an example how you can define it.

Although a bit technical as a approach (this is one of Basic Assets's drawback)  you have to describe it creating an XML file.

A very simple example of a Basic Asset descriptor looks like this:

<ASSET NAME="Simple" DESCRIPTION="Simple"  PROCESSOR="5.0"
       DEFDIR="c:\Fatwire\Shared\Storage\Simple">
  <PROPERTIES>
    <PROPERTY NAME="value" DESCRIPTION="Value">
        <STORAGE TYPE="VARCHAR" LENGTH="80"/>
        <INPUTFORM TYPE="TEXT" WIDTH="50"
          MAXLENGTH="80" REQUIRED="NO" DEFAULT="" MULTIPLE="NO"/>
    </PROPERTY>
  </PROPERTIES>
</ASSET>

What a basic asset provide

It is a simple asset, maybe the simplest possible, but this example  demonstrates what an Asset can provide.

First , as asset, like a database table, is essentially a sequence of fields, here called properties. As you can see here, each property has a storage type, so basically properties are translated in  fields of a database table.

But you can also note an important difference from database table: for each field, it is declared  how the field will be edited. In this example, you simply use a simple input text form, but with some constraints:

• width 50 characters,
• up to a maximum of 80,
• the property is not required,
• default value  is a void string.

It is however also pretty simple declare a property of type blob, that  will generate a form with a file upload. The file, once uploaded, will be stored in a folder. As you may remember we just discussed that tables may have attachments. Here we define the folder where attachments will be stored, and  thus this information is mandatory.

Basically we have seen so far how an Asset extends the concept of database table providing informations for forms editing and attachments. Actually there is more, there are many pre-defined asset fields generating various for but we will investingate on those features later. For now, just understand the concept.

Please note that Basic Assets  have however some limitation.  Most notably:

• they require coding to be defined
• they are a bit inflexible so you cannot add fields later unless you manually alter database tables
• they have limited relationship (actually provided by a feature called Association)

In the following of this tutorial we will see how those limitations has been overcome by  Flex Assets, at the price of increased complexity and lower performances (however compensated by the widespread use of caching in WCS).

WCS Tutorial 1.4, understanding Xcelerate

So far we have introduced only the basic concepts of the ContentServer layer. Basically it is the intepreter of the underlying  web programming language of WCS.But WCS is an application, and his value is mostly there. Se we  start now examining applications.

I prefer to distinguish the different underlying modules and use their internal name, because understanding which module we are talking about make easier understanding how the various modules interact. For example we will see that a Role cannot impact on ACL because the ACL belongs to the underlying layer.

Now, let's examine the first applicative layer built in ContentServer, Xcelerate.

Xcelerate

Xcelerate is the internal original name of the CMS module (at some point in the history called also CS-Direct), and it is still available today,  and is better known as  the Advanced Interface. You can still see the Xcelerate name when you log directly into the WCS advanced interface using the  URL

http://www.yoursite.com/cs/Xcelerate/LoginPage.html

Also you can see many elements named Xcelerate when you look at the ElementCatalog table.

In ContentServer we have only a concept of code. We have seen that code is actually stored as an attachment to a database table. I remember that a single piece of code, stored in a file and attached to a record in the ElementCatalog table is called Element. Basically, managing elements is the main job of ContentServer.

Xcelerate introduces a new concept, the Asset. Creating assets, managing them and providing a rich user interface for this purpose is instead the main job of Xcelerate.

What is an asset? In its simplest incarnation, an Asset can be thought as a single record in a database table. However it is not just this.  Asset can be of different types, and each asset type has their own methods for editing. So basically for each asset there are associated  forms to create and modify it.

It is important to understand that an asset is not limited to a single database table: it can actually store references to other tables. For example, the asset Template, since it allows to edit an element, is stored mainly in the Template table but it references (and allows editing) also record stored in the ElementCatalog and SiteCatalog tables. In a sense, Xcelerate provides also a user inferface for ContentServer through some specific assets.

System Assets

WCS comes bundled with a number of predefined assets. Those assets are used to implement  some standard WCS features.

As an example, let's consider the asset Page. This asset is used to build a the hierarchy of web pages in  our web sites, and there is a standard feature, called the site plan, that renders this hierarchy. We will discuss of Pages and SitePlan in greater detail  later. So far I used it only as an example of system asset. But before we move on, an important note:

A Page is NOT used to represent a single WebPage. I mean, it is wrong to assume (as many does) that you have to create a Page to render each asset as a Web Page. Page are actually used to model Sections in your site, not the complete site.

It is also immediately worth mentioning that ElementCatalog and SiteCatalog tables have a counterpart as an assets.

Basically a single entry in the ElementCatalog table can be created creating an asset CSElement, while an entry in the SiteCatalog table can be created with an asset SiteEntry. Please note that each asset has also its own table (and sometimes other additional side tables). So CSElement and SiteEntry are stored each each in 2 different tables (CSElement/ELementCatalog and SiteEntry/SiteCatalog).

We already mentioned the Template asset, that is also the more widely used asset when you build websites. This asset actually creates both an entry in the Template table but also it creates an entry in the ElementCatalog, and even one or more entries in the SiteCatalog table. So basically a template is stored at least in 3 tables: Template/ElementCatalog/SiteCatalog.

We mentioned also the Page asset, and while it is stored mainly in the table Page, but the hierarchy on pages is stored in a separate table called SitePlanTree.

So far so good providing examples  that clarify there is not an one-to-one relationship between tables and assets.

Next step will be to examine custom asset defined in Xcelerate, the so called Basic Assets.