Please stop using libraries and learn the tags!

In the  projects I have worked in the latest two years, a pattern emerged: developers were developing using libraries of code but  not using directly fatwire tags.  So, instead of using fatwire tags, they were calling all the time some "elements".

That is puzzling. Why they do this way? Why they use those libraries instead of just LEARNING THE CODING OF THE PRODUCT? There is no real advantage of using those libraries. A lot of code is needed to pass the parameters, other code is required to read the result, and overall it is more complex than just using the fatwire tags in the first place. What is worse, fatwire tags are documented. Those libraries are usually NOT.

I suspect this is a common pattern because implementers follow directions from someone that is providing the library. However, the library that they use is very often (I would say always) much worse than using directly the fatwire API. This is because Fatwire does not really provide features to write libraries in templates. They should code custom tags, instead. Something very rarely done.

I know that Fatwire API is somewhat confusing; for example having to extract each attribute before rendering it looks to be too much effort. But this is not an excuse for avoiding to learn it.

For example to render an attribute of an asset, an assetset:getmultiplevalues and then a ics:listget it is all that is needed to render an attribute.

It is shorter than calling the "getpageattributes" call I have seen too many times.

Real code I saw yesterday do:

• collect a number of parameters
• calling the element that extracts ALL the attributes and store in an hashmap
• retrieve the hashmap
• extract the values in java variables
• sometimes bring back those variables in ics variables
• add a lot of checks for null values

But using a library can generate be much worse results than this.

I have see in a project developers calling all the time a library that was extracting the attributes filtered by date. EVEN FOR NON DATE SENSITIVE FILTERED ATTRIBUTES.

So the coders, instead of just extracting the attributes they were needing, did some dirty trick with variables to simulate a different time range and disable the filtering when it was not needed. An extreme case of total ignorance of the enviroment they are working in. Code 10 times more complex totally unreadable and filled with hack and tricks.

So I feel to give a big warning to all the Fatwire Customers (and I know many of them  read my blog and listen to me): check that the developers knows Fatwire coding. It is not that hard. Below there is a list of questions developers will have to know the answers (without the answers... so developers that will try to be prepared for my test will have to learn).

Unfortunately many indians and chinese companies have this bad habit: since they cannot find the developers with the proper skills, they throw in the team generic java and jsp developers, and instead of providing training, they put some team leader that provides a library. Then they (regularly) lose the team leader and the developers are left hacking the library without never learning the enviroment.

Simple Test: do you know enough of fatwire for coding?

• What is an IList?
• How do you extract fields os an IList?
• How do you loop an IList?
• How do you load a basic asset?
• How do you extract fields from a basic asset?
• How do you load a flex asset?
• How do you load attributes of flex assets?
• How do you read a single value?
• How do you read multiple values of an attribute?
• What you get when an attribute is a blob?
• What you get when an attribute is an asset?
• What is the impact on the cache using a calltemplate?
• What is the impact on the cache using searches with search state?

CSDT Toolkit

I  recently had to perform a large migration and I choose to use to use CSDT. Well, this may be contradictory with some post I did in the past, where I stated I was don't  like CSDT. Actually, what I dislike is the CSDT eclipse plugin.  While it has some usefulness, it is not that great andI currently just use Eclipse in a local jumpstart using this trick for editing.

But CSDT command line is pretty useful when used as an export/import tool. It could be better however. Using it, I learned a few lessons and I wrote a couple of scripts to make it easier to use. I stored them in my FatGoodies repository, here. You can downloading the scripts with git or just display them raw and save them locally (there are just 3 files).

CSDT jars are  not included though. Being proprietary software, I cannot distribute it. You have to take the relevant jars from your Fatwire/WCS installation and copy them in the lib subdirectory where you placed the scripts. Read the README for the list of the required jars.

There are 2 scripts: a launcher, and an exporter. Scripts are written for the BASH shell and works out of the box in common Unixes (Linux / Mac OSX). For Windows you need to install CygWin or another unix like environment with Bash like the  one that comes with Git for Windows. I actually use this last one, not Cygwin, but I do not expect problems with Cygwin

The Launcher

The first script is the launcher. It actually comes in 2 flavors: csdt-unix.sh for Unix and and csdt-win.sh for windows.  The script does the dirty work of building the classpath you need to lanunch csdt and provide all the numerous parameter that are making a pain to use it.

However the intended use of the launcher is from a  wrapper script. You should copy the wrapper.sh in your own file, and  edit it filling in the parameter: location of your content server, username, password and site. Defaults are good for a local jumpstart kit.

Your wrapper will the  invoke  the launcher, and then working with CSDT becomes easy. The launcher is separated from the wrapper because you can have multiple wrappers.  I actually make copies of the wrapper script for each server I want to use: for example staging.sh, delivery.sh and so on.

The wrapper script  synopsis is then just: ./wrapper.sh [<selection>] [<command>] 

If you run it without argument,  you will get the list of all assets in the configured server.

The first argument is the <selection> of the assets and it defaults to @ALL_ASSETS. Its syntax is documented in the Content Server developer tools manual.  Basically, AssetType:AssetId select a single asset, AssetType select all the assets of a given type, @ALL_ASSETS select all the assets and @ALL_NONASSETS select the configurations. It

The second argument is the csdt command to perform, one of  listcs, listds, export, import. You can now do easily stuff like:

A more robust export

The previous script is good enough to launch csdt. However, when performing a full export of a site,  there is a problem. CSDT is somewhat fragile: if there is an inconsistency in the content model , the export fails.

There are many possible incosistencies: for example, an asset using a deleted attribute, or a locked asset or something shared with another site. Many inconsistencies that are not a big issue and survive to publishing and don't impact site rendering are fatal for CSDT.

The problem is particularly evident when you try to export all with @ALL_ASSETS, you may to wait a lot of time for answer, then finally the export stops with an exception. You have to  fix the issue and then run again the script. When you have thousand of assets, the loop "export - fix - try again" can be unacceptably slow.

The solution is, sadly, exporting all the but one by one, collect the errors and then fix all of them at the end.  This is what the second script does.

The synopsis is: ./export.sh <wrapper> [<selection>] 

The script requires as the first argument a wrapper script, one of those created in the first part of the post, providing the parameters to connect to a specific CSDT instance. As said in the previous section, you create one copying the wrapper.sh into another file and then editing it providing the appropriate parameters.

The <selection> is by default @ALL_ASSETS but you can use also something like "Template" or "CSElement" to select only one specific asset type or even "CSElement:123456789" to select a single asset.

The export script will export each asset in the selection separately.

The output will be logged in a file called out/<wrapper>/<asset-type>/<asset-id>

The script has 2 advantages:  it won't stop if there is an error, and it is incremental .

While csdt alone will bomb if there is any error and stop processing, the export script will simply log the error and continue. However you can stop the processing at any time, and then resume: existing logs will be used as markers, and already extracted asset will not be extracted again.

Once you have all the logs, you can search for errors. The simple command

grep -r out/staging Exception

will return a list of assets that logged an exception, something like this

out/staging/CSElement/1327351719222:Exception
out/staging/CSElement/1327351719637:Exception
out/staging/CSElement/1327351720069:Exception

You can then inspect the export that failed the export, fix it and then export them again. Since by default already exported assets are skipped you have to add "-f" to the script to force a re-export. So for export again the first asset of the previous list you may use

./export.sh  staging.sh CSElement:1327351719222 -f

Cross Platform Jump Start Kit

How to use a Windows JDK on Unix (Linux/Mac) and ViceVersa

I am a Mac user and I usually deploy my work on Linux servers, so I work normally and comfortably in an unix environment, both for development and delivery. Because of Java portability and because paths on  Mac and Linux are the same, I do not have any problem moving my JumpStartKit or local install between Mac and Linuux: just copying files and placing them in the same location is enough (paths are harcoded in a Fatwire installation so the paths must be the same).

However one day I found myself,  with a JSK built for my Mac I had to run in a Windows environment, and with a JSK built by someone else in a windows environment that I had to run in my Mac  (Unix) environment.

I have previously  "converted" those JSK using string replacement tricks, but it is definitely an error prone and annoying task. Furthermore,  sometimes I have to make some changes, using Mac, and then send it back to a windows user.

So I started  to think if replacing strings in JSK configuration files and database to run it on  another environment can be avoided. After some thinking and experimentation, the answer is a full YES. With a couple of simple os level tricks you don't have to change anything.

How Run an Unix JSK under WIndows

This is easy.

The basic finding is that Java treats slashes and backslashes in the same way.

So a path like /Developer/Fatwire/jsk (a unix absolute path) it is actually interpreted in windows as "\Developer\Fatwire\jsk", that is an absolute path too, except that is relative to  the current drive.

So, to run an Unix JSK in Windows, all is needed is  placing it in the corresponding directory it would be under Unix, then  change the current drive when you start it to  the one where the JSK is installed.

To be sure of the current drive, you do better to start JSK command line, as explained below, instead of relying on the launcher provided by the JSK itself. Here is the procedure

•  Open the Command prompt
•  Change to C: if I placed the JSK in C:/Developer/Fatwire/JSK
•  Go to C:/Developer/Fatwire/JSK/App_Serverver/apache-tomcat-6.0.30/bin
•  Copy catalina.bat.org to catalina2.bat
•  Start it from the command prompt using catalina2.bat run

Please note you may need to tweak the catalina.bat adding more memory, more perm gen space and so on. For a hint of the changes needed, give a look in the original catalina.ba,t looking for JAVA_OPTS and CATALINA_OPTS.

How Run a Windows JSK under Unix

It is slightly more complex to run a Windows JSK on Unix without changing strings in the JSK.

The problem here is that a Windows path is usually something like "c:\Fatwire\JSK\7.6.2\". Now, Java does  not care of the slashes. Reverse slashes are translated in forward slashed automatically.

The problem is that a path starting with C: is an absolute path in Windows but… a relative path on Unix! So trying to start JSK simply won't work unless you do something fix this confusion.

So my technique is installing the JSK in the same directory in the filesystem without the drive name; in the example before, in /Fatwire/JSK/7.6.2

Then I open the terminal,  I go the tomcat  bin directory (something like /Fatwire/JSK/7.6.2/App_Server/apache-tomcat-6.0.32/bin), do the trick described before for Windows to start it (I copy catalina.sh.org to catalina2.sh, then I tweak the OPTS taking the parameters from the catalina.sh).

Now, before starting tomcat I also do this :

ln -sf / C:

this way you create a symbolic link named "C:" (that is legal file name on unix)  pointing to the root folder.

Now you can start tomcat with ./catalina2.sh run

This way, the current directory is the bin directory of tomcat. Every path is relative to this directory. This directory normally won't change, at least I am not aware of any internal change directory inside Fatwire.

So any reference to paths will start from C: that will point to the root folder. Since you have placed the JSK in the same position as it was in Windows, now every path now is resolved correctly.

That is all. Enjoy now working in a team without having either to switch to windows or (worse) run the JSK in an emulator(a real pain I do not advise to anyone unless you have really a lot of ram)

Seven signs of a bad Fatwire implementation

I worked as a Fatwire consultant many times, and it was a very common task reviewing and assessing the quality of  the code developed by other developers (usually because the site was presenting a number of problems).

Since I saw  some mistakes many times, I decided to write down a list of the more common errors. You can check your code base against this list.

If your code shows one or more of the following signs, you are authorised to be a bit worried. Usually this means  your developers were  not too skilled on Fatwire, they misunderstood something basic, and your code may need to be fixed.

Please note: none of the following mistake is lethal but when those signs shows up, usually there are many more other, more specific, errors. Because it simply means developer never completed just the basic training before starting the work.

1. There are numeric ids in template code 

While this mistake is probably not the worse and could be fixed quickly with some manual work,  when I see this, I consider the bad sign #1 because almost certainly there is much worse in the code

It is a mistake to use numeric id in code because the whole point of a CMS is separating content from presentation. Content should be easily  changed all the time.

Since new content  have different id each time, committing to a numeric id means committing to a particular content, thus making  it impossible to change some particular content without breaking the site. Is it a shortcut taken by developers that usually did a bad job modelling the code and then they have no other ideas how to get certain content.

Furthermore, there are plenty of ways of avoiding to introduce a direct numeric id: creating relationship between assets, use template maps, even loading assets by names instead of by id.

So when you see this mistake, I suspect there are some misunderstanding of Fatwire development to a very basic level.

2.  Code is filled with direct sql queries

This is another terrible sign when I see this.

Actually, while it is is not strictly forbidden using direct queries in Fatwire, and there are a few cases where they are  actually required (for example in bulk approval), there are also plenty of libraries  wrapping direct database queries with more high level functionalities that should be used in preference.

Resulting code using queries will be longer, probably less secure (opening the door for  sql injection for example), and sometimes inefficient . Tag libraries usually do a better job of accessing the database than direct queries,

Furthermore, tag libraries do more than just querying the database: they do caching,  dependencies generation and so on. All of this  won't happen with direct database queries.

My feeling when I see code filled with SQL queries is that developer never understood how Fatwire tag libraries really works and their relationship with the content model. They are coding in the "php" way and they are not really  Fatwire developer. Probably they are just jsp or php coder in a hurry coding without a clue of the system they are using.

3.  Templates do not have a regular naming structure

Fatwire is a  content model driven CMS. The main underlying idea is the content is type.  You may  have assets types like Page, Article, News and so on. Templates act as rendering functions on each type. Furthermore,  template rendering often rely on a form of polymorphism:  you specify only the rendering function, but  you expect the same function to be available for many different types. This works if you follow regular naming rules.

In short, you should expect  for each content type a set of standard rendering functions:  for Page something like Page/Body, Page/Detail, Page/Summary; for  Article: Article/Body, Article/Detail, Article/Link. The regularity is important when you call a template specifying only the rendering functions you want, as the standard render:calltemplate tag expects  (for example a Summary, but it can be the summary of a Page or of an Article or of a News).

So I expect to see something like this

• Page/Body
• Page/Summary
• Article/Body
• Article/Summary
• /Layout
• /Link

When instead I see something like this

• /HomePageHeader
• /LeftSummaryNews
• News/HighLevelDetail
• Article/HomePageBox

I immediately suspect the rendering logic is broken (or at least it does not match best practices)

4. There are too many flex families

Flex Families are pretty rich. You can do a lot with a single flex family. Just with the types generated by the "new flex family" wizard,  with a single content and a  parent type,  you can create many content definitions (subtypes).  You can further refine your flex families adding  specific content and parent types inside a single flex family.

Your site usually just requires a single flex family, or a limited number anyway. When I see there are tons of flex families, I suspect immediately the worst.  Usually I find  there is a single content definitions for each entire flex family, and developers created a new flex family for each single content type required.

This is a bad sign, it means developers never understood something pretty basic on flex types.

Aside from duplication of attributes, one of the problems with this approach is that some relationship are allowed only within the same flex family. Having a full flex families for everything  needless limit you in creating relationships between assets.

5. You have to add each asset to a page to display it

This is another very common mistake. Many developers associate the idea of a "Page" asset with the "Web Page" concept and they end up  thinking: for each web page, I have to create a Page asset, and add assets to it. Then  they code the whole site this way.

This is just conceptually plainly wrong. There is nothing preventing you to render a specific asset as a full web page, when it is appropriate. When you have to render a specific article, and your web page only contains that article, you should  render it using as an entry point that only that asset,  not a Page that contains the asset.

Pages in Fatwire are containers of other assets, and are used mostly to build section pages and laying out the siteplan of a site. But a single asset (an article, a news, a faq) can be rendered (and normally is) as a full web page. Pages are usually the inner nodes of the site plan while non-Page assets are the leaves.

It is a completely unnecessary burden for the editorial team forcing them to use a Page to rendering a single assets: editors will have to triple efforts, because this way an extra Page asset must be required required and each asset must be be placed manually in that page. All of this is unnecessary.

6. Templates called with a lot of parameters

Templates are usually cached; to create cacheable templates, you need to limit the number of parameters, because for each combination of values parameters, a new cached element will be generated and stored.

Sometimes developer "forgot" (or just plainly ignore) about the cache, and treat template like "subroutines. Then they start passing around all the parameters they feel are required for implement every rendering, even parameters that are actually used only by inner templates.

This approach normally causes an explosion in the number of cached templates with many other nasty side effects. To limit the size of the cache sometimes many of those templates will be uncached, but this will in turno slow down the site.

Actually, the absolute minimum number of parameters (ideally, only the standard c,cid ) should be passed to a template.  Relationship between assets should be extracted by the content model.

Of course doing so  lead to multiple access to the same informations. Apparently some of them were already available before, so developers felt worth to be passed as a parameter.

But it is the job of the cache to speed up the rendering, so better to repeat the same lookup and then cache the template, that doing a lookup only once and leave the inner templates uncached because no more cacheable given the high number of parameters passed.

7. Multiple parents instead of asset relationship

This is confusion between relationship and inheritance.

Assets can be linked each other with an attribute of type "asset". Rendering the attribute will bring up  an id that can be used, while rendering, to call another asset that renders this related asset.

Unfortunately is is possible to relate assets also declaring an asset to be the parent on another asset. And it is possible to give multiple parents to an asset.

As a consequence, some people assume that the "parent-children" relationship is the way assets are related each other, so they ends up creating only parents and putting each other in relationship with the parent-children relations.

Unfortunately, parenting has a bad side effect: inheritance. Fatwire for each asset keeps in the database informations to easily locate not only his attributes, but also all the attributes of the parents, that are inherited.

Needless to say, this poses the unnecessary burden of extra informations kept, slower editing (sometimes way slower)  slower retrieving, and finally for  many relationship, accessing to related assets becomes awkward and limited.

The most frequent error in Fatwire site development

One of the more frequent mistake in Fatwire Development  that I have seen so far is overlooking the caching behaviour of render:calltemplate.

This mistake is so frequent that I take it for granted. It is always the first thing I check. I have seen it  almost in all the project I was consulted for support. The usual symptom is  that have performance problems.

Unfortunately, it can be a devastating error if discovered too late. Because calltemplate is ubiquitous, if you used it incorrectly you may have to revisit the code of your entire site, if not a complete rewrite.

In a nutshell, the behaviour of render:calltemplate is: retrieve a cached element if it is available, or execute the code to generate the html, then cache its output. In general, in 99% of the cases, calltemplate should NOT be executed: it should instead return a cached element. This behaviour has a deep impact on the way template code must be written.

If you write your code incorrectly, the  template cannot be cached. As a consequence, performances will be very bad, because retrieving content from the database is very expensive. This is the whole reason because there is a caching in place: avoiding to retrieve content from the database every time.

What render:calltemplate really does

Probably the name of the tag itself is misleading: many developers thinks that calltemplate will actually always CALL a template every time. It actually does not always happen.

Calltemplate takes all the parameters  it was called with , order them in alphabetical order and build a string like c=xxx,cid=yyy,p=zzz,site=xxx,... (note the alphabetical order to make sure that you do not have cid=yyy,c=xxx different from c=xxx,cid=yyy when you have the same xxx and yyy))

This string is used as a key for the cache, then the cache is looked for. If a cached element is found, the cached output is returned. Otherwise, the referred template is invoked and the code executed. I cannot stress more that in general executing a template can be very expensive, performance wise. Getting attributes from the database may require a lot of queries, many of them involving a join (another expensive operation if you have large tables).

To get high performances, in general you should try to get zero calls to the database in 99% of the calls.

Each template has a list of parameters that will be used as "cache criteria". When you call the template, the list of parameters is checked against this cache criteria. If you call a template with a parameter that is not listed as a cache criteria a warning in the log is generated.

Please note that instead the render:callelement has a different behaviour: it is always invoked and it is usually used to invoke standardized html generation logic.

You do not have restrictions on the arguments you pass to a callelement, because its output will not generate a separate cached element: instead its output will cached as part of the calling template. Of course if it is database intensive then it will have bad performances.

What you cannot do in a cached template

Given this behaviour, there is a number of coding practice that are forbidden

First and before all, side effects. You cannot set a variable in the environment (something like ics.SetVar("flag", "something") and use it as an implicit parameter for a called template. Something that instead you usually have to do to use a callelement to retrieve informations. Side effects are actually the only way to get return values from an utility element.

Because the template can be cached, all the parameter must be explicitly stated, otherwise you may get the wrong template. The implicit paramente will not work unless the template is uncached (and this is not good for performances).

Another mistake is using a parameter that can change too often. For example, you may pass an uniquely generated id  to a template.

Even if this id is correctly added to the cache criteria, the problem here is that a different id will create a new cached element every time the template is invoked. If you have a site with millions of users, you will end up generating a new cached element for each request!

The result here is a that the cache can became very large. I have seen sites with id passedin absolute freedom growing the cache at the rate of one thousand elements per minute!!!

This mistake will affect performance, but even worse  publication. Because publication involves invalidating the cache,  with a cache out of control, invalidating may take hours…

The Simplest way to use Eclipse for Fatwire Development

Fatwire development has always been difficult. The recommended way of editing templates has been for a long time... ContentServerExplorer. I dislike ContentServer explorer for many reasons. For example:

1. it is a windows only app, and I use windows only in an emulator from MacOS or Linux
2. the editor embedded is so limited that notepad looks more advanced (for example from notepad you can search...)
3. the process of starting an external editor, then commit at every small change you want to test  is cumbersome

So it is natural that I has  always been looking for a replacement,  possibly using a real IDE like eclipse for development of the JSP. You know, completion, formatting, syntax checking, working syntax highlighting...

There are actually a couple of  Eclipse plugins for Fatwire Development. For a while, Fatclipse looked to be an interesting alternative. The plugin worked well and I was happy with it. Unfortunately, it is no more supported and it has not been updated to work with more recent version of Fatwire (currently it shows Fatwire 7.0.3 as the more recent supported version, and I was unable to run it in recent eclipse versions).

Then the CSDT plugins came out, but I am not happy with it. It shares the same checkout/commit philosophy of ContentServerExplorer, and is not well integrated in Eclipse. Basically it is a Java port of ContentServerExplorer, embeddable in Eclipse.

I wrote a blog post on this subject and I even developed my own alternative, FatStart.  I have now found a better way of doing development using Eclipse, so this post is basically my "adios" to FaStart, replaced  by a simpler and  better method described here.

The simpler and better way to develop with Eclipse is... not using any plugin! I discovered it is not strictly required, and you can do all your coding  in a practical and efficient way just using a recent eclipse (supporting folder linking) and a local install of Fatwire, either the JumpStartKit or your own installation using this technique.

A note of warning

The technique describe here is of course completely not official. Furthermore, in a shared environment it can be dangerous. This technique works if you do your Fatwire development in your own local instance in your machine, then you publish the changed templates in a shared enviroment.

However, if you still want to work in the old fashioned way, where multiple developers access to a shared development instance through http, your option is still ContentServer Explorer or CSDT (or even FatStart, although I do not support it anymore).

UPDATE: according Val Vakar, chief architect of CSDT, CSDT does not support shared development, although I was meaning that multiple developer can access to the same Fatwire instance and, as far as I know, it is possible, but I may be wrong.

How to directly edit the Fatwire JSP

Fatwire keeps templates as assets. So each template has some metadata, stored in the database, but the actual code is still  in a file inside a folder. This folder is located in  the Shared/elements folder. The location of the Shared folder depends on the installation. JumpStart places it under INSTALL-DIR/ContentServer/7.x.y/Shared.  In a custom install you decide where the Shared folder is.

So, if you create a typeless template named "Layout", usually you can find the actual code inside the folder Shared/elements/Typeless/Layout.jsp, while a typed template (for example Body for a Page) will be under Shared/elements/Page/Body.ksp   The actual name of the file is show in the template itself,  in the Element section. Fatwire for versioning purposes can change the name in something like Layout,0.jsp, Body,1.jsp and so on. So definitely you have to consult your template asset to know the actual file name.

So it is a pretty obvious idea trying to edit the JSP directly on the disk... However, if you do this in a default installation, you will have the surprise that... nothing happens (unless you restart the application servers). Your changes are apparently completely ignored by ContentServer.

So I thought that, because of some internal magic, direct editing of JSP is plainly not possibile... until I saw an important guy of the Fatwire team doing exactly this! Direct editing of the JSP under elements, and ContentServer actually immediately reflecting the change!

So I became a sort of detective and looked around to understand how  this is possible. An finally I found it,  actually where I expected: in a property of futuretense.ini: ft.filecheck

If you change this property to yes, you can simply edit JSP on the file system, reload your page and the result will reflect your page! Extremely agile and flexible.

So, now that I know I can actually edit JSP directly on the disk, after enabling this property all I need now is a good JSP editor.  For example eclipse. Here is how to configure Eclipse for Fatwire Editing in 5 simple steps.

Configuring Eclipse for editing templates

You need a recent version of eclipse, at least H(elios) that supports linked resources, and the Web Development Tools (so you can edit the JSP). Previous versions do not support resource linking, and  without linking, configuring the eclipse project became difficult and tricky.

Let's see how to do this step by step.

1. Create a new Dynamic Web Project

First step is to create a  new project, more precisely a Dynamic Web Project. You can create it in a workspace, it does not matter where you place it. You will actually refer to the JSP inside the elements folder through a link. So create this  project, and this is what you should see:

2. Link with the WEB-INF folder

Now, we need jar libraries and tag descriptors from Fatwire to enable JSP editing, tag library recognition, java completion and the like . Basically, all the stuff in the WEB-INF folder in the webapp part of your content server installation.

You do not need to copy them in the project (although you can do it if you like). Instead, as shown in the next image, all you need to do is to drag and drop the WEB-INF folder of the cs webapp in the WebContent folder, then LINK it.  See this image:

Please remember to select LINK  (not COPY) when eclipse asks you what to do.

3. Link with the elements folder

Now you can actually link the elements folder for direct editing. So just select the elements folder under the Shared directory and drag it into the root of the project as shown below.

Again, remember to LINK, not copy. Here you cannot copy, it would not work.

Now you can start to edit the JSP. That is all, except that eclipse will show a few errors. Next steps are here only to fix them, but if you can survive with a few errors showing up you are basically done.

4. Disable XML validation

The first error is related to XML validation. Many xml files in fatwire have still the <?XML prefix with a capital XML, and this is wrong for the Eclipse XML parser (strictly speaking, Eclipse is right, those "XML"s follow a draft xml standard, not the official one). So what you have to do is disabling the XML validation.  See the image below:

Open preferences, go to Validation and then disable XML Validator.

5. Hide a couple of JSP Directories

Some JSP present in Fatwire cannot be validated by Eclipse. Most notably, many contains a reference to the portlet jar, others have umbalanced HTML and so on. All those problems are in the system JSP that I am not generally interested in editing. So  I generally just  hide them from Eclipse eyes (and validation).

The simplest way is to go in Preferences|Resources and add an exclusion rule to hide the 2 folders "elements/OpenMarket" and "elements/fatwire". However, this system worked in Eclipse Indigo in Windows but did not work on Eclipse Helios on a Mac.

In this case all I did is to drag a void directory named fatwire and another void directory OpenMarket under elements. I created 2  new links to void directories  hiding the 2 folders creating the validation errors. .

BE CAREFUL HERE: if you drag a void directory over fatwire or OpenMarket, it once happened all the files inside were also removed! So I suggest you temporarily rename the 2 directory to another name, drag the void directory to create the hiding link and then restore them to the original name.

Final Result and Benefits

This is how  the project looks in my eclipse:

You may notice the 2 links pointing to elements and WEB-INF and the two links to hide the OpenMarket and fatwire folder. Combined with the trick described in the first section of this post, you can now can edit the JSP using the complete power of eclipse (including validation and completion) and immediately see any change reflected in Fatwire .

Download and install a development Fatwire Instance (also on Mac)

(article updated after the post)

Fatwire Content Server 7.6 is finally available for direct download from Oracle. You can find it  on http://edelivery.oracle.com . You need to login but  you can get an account just registering for free. Once accessed the site you have a huge list of options to choose with, but Fatwire is there.

Here:

You have to choose Oracle Fusion Middleware then Generic Platform. Then you wil get a long list of Fatwire Products; there are 15 of them including Community and Mobility Server, Analytics and Gadget Server.

The good news is that they are the real products. The great news is that now ContentServer is without the license check, so you do not need to get a license from Fatwire to install it. Legally, if you are an Oracle Partner, you can use the products for Development.

But after those good news, there is a bad news. Disappointing enough, there is not the Jump Start Kit in this list. It is available from the support however.

You may already know that Jump Start  Kit is a version of Fatwire pretty easy to install, that can run in your machine without a database. Actually runs with a database but it uses the embedded embedded Hypersonic SQL Database. HSQLDB is an in-memory database, certainly not good for enough production sites.  However, because Fatwire is  heavily cached application, actually using it with HSQLDB Fatwire runs  fast and reliably enough  for development purposes.

The whole database is also stored on a single text file, and it is easy to backup it: you just need to archive that file, while using Oracle you may need to take a database dump.

It is usually pretty inconvenient  having to run in your machine a full Oracle instance for development. Even if you have a 4 Gb machine, Fatwire, Oracle and Eclipse running together in the same machine are... a bit slow (to use a moderate term). So HSQLDB is definitely my choice to get a local instance for development.

The rest of this post describes how to install the real Fatwire using HSQLDB like Jump Start Kit.

Why Install Fatwire with HSQLDB?

The interesting news is that is possible to install the real ContentServer 7.6 using HSQLDB instead of using the Jump Start Kit, although you may resort in using a small trick.

Please note that what follows is by no means an officially supported configuration.  I have used however in large projects and it works pretty well.

I have never faced any major problem using HSQLDB instead of a real database as long as you use the same version that is used in the JumpStart.

I install the real Fatwire for development mostly because there are no hot fixes available for Jump Start Kit. So if you use your local Fatwire instance for development and another Fatwire for production, you may have to update the production instance without having the possibility to update your development instance.

Indeed having development and production instances out of sync can became a major problem, since the instances must match in version number for publishing.

Another problem is the configuration installation. Sometimes I need to configure Fatwire without Engage, while it is always installed with Engage in JumpStart Kit, with no (easy) way of getting rid of it. Also there are certain demos available in the full fatwire that are not in the JumpStart Kit.

Last but not least, the layout. I really hate the layout offered by JumpStartKit. My preference is to install both the fatwire and shared folder under the tomcat directory, and the database folder under the shared folder.

This way you can distribuite the whole thing (including the tomcat and the database) just zipping the tomcat folder. And you can take a backup of the system  just zipping the shared folder, if you have the database inside the shared folder, as I usually do.

Installing Fatwire with HSQLDB

If you try to install Fatwire in your local machine using the provided installer downloaded from Oracle, r you will see that the installer offers as an installation option one of Oracle, SQL Server or DB2. So it looks like you are required to install it only with a full fledged database even, if you are installing only a development instance in your machine.

I have actually tried to install it using either Oracle XE and SQL Server Express, and it works. So you do not have to use the full Oracle instance, you can use a more lightweight version. However I have found you can actually install the full version with HSQLDB!

I have done some tests, and I was fully successful using HSQLDB 1.8.0 (the version actually used by the JumpStart) but I had some problems using version 1.8.1 . I have not tried newer version like 2.x. You will see that there is MySQL as an option but so far I was not successful, I have no idea which MySQL version is actually supported.

Here is a step by step guide how to install the full Fatwire using Tomcat and HSQLDB.

1. Download and configure a Tomcat 6.0.x from the Apache website.

2. Download HSQLDB 1.8.0 from SourceForge

3. Place hsqldb.jar  in the folder lib of the Apache Directory

4. Edit the file conf/server.xml and add this configuration (change /path/to/data to the folder where you will place the database)

<Context path="/cs" docBase="cs"
  reloadable="true"
  crossContext="true">
<Resource name="csDataSource"
  auth="Container" type="javax.sql.DataSource"
  maxActive="50" maxIdle="10" username="sa" password=""
  driverClassName="org.hsqldb.jdbcDriver"
  url="jdbc:hsqldb:/path/to/data/csDB"/>
</Context>

4. Start the installation of ContentServer download, unzipping it, running  csInstall.sh or csInstall.bat. Now answer to all the questions until you get to the  database choice pulldown. Pick whatever you want (Oracle is fine), and go on.

5. Go on with  the installation until you get  reach the point where the installer asks you to start the application server, as in the following image:

6. Now it comes the trick: click on the button Property Editor, wait for the property editor to  comes up, select Options then Change Database, and you will see the following screen showing up as in the following image:

Here you can change the database. Please note that this will only change the type of database used by Fatwire, but you have to manually configure the datasource csDataSource in the application server. That is what you did in step 4, so Fatwire should work.

7. Now you can  save the configuration file, start the application server and complete the installation. If you have done it correctly now Fatwire should be up and running using HSQLDB.

Install Fatwire on a Mac as a Development Machine

So far, the technique described  works on Windows and Linux. If you try to run the installer on a Mac however, you will  incur in  the surprise that you cannot select an application server. It looks like the configuration system does not detect a compatible application server on a OSX platform.

However, it is all just Java, Tomcat and HSQLDB works reliably on the Mac Java Virtual Machine and there is no reason because you should not run it on a Mac. It actually works.

The limitation of the installer is annoying but however it is easily overcome: just pretend on the Mac you are running on linux.

That is easy enough: edit the script csInstall.sh and add into the last line, just after the command $JAVA:

-Dos.name=Linux

That's it. This way the installer thinks you are on Linux and will let you to choose Tomcat as an application server. I have completed the installation on a Mac a number of times and it works.

A new FatGoodie: FatProject

Yesterday I had to create quickly a new Fatwire Project. This time, since I have CS 7.6 I decided to try CSDT (you know, the new Eclipse Plugin for eclipse development). But...

What is wrong with CSDT

Well, the experience with CSDT did not live  up to my expectation. Don't get me wrong: I think that CSDT is basically fine. However, it is not what I was looking for.

Here a few  things I don't like.

First:  it stores content in his own directory, that is NOT the Eclipse project directory. So I cannot really version control it, unless I create a separate repository and use the directory where CSDT stores files. While it is doable, it is a bit annoying (you wonder: why???)

Second annoyance: I configure it once using my local jump start. Now I am unable to change settings.  I want to use indeed a different Content Server (accessible through a VPN). With CSDT once you have configured it, it looks like you cannot change that configuration anymore, unless you create a new workspace. Not sure why, but definitely annoying.

Last thing, I have to manually publish changes once in a while. While definitely not a big deal, it is annoying nonetheless, especially if I have the habit to see immediately  the changes once I hit save in eclipse.

FatProject was born

So, after this frustation I decided to come back to my FatStart. Well, actually I just picked the ElementDeployer, since I do not need the complete packaging power of FatStart. I only was looking for the ability  to use eclipse to edit and  deploying elements in the quickest possible way. Basically an eclipse replacement for ContentServer Explorer, more efficient that CSDT for this basic task (actually CSDT performs other tasks but I simply don't need them).

All I had to do is to create an eclipse project, copy in it the ElementDeployer (part of FatStart) and configure a link to the cs/WEB-INF directory. I also added a launcher script to perform deployments manually.

So basically I created a bare-bone eclipse project supporting ContentServer and  able to edit jsp (with completion!) and deploy code changes when I save it.

I thought that this personal stuff can be useful for others, so I collected the eclipse project and I released it in my FatGoodies collections as "fatproject":

How to use FatProject

Checkout the code from here:

https://github.com/sciabarra/FatGoodies/tree/master/fatproject

Then import the project  in Eclipse. If you  have CSDT already configured, that is all; otherwise you will have to set up the eclipse variable "CS_CONTEXTDIR" to point to your local jumpstart "cs" directory.

Now, create your elements and templates. You have to follow a couple of conventions for naming them (see the doc). Then  you can just launch the ElementDeployer class.

It will look for an ElementDeployer.prp file to read its settings then it will  publish jsp (with extension .jspf please) to the Content Server you configured.

Then keep it running. Every time you change a file it is immediataly redeployed.

This is basically all I need to code with Fatwire if I don't have to distribute a complete package (for which FatStart is better suited).

Using this simple FatProject I am in a productive loop: I edit the content model in Fatwire and I save it, then I edit the code in Eclipse and I save it, and I am done: just reloading the page (well, don't forget to disable or clean the cache, anyway)  I can see the results of my changes.

And I am happy!

(It took more time to write this blog post than to build fatproject actually)

Getting the Parent Page of an Asset

Problem: you need to know where an asset has been placed in the SitePlan.

Well, I have a similar situation. I designed a site in a such way that the css of an asset is picked looking at  the category of the parent page.

This is not odd: when you navigate a site and reach a page with a given style (that changes according to the sections of the site), if  you  navigate to the detail , usually what  you want is children keep the same style of the parent page.

So, basically I want to locate the parent page (or at least the first parent page, since I know the asset can be actually placed in more than one page).

However, that is the problem. What is the solution?

So, how do you think you can find the parent? The simple solution is:  enumerating all the pages, then check all the children until you find your asset. Well, I am sure you will agree if I say this solution is horrible, both performance wise and coding wise.

Fortunately I found is a better solution that I am posting here. This post is also a  good opportunity to introduce an advanced Content Server techniques as well: the TreeManager.

The solution is basically looking into the table AssetRelationTree (that stores  all the  parent-children relationship) using the TreeManager (an internal server of ContentServerm  accessible with a tag  available in the tag libriry).

How to find the parent pages of a given asset

The problem,  expressed in a more technical way, is: given an asset identified by his content id (stored in the "cid" variable),  find the id of all the pages having that asset as a children. My Solution follows.

Step 1: locate a list of all the children assets in the AssetRelationTree with the TreeManager

<ics:treemanager>
  <ics:argument name="ftcmd" value="findnode"/>
  <ics:argument name="treename" value="AssetRelationTree"/>
  <ics:argument name="where" value="oid"/>
  <ics:argument name="oid" value='<%=ics.GetVar("cid")%>'/>
</ics:treemanager>

Step 2: extract parents using their node id (nid) and getparent

The returned list is only a list of the available relations.
Using the nid of such a relation you can look for the parent.

This is the code to retrieve the parents:

<ics:copylist from="AssetRelationTree"  to="List"/>
<ics:listloop listname="List">
  <ics:listget listname="List" fieldname="nid"  output="nid" />
  <ics:treemanager>
    <ics:argument name="ftcmd" value="getparent"/>
    <ics:argument name="treename" value="AssetRelationTree"/>
    <ics:argument name="node" value='<%=ics.GetVar("nid")%>'/>
  </ics:treemanager>
  <!-- now do something with the found id - see Step3 -->
</ics:listlooop>

Note that I copied the list from  the first TreeManager call because another call to TreeManager will  store the result always in the same  AssetRelationTree list. Since I have to call the TreeManager again I am preventing the list to be overwritten and I am working on a copy.

Step 3: grab parents and do something with them.

Inside the loop in Step2, you will get all the parents of the asset.

You can retrive the id of a parent with

<ics:listget listname="AssetRelationTree" fieldname="oid" output="p"/>

also you can read the type with

<ics:listget listname="AssetRelationTree" fieldname="otype" output="ptype"/>

but it should be always "Page". Now you can do what you want with the parent (in my case, I was just using the retrieved parent to set the p parameter).

Bonus: how to do it all with a simple query

Well, you could retrieve the parent with this simple database query, tested with jumpstart and hypersonic:

select  otype,oid from AssetRelationTree where nid in
(select nparentid from AssetRelationTree where oid = 123456789)

where 123456789 is the cid of the asset you are looking for its parents.

However, as a general rule, I do not trust too much explicit queries because they are database dependent.

Customizing Icons in TreeTabs

I am deploying for a customer a shiny new content management site based on Fatwire;  I am using some custom treetabs for easily access content.

I actually created a new Flex Family, let's say fpContent, then I created a new TreeTab displaying content and parent (let's say fp_C and fp_P), as follows:

Unfortunately, after I created some content, the result  is  not very pleasant:

The biggest problem for an end user is that the icons are not meaningful. It is impossible to distinguish containers (parents) from children.  If you give a look to FirstSiteII, it doesn't have this problem, since parents are rendered as folders, and children as documents.

Fortunately, you can actually customize icons for the TreeTabs. Read on.

How to customize icons for TreeTabs

In your cs webapp folder, look inside c s/Xcelerate/OMTree/TreeImages/AssetTypes.  You will see there, for each asset type rendered in a tree tab,  a corresponding image named after the type plus a ".gif" extension.

So I tried this:

$ cp Product_P.gif fp_P.gif
$ cp Document_C.gif fp_C.gif

Then I  restarted the browser (step required to clean the internal cache of the applet displaying the treetab), et voilà: you get your icons.

Nice,  isn't it?

Differentiating icons for different Flex Definitions

After this result, I became bold and decided to use different icons for different contents. Actually, since I am using a FlexFamily, I want different icons for different Flex Definitions.  This is a bit more trickier.

The biggest problem is that in a TreeTab you need to associate an icon to an asset type. But for a flex family, each content definitions is  actually a subtype!!!

The trick is:

• create a new type
• associate the subtype to type when you create the asset of that type (with the New Start Menu Entry)
• finally, you can render the type in the treetab

So, let's do it, step by step.

Create a new type for the flex famiy

In the Admin tab there is the Flex Family Maker. It  allows the creation of new types for each part of a flex family.

So select Add a New Flex Asset Type, called fpText as in the following snapshot:

Now, the fpText is a new type available for use.  First, you have now to enable the type for the site.

Go to the Site tab, and enable the new available type fpText.

WARNING. You will be offered to automatically enable 2 start menu entries. DON'T DO THIS. Disable the check boxes and carry on.

Be careful here: the Start Menu entries created when you enable the type are generic, so you can associate to that type any subtype. This is NOT what you normally want.

Create the Start Menu

Now, go on the Start Menu section in the Admin tab, click on "New Start Menu", then associate to the type the wanted subtype.

This is the relevant screen that will allow you to select the correct Flex definition:

Render the content in the TreeTab

The reason of all of this work is because  in a TreeTab you can only specify types.

Now, go on the TreeTab editor and add the type "fpText" to the types rendered by that TreeTab.

Finally you can go back to the AssetTypes folder of the applet (the one I mentioned before), and do:

$ cp Article.gif fpText.gif

Now, restart the browser and finally you can see a new shiny icon for the created new type (do you see the icon associated to "Test"?).

That's all.