Connecting dispatchers and publishers

How should I connect my dispatchers with the publishs? 1:1, 1:n or m:n?
To give you an impression how these scenarios could look like I graphed the 1:1 and the n:m scenario.

The 1:1 setup, where each dispatcher is connected to exactly 1 publish instance; for the invalidation every publish is also connected only with its assigned dispatcher instance.

The n:m setup, where n dispatcher connce to m publish instances (for illustration here with n=3 and m=3); each dispatch is connected via loadbalancer to each publish instance, but each publish instance needs to invalidate all dispatcher caches.

I want to give you my personal opinion and answer to it. You might get other answers, both from Adobe consultants and other specialists outside of Adobe. They all are valuable insights into the question, how it’s done best in your case. Because it’s your case which matters.

My general answer to this question is: Use a 1:1 connection for these reasons:

• it’s easy to debug
• it’s easy to monitor
• does not require any additional hardware or configuration

From an high-availability point of view this approach seems to have a huge drawback: When either the dispatcher or the publish instance fails, the other part is not available as well.

Before we discuss this, let me state some facts, which I consider as basic and foundation to all my arguments here:

• The dispatcher and the web server (I can only speak for Apache HTTPD and its derivates, sorry IIS!) are incredibly stable. In the last 9 years I’ve setup and operated a good number of web environments and I’ve never seen a crashing web server nor a crashing dispatcher module. As long as noone stops the process, this beast is handling requests.
• A webserver (and the dispatcher) is capable to deliver thousands of requests per second, if these files originate from the local disks and just need to be delivered. That’s at least 10 times the number any publish can handle.
• If you look for the bottleneck in handling HTTP requests in your AEM architecture it’s always the publish application layer. Which is exactly the reason why there is a caching layer (the dispatcher) in front of it.
• My assumption is, that a web server on modern hardware (and operating systems) is able to deliver static files with a bandwidth of more than 500 mbit per second (at a mixed file scenario). So in most cases before you reach the limit of your web servers, you reach the limit of your internet connection. Please note, that this number is just a rough guess (depending on many other factors).

Based on these assumptions, let’s consider these scenarios in a 1:1 setup:

• When the publish instance fails, the dispatcher instance isn’t fully operational anymore, as it does not reach its renderer instance anymore; so it’s best to take it out of the load balancing pool.
  So does this have any effect on the performance capabilities of your architecture? Of course it has, it reduces your ability to deliver static files from the dispatcher cache. Which we could avoid if we had the dispatcher connected to other publishs as well. But as stated above, the delivery performance of static files isn’t a bottle neck at all, so when we take out 1 web server you don’t see any effect.
• A webserve/dispatcher fails, and the connected publish instance is not reachable anymore, effectively reducing the power your bottleneck even more.
  Admitted, that’s true; but as stated above, I’ve rarely seen a crashed web server; so this case is mostly true in case of hardware problems or massive misconfigurations.

So, your have an measurable impact only in case that a web server hardware went down, in all other cases it’s not a problem for the performance.

This is a small drawbacks, but from my point of view the other benefits stated above outweigh it by far.

This is my standard answer, when there’s no more specific information available. It’s a good rule of thumb. But if you have more specific requirement, it might have sense to change the 1:1 rule to a different one.

For example:

• You plan to have 20 publish instances. Then it doesn’t make sense to have 20 webserver/dispatchers as well.
• You want to serve a huge amount of static data (e.g. 100 TB of static assets), so your n copies of the same file get’s expensive in terms of disk space.

If you choose a different approach than the 1:1 scenario described in this blog post, please keep these factors in mind:

• How do you plan to invalidate the dispatcher caches? Which publish instance will invalidate which dispatcher cache?
• How do you plan to do maintenance of the publish instances?
• What’s the effort to add or remove a new publish instance? What’s need to be changed?

Before you plan to spend a lot of time and effort into building a complex dispatcher scenario, please think if a CDN isn’t a more appropriate solution to your problem…

 

How to remove lock files in CQ / WEM

How to remove lock files in CQ / WEM 

Problem :

javax.jcr.RepositoryException: The repository home /crx-quickstart/repository appears to be in use since the file named .lock is locked by another process.
at org.apache.jackrabbit.core.util.RepositoryLock.tryLock(RepositoryLock.java:166)
at org.apache.jackrabbit.core.util.RepositoryLock.acquire(RepositoryLock.java:138)
at org.apache.jackrabbit.core.RepositoryImpl.(RepositoryImpl.java:276)
at com.day.crx.core.CRXRepositoryImpl.(CRXRepositoryImpl.java:233)
at com.day.crx.core.CRXRepositoryImpl.(CRXRepositoryImpl.java:288)
at com.day.crx.core.CRXRepositoryImpl.create(CRXRepositoryImpl.java:193)
at com.day.crx.j2ee.CRXRepositoryStartupServlet.createRepository(CRXRepositoryStartupServlet.java:173)

Reason why above happen, If you forcefully shut down CQ sometimes .lock files does not get deleted.

Solution :

You can use following command in Linux/ Unix platform to delete all lock files

find <Path to crx-quickstart>/repository -name "\.lock" -exec rm '{}' \; -print

AEM Best Practice: OSGi Configurations

one of the best practices I’ve come across recently in AEM is to never use the OSGi console directly to edit configurations. People may argue “But that’s what it’s there for!”, and indeed it is an easy interface to use for configuration changes. However there are drawbacks to managing your configuration changes this way in a AEM project. 

Let’s demonstrate this by changing the configuration of the Apache Sling GET Servlet.



When you edit a configuration in the OSGi console and save it, the updated configuration is saved to the JCR, generally under /apps/sling/config or /apps/system/config depending on the component you configure. First thing to note here is that you have no control over where this node is saved to, or if this configuration should be for a specific runmode – the parent folder is called config, not config.author or config.publish (for more information on runmodes, see How to set up run mode and my previous post Confirming the Runmode).



Secondly, and more importantly, the configuration node that has been created is of type nt:file. Therefore it contains a child called jcr:content, which has a property called jcr:data which contains the binary contents of the file. So as you can see (or more accurately NOT see), the details of the configuration itself are not readily apparent. When viewing this binary, we see the following:

aliases=”xml:pdf”
index=B”false”
enable.txt=B”false”
index.files=[“index”,”index.html”]
enable.html=B”false”
enable.json=B”true”
enable.xml=B”true”
json.maximumresults=I”600″

Why is this bad? What happens when you want to change a single configuration value here programatically, or even through CRXDE lite, CRX Explorer, or curl for that matter? You would need to write code that extracts the binary contents of the file, manipulate this content to adjust the values to the desired configuration, then write this back to the jcr:data property of the file contents. This is quite messy, if all you want to do is change a single value of a complex configuration and leave the rest the same.

So what’s the alternative? Create the configuration node manually, and use properties for each of the configuration settings. The configuration screen in the OSGi console gives you virtually all the information you need to create this node yourself. In addition, you also have the ability to choose the location of these settings, and therefore you can store them under you project branch, and also control the runmode in which they are used. This can be done in 3 simple steps:

1 – Create the folder structure

Create folders /apps/myproject/config.author and /apps/myproject/config.publish. Make sure the config.author and config.publish folders are of type sling:Folder

2 – Create the configuration node

Create a node of type sling:OsgiConfig under the runtime folder you wish to configure. The name of the node should be the same name as the Persistent Identity (PID) of the configuration in the OSGi Console. HINT: If this is a factory class and can have multiple instances, use the Factory Persistent Identifier (Factory PID) and append the name with ‘-xxxxx’, where xxxxx is a unique identifier.

3 – Add the names and values from the OSGi configuration as properties

Using the label in brackets at the end of the configuration description as the property name, and the value of the configuration as the values of the properties.

That is it! Now, if you change the values of the configuration nodes’ properties, you can refresh the OSGi console and see these changes reflected to prove the configuration is taking effect. So from now on, whenever you need to change a configuration, you can make the change directly in the JCR, as well as easily make any configuration changes via curl or java, without needing to deal with file binaries.

 

CQ5 : Nothing Showing in CRXDE Lite

 

When working on a CQ5 project for a client that involves multiple development environments, I came across an issue on some of the dev servers. For most of them, logging into CRXDE Lite gave me the full tree of the JCR repository. On a couple of others, CRXDE Lite displayed nothing more than the root node.



Viewing the error.log gave me this error:
28.01.2013 23:02:35.405 *INFO* [10.73.64.4 [1359414155401] GET /crx/server/crx.default/jcr%3aroot/.1.json HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Resource /crx/server/crx.default/jcr:root/.1.json not found

After asking around, I discovered that CRXDE Lite used WebDAV in order to communicate with the JCR. After checking the CQ5 OSGi console, I saw that the following bundles were not running:

Apache Sling DavEx Access to repositories
Apache Sling Simple WebDAV Access to repositories

After starting these two bundles and refreshing CRXDE Lite, all was well again.

CRXDE Lite code formatting

One thing I always missed in CRXDE Lite was the lack of any code formatting tools. I always had to copy the code out into Notepad++ or Eclipse to format the code and then paste back into my JSP files. Not very convenient.
Today I accidentally stumbled across what I expect is an undocumented feature in CRXDE Lite – Code Formatting!

Shift-Tab

If you select any or all of the code on a page, and then hit Shift-Tab, it doesn’t tab the code back one tab stop (as you would expect), it actually formats the code.

Before

After

Version

This was done with CRXDE Lite in CRX 2.4.23 (CQ 5.6.0.20130125) that comes with Adobe Livecycle ES4.  It seems to work in Chrome, Firefox and Intenet Exploder 10.

 

CQ5 Tutorial: Customising the Login UI

When working a CQ job for a client, it’s always nice to personalise the login screen for them. The following is a guide on how to make simple customisations of the login UI, leaving the underlying logic behind it intact.

The login UI code is located in the JCR at /libs/cq/core/content/login. As with any general overriding, we need to copy the parts we want to update from the /libs folder to the /apps folder. Generate the folder structure /apps/cq/core/content. If you take note of these folders under /libs, you’ll notice that the cq and core folders are of type nt:folder, while the content folder is of type sling:orderedFolder. To avoid complications, it’s best to keep these folder types intact when recreating the structure in the /apps directory. Also note, that the core folder has a property of jcr:mixinTypes, type Name[], value rep:AccessControllable.

Once the folder structure is created, copy the login folder from /libs/cq/core/content/login to /apps/cq/core/content/login. Open the new login folder, and you should be able to see how the styling of the UI is put together – through css, images and a little javascript. As an example, to change the main background image, overwrite the background.png image in the login/bg folder. Clear your cache and check out the result in the browser.

Hmmmmm not what we expected! Notice in the address bar that the URL still points to /libs. Open a browser debugging tool and have a look at the traffic. Everything is still being served from /libs, including our ‘new’ image, but now the css is being served from /apps, but it’s forbidden!

Luckily, there’s an OSGi setting that defines where the default login page is located. Log into the OSGi console, select the Configuration tab, and open the Day CQ Login Selector Authentication Handler. Update the Default Login Page setting from /libs/cq/core/content/login to /apps/cq/core/content/login and save. Have a look at the change in the browser (after clearing your cache).

OK, the css is back to normal, but now our new image is missing. Note the URL in the address bar though – we’re now serving the login from /apps as we want. But if you look at the web traffic, you see that all of the resources are coming from /apps, except backgroundimage.png which is still coming from /libs.

If you take a look at the file /libs/cq/core/components/login/login.jsp and search for backgroundimage.png, you’ll see the cause – the path to the resource has been hard coded:
Resource bgImg = resourceResolver.getResource(“/libs/cq/core/content/login/bg/background.png”);

Now there’s a couple of ways around this. We could copy the /libs/cq/core/components/login folder to /apps/cq/core/components/login like we did for the content and change the value there. But this would mean that we also need to maintain the other logic that is in the component. If the component is updated in a service pack etc, we would not automatically get the update, since we’re using our own custom version of the login component. Luckily, Sling provides a way around this, and lets us achieve our goal via another setting in the OSGi console.

On the Configuration tab, open the Apache Sling JCR Resource Resolver, and add the following to the URL Mappings:
/libs/cq/core/content/login/</apps/cq/core/content/login/

What this does is say any path in the first part (/libs/cq/core/content/login/), if it’s an outgoing mapping (<), map it to the second path (/apps/cq/core/content/login/). Since the code in the login.jsp component is an outgoing mapping, this will force the resource to be found in /apps instead of /libs. Save the configuration, clear your browser cache, and view the login page again.




All going well, you should see your new image on the login screen, and all other css and images are being served from /apps instead of /libs. Now you can move forward and change the other images and styles to suit the branding you desire.

CQ/AEM node operations- add,delete,Recursively fetching the nodes

CQ/AEM node operations- add,delete,Recursively fetching the nodes

Delete a node

//Below code delete 'textRenderer' node if present under /content/projectname

private static final String TEXT_RENDERER_NODE_URL = "/content/projectname";
rendererNode = currentSession.getNode(TEXT_RENDERER_NODE_URL);
                                    
                                    
                                      if(rendererNode.hasNode("textRenderer"))
                                      {
                                          Node delNode = currentSession.getNode(TEXT_RENDERER_NODE_URL + "/textRenderer");
                                          delNode.remove();   
                                          currentSession.save();
                                      }
                                
Ensure to save the session after any node modification.


Add a node
//Below code adds 'textRenderer' node of type 'cq:Page' under /content/projectname

private static final String TEXT_RENDERER_NODE_URL = "/content/projectname";

Node fileNode = rendererNode.addNode("textRenderer", "cq:Page"); 
fileNode.addMixin("rep:AccessControllable");
currentSession.save();
                                    
                                    
                                    =======================================
                                    
Recursively fetching the nodes under CQ node

Below code recursively iterates over a root node and populate the node attributes.

// Node- the root node to be iterated
// Session - current session
 public static void visitRecursively(Node node, Session currentSession) {
                        
                                        
                    try{
                            
                        
                    // get all child nodes
                          NodeIterator list = node.getNodes();
                                 
                          while(list.hasNext())   {
                           
                  // get child node
                             
                          Node childNode = list.nextNode();
                          // Verify child node for cqPage type
                          if((childNode.hasProperty("jcr:primaryType")) && (childNode.getProperty("jcr:primaryType").getValue().getString()).equals("cq:Page") ){
                            
                            Node jcrNode = childNode.getNode("jcr:content");
                              
                            // Iterate some of the page properties
                            String articleTitle="";String jcrDesc="";String jcrTitle="";String keywords="";
                            if(jcrNode.hasProperty("articleTitle")){
                                
                                articleTitle = jcrNode.getProperty("articleTitle").getString();
                                log.info("articleTitle--->"+articleTitle);
                            }
                            if(jcrNode.hasProperty("jcr:description")){
                                
                                jcrDesc = jcrNode.getProperty("jcr:description").getString();
                                log.info("jcr:description--->"+jcrDesc);
                            }
                            if(jcrNode.hasProperty("jcr:title")){
                                
                                jcrTitle = jcrNode.getProperty("jcr:title").getString();
                                log.info("jcr:title--->"+jcrTitle);
                            }
                            if(jcrNode.hasProperty("keywords")){
                                
                                keywords = jcrNode.getProperty("keywords").getString();
                                log.info("keywords--->"+keywords);
                            }
                             
                            String pagePropertiesString = "articleTitle--->"+articleTitle + "jcr:description--->"+jcrDesc+"jcr:title--->"+jcrTitle + "keywords--->"+keywords ;
                        
                        
                          log.info("Page Properties :---> Node "+ childNode.getName()+ "Properties : " + pagePropertiesString );
                          
                        
                          }
                             
                          visitRecursively(childNode,currentSession);
                          }
                          }
                          catch (RepositoryException rpe){
                              log.info("Exception in recursive listing:");
                          }
                                 
                }

Date Formatting with Sightly

Now that more and more peo­ple doing Sightly imple­men­ta­tions an often heard ques­tion is how to do Date-formatting in Sightly.

In this small blog­post I will show you a way to achieve that.

 In the exam­ple below I will for­mat the last-modified date of the page, dis­play­ing it in a for­mat of dd/MM/yyyy.

I use Sling mod­els for this together with the Sim­ple­Date­For­mat class.

<div data-sly-call="${ dateFormat @ date=currentPage.lastModified, dateFormat='dd/MM/yyyy' }"></div>

<template data-sly-template.dateFormat="${ @ date, dateFormat }">
<div data-sly-use.formatter="${'com.yourproject.DateFormatting' @ date=date,dateFormat=dateFormat}">
Formatted value : ${formatter.formattedValue}
</div>
</template>

With data-sly-call I am ‘invok­ing’ the tem­plate defined with the name ‘dateFormat’.

The tem­plate receives two para­me­ters, date and date­For­mat. In the tem­plate itself a Java-class is used, and that Java-class gets two para­me­ters date and dateFormat.

From the Java-class the value ‘for­mat­ted­Value’ is avail­able and it printed.

The code of the Java-class is avail­able here.

@Model(adaptables=SlingHttpServletRequest.class)
public class DateFormatting {
 
  @Inject
  private Calendar date;
 
  @Inject
  private String dateFormat;
 
  public String formattedValue;
 
  @PostConstruct
  protected void init() {
    SimpleDateFormat formatter = new SimpleDateFormat(dateFormat);
    formattedValue = formatter.format(date.getTime());
  }
}

The code in the Java-class is quite sim­ple because date and date­For­mat get injected auto­mat­i­cally. There­fore the only ‘real’ code you need to do is only to call the format()-method.

Hope­fully this gives you an idea how to do for­mat­ting with Sightly.

4 Ways to Try Javascript Use-API With Sightly

Here’s four exam­ples on how you can use the Javascript Use-API. You can already find some other exam­ples and doc­u­men­ta­tion on this Sightly doc-page.

Hello World

Let’s start with the ‘Hello World’ exam­ple, to get a basic idea how the Use-API works.

I have this in my Sightly-component:

<div data-sly-use.hello="helloworld.js">
 ${hello.world}
</div>

And this is my helloworld.js

"use strict";

use( function () {
  var hello = "Hello World";
  return {
        // world can be used inside your component
        world: hello
    };
});

I hope this exam­ple is quite self-explanatory. ${hello.world} works because we define ‘hello’ in the use-api data-sly-use.hello, and ‘world’ is defined in the helloworld.js file.

Using pre-defined objects

Here an exam­ple on how to use pre-defined objects within your Sightly-JS file. In this sam­ple I am sim­ply out­putting some val­ues in order to show you how to use the pre-defined objects.

This is in my Sightly component:

<div data-sly-use.predefined="predefined.js">
 ${predefined.pageName}, 
 ${predefined.title}, 
 ${predefined.resourceName}, 
 ${predefined.resourceTitle}
</div>

This is my predefined.js file:

"use strict";

use( function () {
 
 var pageName = currentPage.name;
 var title = currentPage.properties.get("jcr:title");
 var resourceName = granite.resource.name;
 var resourceTitle = properties.get("jcr:title");
 
    return {
        pageName: pageName,
        title: title,
        resourceName : resourceName,
        resourceTitle : resourceTitle
    };
});

Pass­ing argu­ments to the Sightly-JS file

In this exam­ple I will pass on some para­me­ters to the Sightly-JS file.

This is my Sightly component:

<div data-sly-use.params="${ 'parameters.js' @ value1='feike', value2='visser', seperator=' '}">
 ${params.newValue}
</div>

Please note the ${ ” } nota­tion to define the Use-API.

This is in my Sightly JS-script:

"use strict";

use( function () {
 
     // you can reference the parameters via this.
 var retValue = this.value1 + this.seperator + this.value2;
 
    return {
        newValue: retValue
    };
});

Pass­ing in Java-backed objects in your JS-files

In this last exam­ple I am pass­ing in a Java-backed object as a para­me­ter and call a method from that. Here I am using the xss­API, and call­ing the method encodeForXML().

This is my Sightly component:

<div data-sly-unwrap data-sly-use.encode="${ 'xmlEncoding.js' @ xssApi = xssApi, valueToEncode='hello world'}">
${encode.encodedValue}
</div>

This is my Sightly JS-file:

"use strict";

use( function () {
    return {
        encodedValue: this.xssApi.encodeForXML(this.valueToEncode)
    };
});

You can see how I access the xss­API via this.xssApi and then call a method from it.

I hope you found these exam­ples use­ful and that they allow you to start using the Sightly JS-Use API. Let me know how it goes!

Adding Packages into the AEM Jar-file

let’s cover how to auto­mat­i­cally install your own pack­ages when you install AEM6 for the first time. This is a handy way to avoid large man­u­als (that we don’t read any­way  ), and espe­cially for mak­ing sure the devel­op­ment team is in sync with content-packages and frame­work releases

Pre-req
What you need to have is the orig­i­nal aem6 jar-file, and the pack­ages that you want to install.
In this exam­ple I want to install the SP1-package, and pre-package that into the aem6 jar-file.

I have this file-structure

/aem6_builder
  - aem6.0.jar
    /static/install
    - aem_60_sp1.jar

Now exe­cute the fol­low­ing com­mand in /aem6_builder:

jar uvf aem-6.0.jar static/install/aem_60_sp1.zip 

You get an out­put like this, mean­ing that the sp1-file is now merged into the aem6 jar-file:

adding: static/install/aem_60_sp1.zip(in = 233453836) (out= 233522386)(deflated 0%)

At this point you sim­ple hand-out this file to the devel­op­ment team to be sure they all have SP1 installed.
If you want you want extend this, to also install other pack­ages with con­tent or apps.

Happy installing!