The "Sleeping beauty"-effect in agile software architecture

Agile projects, above all those with multiple Scrum teams, need a moderated architecture process, in order to facilitate architectural decisions. It is advantageous to decide as late as possible, nevertheless the correct point in time should be chosen intentionally. If not, important architecture decisions could be overlooked or made too late.

The promise

In the ideal agile world, architecture grows by evolution (“no big upfront design”). An architecture will not be sketched out with great detail before development starts, but it will be created instead step by step during the work in the sprints. Decisions will be taken when triggered by work on a current user story. Until then, the team has time to gather know-how which will come in handy for evaluating architectural decisions. Since the “big upfront design” doesn’t have to be adjusted along the way, requirements can be accepted more easily und can be integrated into development more quickly. As a consequence, risk will be lower and efforts will decrease – that’s the hope, at least. Even the quality should increase, because decisions can be made with more experience. All this is based on the implicit assumption, that all knowledge needed to take the decision will be accessible for all persons involved at the correct point in time. 

The reality

This assumption is – as we will see – incorrect, which leads to some negative effects. Some preliminary work on the architecture already helps with the beginning of an agile project. The question, which part of the system the scrum team should work on can only be addressed sensibly, when there is a rough breakdown of the “system”.

Basically, it’s not so bad an idea to decide late. In fact, there are many architecture questions which can be postponed. E.g. you don’t need to decide on a browser, when you don’t work with browser specific functions. You don’t need to talk about implementation details of microservices, which will be accessed via webservice API. To decide late indeed means to decide with greater knowledge and therefore, to decide with greater quality. Thus, the focus can be shifted to the current and immediate problems.

Picture 1: Sleeping beauty phase

Yet, this also is the catch: the “immediate” problems tend to push the postponed decisions from the agenda. They enter a “sleeping beauty” mode, and waking them up is hard. Somewhere in the project wiki they are documented, but who keeps track? All of a sudden the team faces the problem, that a new component has to be implemented in the next sprint – but the decision about its interior technology has not been taken. Important business information and conceptual information is missing, which is necessary to make the decision, but which does take some time to acquire. As we see, the assumption of the “always available information” is not true.

Due to the „sleeping beauty“ effect, the project gets late and misses its schedule – which could be prevented by some preliminary work. The same effect also casts its shadow on the hopes for better quality, since decisions taken under time pressure tend to be suboptimal. Furthermore, non-functional requirements also fall prey to that effect, which can lead to even more ugly consequences.

Best practices

But there is an effective method against the „sleeping beauty“-effect: somebody has to watch the “sleeping” decisions and has to ensure, that necessary knowledge is acquire early enough. To that end, architecture decisions must be moderated with regard to all teams on the project. The moderating instance has to take care for postponed decisions and is responsible to get them up on the agenda right on time. That’s early enough, to carry out important preliminary work, e.g. to clarify business relevant questions. Some decisions even shouldn’t be postponed at all (e.g. because there are dependencies to other topics). This weighing-up should also be done by the architecture instance (you might call it a “guild”).

Sometimes, there are dependencies between various decisions, which may even apply to different teams at first sight. Those overarching dependencies must be identified, tracked and resolved, in order to achieve overall consistency. Experience shows, that it’s a good idea to transfer that responsibility to a dedicated group of people. This enables evolutionary development of the architecture without taking the risk of the “sleeping beauty” effect.

Devoxx vs. JavaZone

Every now and then I get queried which of the Java and Web conferences in Europe I would recommend. Often, the most sensible answer seems to be: “the one closest to you”.

Nevertheless, since I visited JavaZone in Oslo in September 2014, I think it may be time to make a thorough comparison after all.

But which are the criteria? The number of rock star speakers? The volume of soft drinks available for free? Alas, I came up with the following criteria:

Variety of topics

This is especially important because most people visit conference in order to peak over their horizons or to deep-dive into certain areas. As a minimum, conferences should cover the topics architecture, mobile, methods, coding and research, while all might bear different names or might be slit up into several sub-categories.

Quality of talks

Of course, the presentations must have content and meaning and they must be given in an inspirational manner as well. These are high expectations, and not many can live up to them. Especially talks, which center around products or have a high marketing potential, tend to be boring and to be of low quality.

Choice & Availability

If you have picked up your favourite talk and you can’t see it, because the room is crammed, that’s bad. It’s even worse if there is no sensible alternative, because not enough parallel tracks are available.

Community feeling

You also visit a conference in order to meet other developers and to talk to fellow architects. This is enhanced if the conference introduces a feeling of community into all its visitors. This can be achieved via events which include all or many visitors, by special after-dinner events, by talks which address all of the community and so forth.

Organisation & Infrastructure

You need to find your way to the conference, you have to know the program, must find the correct room and you need a working wi-fi. Especially the last issue is always a source of trouble, but it’s getting more and more important every year.

Venue & after-conference program

If you want to enjoy your conference, you need good lighting, cosy seating, and a venue which makes you want more of it. And in the night you need some distraction from all the hard-brained conference stuff, in order to be fit again next day.

Food & beverages

Everyone who ever organized a party knows: food is the one single most important thing. Thre is a saying “full belly does not easily study”, but that’s even more true for an empty stomach.

For a comparison, I gave each of the conference between 0 and 5 points for each of the criteria. The result looks like this this:

Criterion	Devoxx	JavaZone
Variety of topics	4	3
Quality of talks	4	4
Choice & Availability	4	2
Community feeling	5	4
Organisation & Infrastructure	5	5
Venue & after-conference program	4	4
Food & beverages	2	5

Let me explain, how I came up with the numbers:

Variety of topics: Devoxx is in lead, because it covers nearly any topic one can think of. JavaZone is only shortly behind since the program also is very versatile.

Quality of talks: The quality is above average for both conferences. Due to the sheer amount of sessions, there are in absolute numbers more mediocre talks at Devoxx, but this does not lead to a full point more for JavaZone.

Choice & Availability: Devoxx is clearly better here because all of its talks are in English (JavaZone has Norwegian talks still). Furthermore, the conference is longer (3 vs. 2 days) and contains more parallel sessions. So, you’re in for some hard decisions at Devoxx. Nevertheless, Devoxx does not earn the full score, because it is so over crowded, that you cannot see all the talks you want.

Community feeling: JavaZone is a true community conference. The same is true for Devoxx, which is a little bit ahead, because there are Keynotes (not at JavaZone) and opening and closing talks which appeal to the whole of the visitors.

Organisation & Infrastructure: This is excellent at both conferences and also includes the awesome websites and the (mostly) working wi-fi.

Venue & after-conference program: Devoxx can gather further points here because it offers an evening program at two days. But this is reduced by the venue being too much outside of the city center.

Food & beverages: Food is traditionally poor at Devoxx and excellent at JavaZone.  Devoxx has 2 points, because of its Belgium Fries night.

Which one of the conferences to prefer? It depends on which one is closest…

Portal Anti-Patterns - Installment 1: Misuse of portlets

In my IT consulting projects I regularly get the chance to inspect enterprise web portals and to take a deep look at their software architecture. What my team and I find there is troublesome at best. And the problems are the same in nearly all the portals I checked, thus it’s sensible to call them anti-patterns. They present a great opportunity to learn from mistakes already made lest you don’t repeat them.

In this little series I’d like to share this experience with you. This time, my topic is the use (or rather misuse) of portlets.

Anti-Pattern: “Page Portlet”

Portlets are a great way of bringing modules to the user interface. They also offer possibilities of reuse while at the same time encapsulating logic and data. Therefore, as a best practice, a portal page is expected to consist of multiple portlets, a reasonable number being 3 to 8. Nevertheless, I often encounter the so-called “page portlet” syndrome: each page of the portal is made up of one single portlets (or sometimes two portlets, one of which only represents navigation of header or something similar.

"Page Portlet" Anti-Pattern

This spoils all the positive aspects of portlets. Above all, portlets of this size are often so specific to a certain use case, that reuse is not possible effectively. Furthermore, it prevents approaches like responsive design to work on portlet level – everything has to be implemented in the portlet and thus cannot be configured and controlled by the portal server.

A “page portlet” should only be used as an intermediary step, when an application has to be split up into multiple portlets but in the current release there was not enough time to accomplish that. Then, you could tentatively integrate the whole application as one portlet into one portal page – but certainly not as final solution.

Achieving reuse with web portals

When I ask, what to expect most from a web portal, ten years ago I would have received the answer “Single-Sign-on”. Nowadays, the answer is “Increase time to market by reuse”.

So, reuse is the word. People don’t want to reinvent the wheel (which is good). And people expect portals to achieve that virtually automatically. This expectation is doomed to fail.

But first, what is a web portal after all? Basically, portals are a technique for integrating applications and content (s. picture) in the front-end, meaning: at the GUI level. That’s what sets portals apart from other integration technologies like shared databases or messaging, which operate on other architectural tiers. Anyway, we should remember that portals were thought to work on GUI level.

Portlets and portals don’t know anything about reuse, meaning that there is no “reuse” property built inside them. But that is exactly what – at least to my experience – people expect them to do. Even worse: portlets are no “building blocks” for applications and were never meant to be. They are elements of a portal user interface, which are based on some (very deeply technically defined) standards. This means, there is no concept of deconstructing an application into portlets. There is no “standard” how to do this. Although technically GUIs can be decomposed into portlets (which can make sense), they don’t help when you are facing problems like where to place the business logic, how to build reusable “business components” and how to communicate between them.

But, if portlets on there own can’t help – what can? Does it make any sense to use a portal? This question is important, because in some cases, it may be more appropriate to create a “simple” web application instead of using a portal product. This is a serious choice, since using a portal platform comes with a price: the developers not only need to know all the basic Java and JEE technologies, but they also need to have decent knowledge about the respective portal platform and they need to come to grips with a “utilization concept” of the portal. This means, they need to understand how to employ the possibilities of the portal best in their current business situation.

Nevertheless, using a portal can make sense, above all when you already have or want to build more a set of business applications which share an overlapping business process or which share parts of the user interface. If they only share parts of the logic, it’s no point using a portal, because portals work on the user interface, remember? This kind of reuse goes by the name of "platform reuse" (cf. picture) and can be achieved with a portal. But, it does not establish itself just by using the portal - you have to develop a concept how to use the tools given to you by the portal and how to use them on the respective applications.

Reuse with a portal platform

Given you decided that you still want to use a portal – we now have a first general rule:

Guideline No 1: Always search help from a portal expert – somebody who already built portals of comparable complexity and who knows about the possibilities of the chosen portal framework.

The second rule is even more important, especially when you want to achieve better time-to-market:

Guideline No 2: Establish a concept, a process and an organization to foster reuse. Always consider business and technical concepts for reuse.

And that is my bottom line here: reuse cannot be achieved by a portal or by any technology on its own. You always need creative people to think about it and to take initiative in order to get reuse right. Super duper portal technology does not resolve the reuse issue for you. You have to sit down, work out a concept and then implement it. Again, there is no silver bullet.

Let’s look at an example: you have two applications which share some functionality, say the lookup and presentation of a specific product. The search dialogue in the GUI is similar, the lookup logic is nearly the same, and they just work on different parts of the product catalogue. Also, the presentation of the product in the GUI is nearly identical – only the texts, icons and images are different. Although there is big similarity, you wouldn’t achieve any reuse just by using a portal. The programmers could be lucky to recognize the similarities but they would not take the extra time and effort to make all the code reusable, since they always are under pressure to meet the next deadline. Believe it or not: you won’t get reuse all by itself, without doing something for it. In our example, we have to set up a team, which reviews the business requirements and recognizes the overlapping parts in our search process as described above. At that point, a decision has to be made whether to invest in a reusable component “search product” which could include one or more reusable portlets. Time and money have to be invested – and we need a team which is capable of designing, building and maintaining a reusable component. Once you own this reusable components they will help you to free resources from re-inventing the wheel, they will help you to incredibly reduce your time-to-market and will give you an overwhelming quality boost.

So, products provide the basic means by which reuse can be achieved, but it still needs the right people to bring that possibility to life.

Software Architecture - more than documents

As I read the following sentence in an IT magazine some days ago it immediately caught my attention: „Also, in smaller projects architecture (the documentation of the software solution) is obligatory. “.

That is, where I have to object: architecture is not only the documentation. Architecture even exists without any documentation, and it can be a good one, mind you. These days, many people tend to overly concentrate on the documentation of the architecture rather than on the methods and processes needed to conceive it. For instance, the arc42 template is a good one (and I don’t want to be misunderstood on that) but it also only focuses on documentation.

According to my understanding (and I am talking mainly about software architecture here – as opposed to business architecture, for instance), architecture refers to three different aspects:

-         the process of conception of the software system (some people call that “technical design”)

-         the inner structure of a software system (and its sources)

-         the documents needed to describe that structure (e.g. component view)

These three aspects are correlated. The following diagram depicts this relationship.

Picture 1: The three aspects of software architecture

Although correct and sound documentation is necessary to understand and communicate the architecture, it is not the same as the architecture. If we would conceive, design and build an app with an excellent architecture and install it and after that throw away all the documentation, the architecture would still be in existence, would still be splendid and would hopefully still exhibit all the nice quality criteria it was designed to. Our app would still be very stable and would respond to user input very quickly.

As I wrote in another post, software architecture is a set of concepts which resolve the functional and non-functional requirements of the system (cf. http://juriswritings.blogspot.de/2012/03/what-is-software-architecture.html). Therefore, these decisions must be safe and sound. In order to make sensible decisions the architects above all need experience and good requirements documentation (e.g. domain models and use cases).

As the project progresses and the bigger the project gets, the architects also need models of the architecture, because they may have to make decisions which change parts of the architecture already discussed. Thus, documentation is important, not only to help new people to get on the project and to let people do maintenance later. But – and that is my point here – it is not the same as the architecture and it is also not the first thing which needs to be created. Architects have to decide and they have to invent sensible concepts. Documentation is merely a tool to help them do their work – and we should keep that in mind when documenting.

BTW: the said article is “Architecture for eternity” in “Entwickler Magazin 04/2013” by Nils Arndt. The sentence is below the fourth sub heading, which is on page 2 of the article.

An HTML5 canvas component for ExtJS 4

As I described in another post, ExtJS 4 features components for graphics ("Ext.draw.Component") which are based on the standard HTML SVG tag. But what if we liked to use a <canvas> tag? ExtJS does not bring one of those - thus we have to create one.

We also need a nice little showcase. For that, I implemented a small application, which simulates a solar system, with a configurable number of planets, moons and stars. The application calculates the gravity which affects the objects and then determines their new positions. This is done repeatedly, leading to a simulated continuous motion. The positions (and trajectories) of all objects are displayed in the canvas element. Picture 1 shows the solar system simulator in action.

Picture 1: Solar system simulator written with the canvas component

I put the canvas component in a separate file (canvasPanelClass.js), thus it should be easily reusable. The component utilizes the powerful object-oriented features of ExtJS. It extends Ext.Panel and creates the canvas element in the constructor method. In the afterrender listener (when the component already is complete) it stores a reference to the canvas in a private variable (this.canvas = this.items.items[0].el.dom;).

After that, the canvas is ready to be used. The component already brings some important methods, e.g. drawCircle( 50, 50, 10, "red" ) which draws a red circle with a radius of 10 pixels around the point 50, 50.

And now, the most interesting part of the source code:

        Ext.define('CanvasPanelClass', {
            extend: 'Ext.Panel',
           
            gridColor: '',
            ctx: null,        // is set when rendered
            canvas: null,    // is set when rendered                       
            bodyStyle: { background: '#000' },
            frame: false,
            margin: '2 2 2 2',
           
            listeners: {
                afterrender: {
                    fn: function(){                       
                        this.canvas = this.items.items[0].el.dom;           
                        this.ctx = this.canvas.getContext("2d");                                       this.clear();                       
                    }
                }       
            },           
           
            constructor: function(config) {
               
                //define this here because we can set width and height
                this.items = {
                    xtype: 'box',
                    autoEl:{
                        tag: 'canvas',
                        height: config.height,
                        width: config.width
                    }
                };
               
                this.tempCanvas = document.createElement("canvas");
               
                CanvasPanelClass.superclass.constructor.call(this, config);
            },
                    
            drawRect: function( x, y, width, height, lineWidth, color ) {
                this.ctx.strokeStyle = color;
                this.ctx.lineWidth = lineWidth;
                this.ctx.strokeRect(x,y,width,height);               
            },
           
            fillRect: function( x, y, width, height, color ) {
                this.ctx.fillStyle = color;
                this.ctx.fillRect(x,y,width,height);               
            },       
           
            drawCircle : function( x, y, radius, color ) {
                this.ctx.beginPath();
                this.ctx.strokeStyle = color;
                this.ctx.fillStyle = color;
                this.ctx.arc( x, y, radius, 0, Math.PI*2, true );
                this.ctx.closePath();
                this.ctx.fill();       
            },           
           
            putPixel: function( x, y, size, color ) {
                this.ctx.fillStyle = color;
                this.ctx.fillRect(x,y,size,size);           
            },
                  
            clear: function() {
           
                // Store the current transformation matrix
                this.ctx.save();

                // Use the identity matrix while clearing the canvas
                this.ctx.setTransform(1, 0, 0, 1, 0, 0);
                this.ctx.clearRect ( 0, 0, this.getWidth(), this.getHeight() );

                // Restore the transform
                this.ctx.restore();                                      
            }

        }); // define CanvasPanelClass

Then, the canvas component can be used like in the following code snippet. (There is an array of canvasPanels because you might want to create more than one canvas.) The mousedown handler shows how an application can react on mouse clicks into the canvas.

      canvasPanel[0] = new CanvasPanelClass({

            height: 500,

            width: 500

      });              

      canvasPanel[0].on({

            mousedown: function(DOMevent) {

                             var canvasPanelX = this.getPosition()[0];

                             var canvasPanelY = this.getPosition()[1]

                             var point = DOMevent.getPoint();

                             var eventX = point.x-canvasPanelX;

                             var eventY = point.y-canvasPanelY;

                              // do something …

                             canvasPanel[0].clear();

                             canvasPanel[0].renderGrid(25*50/ZOOM,GRIDCOLOR);

            },

            element: 'body',
            scope: canvasPanel[i] //Ensure "this" is correct during handler execution

      });

      tablePanel.add(canvasPanel[0]); 
  
      canvasPanel[0].clear();    

      canvasPanel[0].putPixel( 50, 50, 1, "green" );