Introducing New Technology in Agile
Generally Agile projects are of very short durations. So, introducing new technologies in such a small time frame is always risky. If you plan to introduce any new technology and it fails royally then the time and cost to set up the infrastructure, along with sprint’s failure will put you down.
One of the important things to keep in mind while selecting a technology for the application will be its fit into the application rather than the popularity it has gained in the market. The suitability of any technology for the demanded solution should be given much more weight while making the decision rather than the market trends.
In one of our projects, there was need for unstructured text analysis; UIMA looked like doing exactly what was needed. But UIMA was using externalization for serializing rather than serializable interface which was making its objects bulky and lots of work was needed to serialize data. So, this functionality was developed in house rather than using UIMA.
It’s also of utmost importance how to not choose a technology. To go for a technology only because of it popularity, ease of use, or price may not lead you to the best of choices available in the technology genre.
“The team will have an inclination towards technologies they feel comfortable working with as each team already has an established skill set. The foremost concern should be the features and functionalities rather than the skill set. Be sure to work with your technologists so that they understand that they’ll be expanding their skill set and getting the training that they need.
Your boss read about it in a magazine, heard about it on the train, it came to him/her in a dream.How do you tell your boss about other options without seeming to diminish or devalue their discovery? This requires tact, and a bit of treading softly. Volunteer to make him/her even more well-informed by expanding the work you’re doing to investigating into other options - at his/her request, of course!
Your friend told you it was 'da bomb! It is in our nature to accept a vivid retelling by someone we know over raw data from an impersonal source. But when it comes to choosing a technology solution, anecdotal evidence does not present us with a comprehensive review of the software, its features, its functionality, or the viability of the company.
It’s expensive, it’s cheap, it’s free. It’s easy to assume that because a product is the most expensive, it must be better than its competition. Similarly, many of us who are on a budget are always on the lookout for a bargain. There are a great many factors to consider when evaluating a technology solution, don’t take price alone as an indicator of value.”
The other important thing is the ability/competence of the team to work on the new technology introduced. It won’t be surprising if you find someone struggling with a single problem for few days.
This is pretty common when new things are introduced. So, it’s a must to pair program while working on the new technologies. For certain problems, the mental bandwidth required is more than one developer's mind, so, pairing helps here for sure.
In Agile, the team size is pretty small, so if only one person keeps working on the new technology, then the team becomes dependent on him. While pair programming the knowledge about the new technology also distributes in the team and the team has a balanced truck number.
One more benefit of pair programming on new technologies is the learning of the team.
The learning of each individual is necessary. Programmers must equip and enhance themselves with the latest technologies in order to be efficient. It not only helps the project but also satisfies the programmer's quest for knowledge. Learning will make them satisfied with their job which in turn will increase the productivity of the individual and the team.
“In order to find the right balance it is important that all the engineers know what they’re doing throughout the sprint. If a sticking point arises, eg. a situation where you notice an engineer stuck with their head down for hours on end, a couple of scrums pass where they say they have got 1 or 2 hours left and they’ll be finished that day but they never do, then you know you have a problem.
Sometimes this can be solved by pairing, but probably it’s a sign the team has not done enough homework.”
Stability, Support and Documentation
Other things to look after will be the support/documentation available for the technology. I have experienced that APIs with no, little or faulty documentation are very difficult to explore. If you are among one of the early adopters of the technology, then it’s going to be more difficult as there would be very less support available in the form of blogs/articles/forums. However, something not very well documented is almost useless unless you devote a great amount of time in exploring it completely.
I tried to use Apache Batik in one of my projects but the documentation and support was not that overwhelming. I encountered problems every now and then and found that not much was available on Internet about the usage of this technology. On the contrary, JavaFX also serves the same purpose but it is quiet well documented and supported.
In one of the projects, a user interface was needed, which can run as a desktop application as well as in a web browser. Scalable Vector Graphics (using Apache Batik) was picked up for this purpose.
The stint with Apache Batik ended soon due to the problems faced. However, Apache Batik was discussed in the team, explored, and then rejected as a solution even before setting up the infrastructure. Then other technologies were searched upon and JavaFX emerged as the winner. JavaFX looked promising, easy to use and an emerging technology so, its infrastructure was setup in a sprint, in which a simple looking UI was created, and then further development happened on JavaFX.
The compatibility of the new technology with already implemented technologies in the project is also of great importance. Like in the previous project mentioned, some swing component was already in place, and JavaFX provides a feature to embed swing components which made those swing components reusable and saved some rework.
Support on a particular technology is also quiet important as you might be stuck at a dead end and need a way through. Open source products score over here as even if you don’t have any support for the technology, you can see the code and change it. If it’s not open source, then you might find yourself blocked at some places if adequate support is not available for the technology. For open source technologies, you can also find companies who can modify and customize products for you if you don’t have in house capability to do this.
Most popular open source solutions are extremely well documented and a variety of free and commercial technical support options are available. Due to the nature of community development, documentation and instructions are often written from a variety of viewpoints - creating well-rounded information, instruction and tutorials. In addition, open source projects can't hide usage techniques, due to the free availability of the code. Free technical support is often available in the form of mailing list or newsgroup discussions; nevertheless some background research, knowledge or experience is often required.
"The availability of the source code and the right to modify it is very important. It enables the unlimited tuning and improvement of a software product.
The right to use the software in any way. Which helps in turn to build up a market for support and customization of the software, which can only attract more and more developers to work in the project.
There is no single entity on which the future of the software depends. If the group or company that originated the code decides to stop development, it is always possible to fund another software group to continue the maintenance and improvement, without legal nor practical limitations.
No per-copy fees can be asked for modified versions. People trying new technologies are able to integrate and adopt them immediately, without the hurdles of commercial or non-disclosure license agreements."
Figure out the time since when it has been in the market and its expected tenure. There is no point in using a soon to be outdated technology. Nowadays, technologies evolve at lightning speed, so, always check for the competitors of the technology, you might get some surprises.
Iterative Induction of Technology
In order to introduce new technologies/frameworks, it’s always good to take small steps at a time. It’s nice to explore the technology a little bit and then to analyze whether it suits as a solution. Then try to solve the smallest/trivial business problem using the new technology/framework. Don’t depend on it completely in the beginning, just let a small part of application use the technology and see whether it works or not.
This helps in getting used to the syntax and other features of the technology/framework. The other big advantage of doing things this way is that you do not need to devote time separately for setting up infrastructure to use all the new things.
In one of my projects, Drools 5 was chosen as a solution for some problem. But, the applicability and feasibility of using Drools 5 was not known. Other than this, Drools 5 was new for all the members of the team. So, one sprint was aimed at just consuming Drools 5. A simple problem was thought of to be solved with the help of Drools. This gave the certainty that Drools will be set as an infrastructure in this sprint. Along with this, the team members will get time to read about it.
Solving a simple problem using Drools gave a chance of hands on which boosted the confidence in the technology.
Being a Java Project, for Drools 5, the dependencies were added in the pom. The Spring beans were defined to create the session and a simple drl file was written. A drl file has a Drools specific syntax in which rules are written. The code executed some rules on objects which were passed to it using the session beans. After this sprint, Drools infrastructure was in place, the team was aware of its syntax and other techniques. And most importantly, there was a solution using Drools 5. So, the confidence in technology increased.
“Introduce a new software technology incrementally with enough successes over time for the technology to gain traction from the point-of-entry outward into the rest of the organization. If the introductions don't succeed for whatever reason, the overall risk and cost to the organization is low.”
The problem for which the solution has been implemented should not be that important in this phase. The main aim is to set up the infrastructure for the new technology. Once the infrastructure for the project is set up completely, and it looks comfortable enough to carry on, then implementing it will be less risky.
Once the application adopts/accepts the new technology and the desired results are achieved, its functional acceptance criteria are achieved. However, once you start digging deep into more complex cases and the technology becomes the part of the application, functional excellence is not the only thing you want. This is where performance related issues might start bothering you.
If you have not been keeping eye on the performance hit by the new technology from the beginning, then it might become a overhead now and this can result in loss of all effort put into the adoption of new technology. So, keep track of serious performance hits while inducting the technology. If the technology is going to be a bottleneck, then try to figure that out and discard it at the earliest.
Risk Taking Capacity
The risk taking capacity of the team at the current state of project can be another factor in deciding whether the technology should be introduced or not. If the project is not that stable and the team is struggling even with the currently implemented technologies and other problems, then it’s not logical to add a new task to cope with the new technology. Introducing new technologies at this stage can hit hard on the team. They will have to handle an extra task of mastering new technology at the wrong time. However, if the team is performing well and the product is stable then go for the kill.
“The combination of high perceived risk and low actual risk occurs when a proposed solution involves technologies or products that have not already been established in the standard technical infrastructure the IT organization supports. Look for an opportunity to introduce a new technology that is not technically difficult to implement, but that has not been used at your company before.
This will drive the traditional project estimate higher without increasing the actual difficulty of your project. “
Finally, “Trust” in the technology is all you need after you have accepted the technology. It is a personal phenomenon. However, the level of trust you have in the technology accepted will drive the ease with which you go with it. It is highly dependent on the individual as said by Bahmanziari, Tammy writes in The Journal of Computer Information Systems about trust.
“The personality of the individual adopter plays an important role in the propensity to trust the unfamiliar. This personality characteristic is often referred to as disposition to trust. Some individuals exhibit high levels of trust without any specific reason to do so, while others exhibit low levels .of trust when higher trust would be reasonably expected.”
The technology adapter must have done some analysis mentioned earlier in the article to form a perception of the technology. Use the information gathered in the analysis to strengthen your trust in the technology. You need trust because the technology has not been tried and tested. Had it been a tried and tested technology, there was no need for trust as you would have already known the bits and pieces of it.
“Most trust theorists would probably agree that although no one has been able to develop a definition of trust that is useful across all situations, both uncertainty and some degree of vulnerability would be necessary for trust to exist. After all, with complete certainty, no trust is necessary. Because this uncertainty exists, vulnerability is created. A technology adopter is operating in an environment of uncertainty; hence their evaluation must include perceptions and incomplete knowledge about the new technology and the provider of that technology. "
The technology introduced should be able to solve the business problem. It should not be selected on the basis of its popularity. Early adoption of technologies should not be done if the project is in bad state. The developers should not fear about the learning curve as it would enhance their knowledge. The technology should be inducted iteratively. Success/Failure achieved in one iteration should govern next iteration steps. Pair programming should be done on new technologies to distribute knowledge in the team. Keep an eye on it before accepting it completely. Have confidence in the technology and you will come out with flying colors.
would be intrested in your thoughts
Re: would be intrested in your thoughts
For example, I completely agree with 'Stabilty, Support, and Documentation' and you have made a great example with your bad experiences with Apache Batik and good experiences with JavaFX. But you use that to go on to say that Open Source does well in this topic when in this case it is the opposite - Apache Batik is open source and JavaFX is not open source (atleast not yet).
The point is to have a Stable product which is well documented. Which is depicted by the Apache Batik vs the JavaFX example.
However, in general, you will find better support for open source projects having an active set of committers. Not every open source project has an active group supporting it, so, it might not be true in all cases. But, the ones which have an active group are really stable and well documented.