Spikes: Definition and Usage in Scrum

Posted by dougdockery on February 23, 2014 in Agile, Scrum with Comments closed |

In my current assignment, the teams that make up the first wave of the Agile transformation seem to be confused about what a spike in Scrum is, how to use a spike, etc.  The following was put together from a variety of sources to send out to the teams in an attempt to come up with a common definition of:

  1. What a spike is
  2. What can a spike be used for
  3. When to use a spike

I’m interested in other’s experiences, etc.  Let me know how your teams are using spikes or if there are success stories that you can share.

Spike Definition and Usage

Spikes are an invention of Extreme Programming (XP), and are a special type of user story that is used to drive out risk and uncertainty in a Sprint.

Spikes, by definition, have a maximum time-box size of one Sprint. That’s your upper bound. Spikes can be as small as an hour. That’s your lower bound. At the end of a Sprint, the spike is done or not-done, just like any other story. When spikes are written on a card or in Rally, they should be written as a question with an enumerable answer. For example “Will a 3 attribute key prove to be unique in the customer and order detail tables?” or “Will Ruby allow us to overcome <a specific limitation> in our current development environment?” or “Will DB2 or SQL Server give greater performance for the application we are creating?”.

A Spike is a technical investigation to produce an answer in regards to some acceptance criteria on a story prioritized in upcoming Sprints. It’s a great way to mitigate risks early and promotes fluid Sprints later in the project. It allows the team to gather feedback and develop an understanding of an upcoming story’s complexity. As noted below, spikes should be the exception, not the rule. If your team has a spike in every Sprint, your team is probably not using spikes correctly.

Spikes may be used for a number of reasons:

  • The team may not have knowledge of a new domain, and spikes may be used for basic research to familiarize the team with a new technology or domain.
  • The story may be too big to be estimated appropriately, and the team may use a spike to analyze the implied behavior, so they can split the story into estimable pieces.
  • The story may contain significant technical risk, and the team may have to do some research or prototyping to gain confidence in a technological approach that will allow them to commit the user story to some future Sprint.
  • The story may contain significant functional risk, in that while the intent of the story may be understood, it’s not clear how the system needs to interact with the user to achieve the benefit implied.

Technical Spikes and Functional Spikes

Technical Spikes are used to research various technical approaches in the solution domain. For example, a technical spike may be used for evaluation of potential performance or load impact of a new user story, evaluation of specific implementation technologies that can be applied to a solution, or for any reason when the team needs to develop a more confident understanding of a desired approach before committing new functionality to a Sprint.

Functional Spikes are used whenever there is significant uncertainty as to how a user might interact with the system. Functional spikes are often best evaluated through some level of prototyping, whether it be user interface mockups, wireframes, page flows, or whatever techniques is best suited to get feedback from the customer or stakeholders.

Some situations may require both types of spikes. For example, consider the following user story:

As a consumer, I want to see my daily energy use in a histogram, so that I can quickly understand my past, current, and likely near term, future energy consumption.

In this case, a team might create two spikes:

Technical Spike

In order to update the customer display, should we push or pull the data.

Functional Spike

Prototype a histogram in the web portal and get some user feedback on presentation size, style, and charting attributes. The outcome of the spike is to determine if the prototype is effective or not.

Spikes should be estimated as in-Sprint tasks during Sprint Planning. The task’s duration should be spent researching and developing some ‘thing’ that can be delivered. That thing can be a working piece of software, workflow, documentation, etc. Ultimately the value from the spike is a direction or re-direction in the course of the feature. If the team estimated that a Spike takes four hours, then ONLY four hours should be spent researching or developing. Prototypes, Proof of Concepts (PoC), and Wireframes all fall into the classification of Spikes.

Guidelines for Spikes: Estimable, Demonstrable, and Acceptable

Like other stories, spikes are put in the backlog, estimable and sized to fit in an Sprint. Spike results are different from a story, as they provide an enumerable answer rather than working code. A spike may result in a decision, a prototype, storyboard, proof of concept, or some other solution to help drive the final results. In any case, the spike should develop just the information sufficient to resolve the uncertainty in being able to identify and size the stories hidden beneath the spike.

The output of a spike is demonstrable, as an enumerable answer, during the Sprint Demo. This brings visibility to the research and architectural efforts and also helps build collective ownership and shared responsibility for the key decisions that are being taken. And like any other story, spikes are accepted by the Product Owner when the acceptance criteria for the spike have been fulfilled.

The Exception, not the Rule

Every user story has uncertainty and risk — this is the nature of agile development. The team discovers the right solution through discussion, collaboration, experimentation, and negotiation. Thus, in one sense, every user story contains spike-level activities to flush out the technical and functional risk. The goal of an agile team is to learn how to embrace and effectively address this uncertainty in each Sprint. A spike, on the other hand, should be reserved for the more critical and larger unknowns. When considering a spike for future work, first consider ways to split the story into smaller, easier to understand or less risky parts. Use a spike as a last resort.

Consider implementing the spike in a separate Sprint from the resulting stories

Since a spike represents uncertainty in one or more potential stories, planning for both the spike and the resultant stories in the same Sprint is risky, and should generally be avoided. However, if the spike is small and straightforward and a quick solution is likely to be found, there is nothing wrong with completing the stories in the same Sprint. The key here is to be careful when adding a spike and the stories that depend on the answer to the spike in the same Sprint.


Use Spikes when they make sense to gain a deeper understanding of the upcoming stories. If the team is unable to estimate a user story, it generally indicates that the story is too large or uncertain. If it is too large to estimate, it should be split into smaller stories. If the story is too uncertain to estimate, then a technical or functional spike story can be used to reduce uncertainty, so that one or more estimable user stories result. Don’t be afraid to take on the stories without the Spike and use the Sprint Review as an opportunity for feedback


Leffingwell, Dean – A User Story Primer

Hortonworks Sandbox

Posted by dougdockery on May 15, 2013 in Big Data, Hadoop with Comments closed |

Want to learn about Hadoop and Big Data without having to set up a server, etc?  Hortonworks has you covered with their new Sandbox – Get more information here


Installing IBM BigInsights on CentOS 6

Posted by dougdockery on March 11, 2013 in Big Data, Hadoop, IBM BigInsights |

I have been discussing BigInsights and Streams with IBM since the beginning of 2013.  I wanted one of my teams to start a proof of concept for BigInsights. When we looked for instructions on how to successfully install and configure IBM’s big data offering, we couldn’t find any tutorials that worked for us, so I decided to create one.

For this example I chose to use CentOS 6.0 (CentOS isn’t a supported version of Linux as far as IBM is concerned, but it’s essentially the same as Red Hat so it will work for BigInsights) and am using BigInsights Basic Edition. Both are available free of charge.

The outcome of this tutorial is: I currently have a single node version of BigInsights Basic Edition on a Dell notebook (8 GB RAM, i7 processor) that is running CentOS 6.

Here’s how I got it to run:

Install CentOS Operating System

Download CentOS 6 from mirror.centos.org/6/isos/x86_64/
Install CentOS (during install, choose the Desktop install option)
Log in as root

Download and decompress InfoSphere BigInsights

Download BigInsights from www.ibm.com/software/data/infosphere/biginsights/basic.html
Save the file to the Desktop (for this example the filename for BigInsights is iib2000_linux_64.tar.gz)
Open a new Terminal window
tar -xvvf iib2000_linux_64.tar.gz

Pre-installation Configuration

Set up passwordless SSH
ssh-keygen – t rsa

You will be presented with three prompts, select the default option for each by hitting Enter

Copy public key file to authorized key folder
cat /root/.ssh/*.pub > /root/.ssh/authorized_keys

Test passwordless SSH for root user
ssh root@localhost

You will receive an warning stating that “The authenticity of the host ‘localhost (’ can’t be established” – simply reply ‘yes‘ at the prompt

Create biadmin user that we will use to do the web-based portion of the install
useradd biadmin
passwd biadmin
Enter password 2x

You will need to use vi to add bidamin to the sudo user list. If you haven’t used vi, here are some hints:

  • ‘i’ = start entering text
  • ‘Esc’ stop entering text
  • ‘:’ + “wq’ = save file

Add biadmin to the sudo user list:


Find the line that reads:
defaults requiretty

Comment it out:
#defaults requiretty

Find the line that reads:
## allow root to run any commands anywhere
root ALL=(ALL) ALL

Insert a line to make the section appear as:
## allow root to run any commands anywhere
root ALL=(ALL) ALL

Install rpm packages that I’ve found to be missing (and required) during the web portion of the install:
I’m not sure if everyone experiences the need for these, but each time I’ve done the install it has failed due to a configuration check that says the expect rpm is missing. So:

yum clean all
yum install tcl
yum install expect

You must also disable SELINUX or your will receive the following error:

ERROR PriorChecker – 255 – com.ibm.xap.mgmt.ConfigurationException: selinux configuration not compatible – SELINUX=enforcing

To disable SELINUX:

Navigate to the /etc/selinux/ directory

vi config

Find the line that says selinux=enabled and change it to selinux=disabled
save and close the config file

Starting the BigInsights Service

Now, we’re finally ready to start the web-based portion of the install.  If you use IBM’s installation instructions, they generally start at this step and assume you knew to do all of the pre-install configuration items listed above.

Open a new Terminal window

cd /Desktop


cd into the biginsights directory


Once the server boots up and says it’s ready, open Firefox 

The initialization of the server says that it’s going to give you a URL to enter for the  web-based portion of the install.  In my experience, it never does, so enter the URL:

The following are the images of each screen in the web-based installation:

BigInsights Screen 1

Accept the license agreement:

BigInsights Screen 2

Since I am installing on a single notebook computer, I selected a single node installation

BigInsights Screen 3

Accept the defaults on the File System screen

BigInsights Screen 4

This is where we use the biadmin user that we created

BigInsights Screen 5

All you have to do on the Summary screen is click ‘Install’


BigInsights Screen 6

If all goes well, you will receive the following:

BigInsights Screen 7

Congratulations, the web-based portion of the install is complete

BigInsights Screen 8

Click the console link and the BigInsights Console will open and allow you to manage your implementation

BigInsights Screen 9

Verify the open source software packages that were installed

Open a new Terminal window
To check that all components are sourced correctly, we need to run the following commands:

su – biadmin
enter the password for biadmin if prompted

nano ~/.bashrc

Make sure the following lines exist:

#user specific aliases and functions
source /opt/ibm/biginsights/conf/biginsights-env.sh

Insert a blank line after the source line (above) and then we can add pig, jaql, flume, and hive to the path by inserting the following:

export PATH=$PATH:${PIG_HOME}/bin:${FLUME_HOME}/bin:${JAQL_HOME}/bin:${HIVE_HOME}/bin

Save and Exit

Open a new Terminal Window

To verify that the path changes above work, enter:

su – biadmin
enter the password for biadmin if prompted

Now we can try to start pig to make sure it’s been successfully added to the path.  Enter:


you should receive a prompt that says:


if so, your installation is complete.

Tags: , , ,

Copyright © 2013-2014 Doug's Blog All rights reserved.