The inspiration for this post is KaChing’s wonderful DevOps dashboard. This dashboard empowers Kaching team to do about 50 deployments to production per day.It takes about 5 mins from a code commit to deployment.This brings to the forefront need of effective dashboards in each step in Agile process.While as an organization we might have different tools that provide a high level analytics,it’s important that all this data is collected and presented to do release level risk analysis and status analysis.Let’s check out some common metrics that we track while doing Agile development and see why it makes sense to bubble up this metrics to the Super Dashboard
Code development
Normally unit testing is written as code is being written.This is a compile time statistics and normally resides in your build server (e.g Hudson).This data should be bubbled up to Super dashboard.
Coverage statistics:We would want to measure whether all the code that was written is unit tested or not.Again a very good candidate to put up in the Super Dashboard
Integration Testing
You might be using different tools like FitNesse,RSpec,Cucumber but all this tools would have a statistic that you would normally track to promote the builds.The results from this testing should be consolidated and be part of the Super Dashboard
UI Testing
Again you might be using tools like Selenium,Watir,Silktest or other GUI tools.The pass/fail result ensures Test Completeness and should be made part of the Dashboard
Test Management tools
Several data points are collected as part of your Test Management tools like Quality Center,QMetry.This can be an important data point to understand if enough testing was done or not.Different release data points from this tool should be made part of the dashboard.
Requirements Management Tool
Metrics from tool like RallyDev,TargetProcess will ensure that a particular release is on track and will give the complete picture in terms of real progress(E.g good unit testing numbers don’t mean much if the requirements are not yet completed)
You might have other things that you would like to see in your Super Dashboard.The goal is to just look at it and turn on the switch on deployment.
