An Agile Software Estimation Technique based on Regression Testing Efforts

Abstract: Agile software methodologies are widely used in a variety of industrial projects because of their volatile and flexible nature. Due to the flexible nature the estimation of cost and effort is a very difficult task in Agile environment. Ignorance of estimation methods may cause serious effects like exceeding the budget, not delivered on time, poor quality and not right product. So accurate estimation of software is critical for both developer and customer,that involves the determination of effort (person-month), duration (in months) and cost (in rupees).In Agile, software is developed in small iterations so regression testing is done to check whether the new changes have not adversely affected the existing features.However, regression testing efforts have not been considered for estimation in previous Agile estimation techniques.In this paper to support flexible and changing nature of Agile, a regression testing based algorithmic estimation technique has been proposed that estimates the more accurate release date, cost, effort and duration  for the project. The effectiveness and feasibility of the proposed algorithm has been shown by considering a case study of small web-based application which shows the efficacy of the proposed method.

1. Introduction

 The success and failure of a software project is determined by accurate estimation. This is the process to calculate the time that will be taken to finish the project, cost of the project and the effort required to complete the project. Estimation is very important task as improper estimation may lead to failure of the software project. It may also increase the budget of the customer and sometimes the nature of the project is also affected[1]. The estimation in the Agile environment is a difficult task due to the changing requirements. The figure 1 shows the Agile software development life cycle.

                                        Fig 1: Agile Software Development Life Cycle

Regression testing is done in Agile to make sure that the new incorporated changes should not have side effects on the existing functionalities and thereby finds the other related bugs. Thus regression testing may consume much time, cost and effort in the project and therefore it is considered as an expensive process. So there is a need for an algorithmic estimation technique to calculate the time, cost and effort of the project using regression testing.

In this paper people and project related factors[3] are considered and regression testing is performed. Based on these factors and regression testing in each iteration an algorithm has been designed for the cost, size and effort estimation of a software project.

In Section 2 different estimation technique in Agile estimation are discussed. In Section 3 related work in field of Agile estimation is described and in Section 4 problems related to estimation in Agile are discussed and Section 5 describes proposed algorithm, in Sector 6 case study is taken, Section 7 contains the Conclusion and future work.

2. Agile Estimation Techniques

Estimation is the process to estimate the cost, effort and the time taken for a software project. The process of estimation starts from the planning phase of the Software development life cycle (SDLC) and this is refined throughout all the phases of the SDLC [1]. Agile methods are volatile in nature and can accept last minute changes. Some approaches of estimation in Agile are discussed in table 1.

Table 1: Agile Estimation Techniques

Technique	Approach	Problem
1.Analogy	This approach is based on the collective experience and knowledge of several Project managers drawn from the results of many specific projects.	This method is not recommended as it may provide very unrealistic estimates.
2. Expert Opinion	In this technique an expert is asked how long a task will take to complete.	The Problem with this technique is that it requires the opinion of multiple experts.
3.Top-down	An overall cost estimate for the system is derived from global properties, using either algorithmic or non-algorithmic methods.	This method does not promote good software engineering practice.
4.Bottom-up	In this method each component of the software system is separately estimated and the results aggregated to produce an estimate for the overall system.	The problem is that how the system is decomposed into different components.

3. Related Work

       This section identifies some of the key research work in the area of Agile development. S.Bhalero, Maya Ingle [2] proposed an algorithm for cost estimation by incorporating various factors with different intensity levels. These factors include mainly project domain, configuration, performance, complex processing, data transaction, operation ease, multiple sites and security. But various factors like communication skill, managerial skill, and familiarity in team are missing.

        The author of this paper [3] proposed an algorithmic method for cost and effort estimation in Agile by considering project as well as people-related factors like quality requirements, complexity, transaction etc.

       Jørgensen and Sjøberg [13] report that preplanning effort estimates can have a major impact on detailed planning effort estimates, even when the estimators are told that the early estimates are not based on historical data or expert knowledge, i.e., should not be considered as relevant information to the estimation process.

       Abrahamsson et al. [6] demonstrated how to collect metrics to measure productivity, quality and schedule estimation, cost and effort estimation for an Agile Software Development project using XP.  They discussed the concept of Agile development, presented processes, roles, practices, and experience with 10 agile development methods, and compared the methods with respect to the phases that they support and the level of competence that they require. They provide evidence that Agile methods are ‘‘effective and suitable for many situations and environments”.

        Williams et al. [9] investigated the usage of a subset of XP [11] practices at a group in IBM. The product developed at IBM using XP was found to have significantly better pre-release and post-release quality compared to an older release. The teams using XP reported an improvement in productivity, schedule, cost and effort estimation. In addition, customers were more satisfied with the product developed using XP because the teams delivered more than what the customers had originally asked for.

      Heemstra [10] surveyed 364 organizations and found that only 51 used models to estimate effort and that the model users made no better estimate than the non-model users. Also, use of estimation models was no better than expert judgment.

      Finnie and Wittig [12] applied artificial neural networks (ANN) and case-based reasoning (CBR) to estimation of effort. Using a data set from the Australian Software Metrics Association, ANN was able to estimate development effort within 25% of the actual effort in more than 75% of the projects.

4. Problems in Existing Methods

       Most of the existing effort estimation techniques have been developed to support traditional sequential software development methodologies whereas Agile Software Development is iterative in nature. If these traditional techniques are used for effort estimation of Agile software projects, then the results will be definitely inaccurate. Based on the critical study of various research work discussed above, several problems are there in existing estimation and tracking methods for Agile software developments (ASD).

The first problem is that regression testing is not considered in previous agile estimation methods. Since the software is developed in small iterations in Agile so there is a strong need of regression testing to check whether the new added code has not affected the previous one. Therefore regression testing consumes much time, cost and effort. So there is need to consider the regression testing related efforts also.

 Second, as the requirements are taken in Agile Software Development in terms of Story-points which describe the size of a particular requirement and since a story point is a relative value, the total story point value can fluctuate with a slight variation in the baseline story point. To set the baseline story point, the Agile team finds the simplest user story and determines story points of other user stories based on the baseline. If the baseline story point changes, other story points also get changed. Thus, an absolute cost metric rather than a relative metric is necessary.

 Third, since the story points and velocity are used to estimate the initial size of the project, a schedule is derived using a simple linear extrapolation. But if regression testing is done then the velocity of the project is de-accelerated. So, there is a need that the velocity should be measured at the end of iteration after considering the regression testing efforts and the various factors which affects the project. 

 In Agile environment, at the initial stage of a project, there is high uncertainty about various project attributes. The estimates produced at early stages are inaccurate, as the accuracy depends highly on the amount of reliable information available to the estimator. Agile Estimation methods may lead to the errors in case of inexperienced Agile team. Therefore, there is strong need of analyzing the factors that affect the estimation of the Agile project.

Therefore an algorithmic effort estimation model is needed to predict development effort of Agile Software development.

5. Proposed Regression Testing Based Agile Estimation Model

                There are a multitude of factors that affect the ability of an estimator to accurately estimate the efforts. Accurate estimation requires a multidimensional view to produce accurate and effective estimates. The various factors which can affect the cost and duration of the project have been taken in [3].However, regression testing efforts have not been considered while estimating with the project and people related factors. To accommodate the regression testing efforts which consumes much time and cost during every iteration, so a regression testing based Agile estimation technique is being proposed in this section.

The level of the factor can be low, medium or high in the project. Unadjusted Value of the Story-point (UV) is provided according to the level of the factor. At low level UV is 1, at medium level UV is 6 and at High level UV is 10 as shown in table 2.

                                     Table 2: Hypothetical values of Story-points

S.No	Hypothetical values of Story-points
	Unadjusted Value(UV)	Level(L)
1.	10	3
2.	6	2
3.	1	1

Various People and Project related Factors are often unpredictable and unexpected. They may decelerate or accelerate the project and affects cost and time of the project. The factors which affect the most are  shown in table 3.

                                     Table 3: Project and people-Related factors

S.No	Related Factors
	Project-Related Factors	People-Related Factors
1	Type of Project	Communication Skill
2	Quality Requirement	Familiarity in Team
3	Hardware and Software Requirement	Managerial Skill
4	Ease of Operation	Security
5	Complexity	Working Time
6	Data Transaction	Experience of Previous Project
7	Multiple Site	Technical Ability

The above people and project-related factors and regression testing both impacts on the estimation. So an estimation algorithm based on regression testing efforts is proposed and implemented.

5.1 Proposed Algorithm

·         Identify the total number of user-stories(US) and story points(SP) per user story and compute Baseline Story Point (BSP)

                 BSP=US*SP

·         Identify the People-related and Project-related factor which affects the story points in agile environment where P= {p_1, p₂,....p_i...,p₁₄ }, where  1< i <=14

·         Identify the level set L for all factors where L={1,2,3}

·         Assign the values to each factor according to the level of factor.

o    If L=1 then Value = 1 ,

o    If L=2 then Value = 6 ,

o    If L=3 then Value = 10

·         Compute the Estimated Story Points(ESP) as

ESP=BSP+0.1(UV)

·         Compute the Velocity from first iteration as

 V = Story point completed in one iteration/ Story point in one user story.

·         Assign Time Rate (TR) and Velocity Rate (VR) to each of the factor based on the project and team.

o    TR= {tr₁, tr₂...tr_i...tr₁₄} where 0<=i<=3

o    VR= {vr₁,vr₂…vr_i… vr₁₄) where 1<=i<= 0.93

·         Compute the Decelerated Velocity by considering various factors to optimize the velocity

          DV= V*VR

·         Compute the Estimated time required for the project

o    Estimated Development Time (EDT) = ESP/Velocity (in Days)

o    Estimated Regression Testing Time (ETT) =å (T_env + T_rts + T_rtp + T_rg + T_ra) where T_env Environment setup for testing, T_rts Regression test Selection, T_rtp Regression test Prioritization, T_rg Test Report Generation , T_ra Regression Test Analysis

·         Compute Total Estimated Time (TET)

               TET=EDT+ETT

·         Compute Estimated Regression Testing Effort(ERTE) 

        ERTE = Regression Testing Effort per Iteration*(No. of iterations-1)

·         Compute Regression Testing Time (RTT)=Regression testing time in one iteration

·         Compute Total Estimated Effort (TEE)

        TEE=ESP+ERTE

·         Compute Total Estimated Time (TET)

               TET=EDT+ETT+RTT

·         Compute Total Estimated Cost (TEC)

             TEC=TEE*Cost per story-point

 6. Case Study

In this section, a small project of web-base application with a total of 30 user-stories has been considered. The various inputs are as below:

 Inputs:

No. of user stories (US) = 30

No. of story-points in one user-story (SP) = 10 SP

BSP=US*SP=300 

No. of story point completed by the team in one iteration = 50 SP

No. of days in one iteration (iteration time) = 10 days

No. of working days per month = 22 days

No. of working hours per day = 8 hrs.

Regression effort per iteration=10 SP

Cost per story-point=50 Rs.

In this case study all the factors have been considered at medium level. Time Rate and velocity factor refers to a numerical quantity that relatively represents the effect of the attribute on the time of completion and velocity respectively. Hypothetical Values of proposed factors at Level 2 are given in Table 4.

                        Table 4: Hypothetical Values of Proposed Factors at Level 2

	Type of Factors	Level	UV	VF	TR
1.	Type of project	2	6	.98	2
2.	Quality req.	2	6	.97	2
3.	H/W and S/W Requirements	2	6	.98	1
4.	Ease of Operation	2	6	.99	2
­5.	Complexity	2	6	.97	1
6.	Data transaction	2	6	.98	1
7.	Technical ability	2	6	.97	2
8.	Tool availability	2	6	.98	2
9.	Multiple site	2	6	.99	1
10.	Communication skill	2	6	.98	1
11.	Familiarity in team	2	6	.97	2
12.	Managerial skill	2	6	.97	2
13.	Security	2	6	.98	2
14.	Working Time	2	6	.99	2

 Regression Test  Strategy

The basic purpose of the regression test is to ensure that functions in previous deliveries are not adversely affected by new code. After each iteration, regression testing is needed as each iteration incrementally adds more time and effort. So the average time required for various regression testing activities is considered based on historical data as shown in table 5.

Table 5. Average Time Required for Regression Testing Activities in a small project

Regression Testing Activity	Avg. Estimated Time
Environment setup for testing	3 Days
Regression test Selection	1 Day
Regression test Prioritization	2 Days
Test Report Generation	9 Days
Regression Test Analysis	2 Days
Bug fixing	1 Day
ETT	18 Days

 Evaluation and results by implementing the proposed algorithm

·         Unadjusted Value (UV) = All factors  at medium level and  total of 14 factors UV=6*14= 84
·         Estimated Story Points (ESP) =BSP+0.1(UV) =300+0.1(84) =308.4
·         Initial Velocity = Story point completed in one iteration/ Story point in one user story =5 SP / Day
·         VF=Average of VF of all the 14 factors=0.96
·         Decelerated Velocity (DV) =V*VF=5*0.96=4.80 SP/ Day
·         Estimated Development Time (EDT) =ESP/DV=64.25 Days
·         Estimated Testing Time(ETT) = 18 Days
·         Total Estimated Time TEE=EDT+ETT=64.25+18=82.25 Days
·         No. of Iterations=TEE/Time for one iteration=8.225 iteration
·         Estimated Regression Effort (ERTE)=10SP*(8.225-1) =72.25 SP
·         Estimated Effort (EE) = ESP+ERTE=308.4+72.25=380.65 SP
·         Cost = EE*cost per story-point=380.65*50=19032.50 Rs.

7. Conclusion

In this paper an Agile Software Estimation technique based on Regression testing has been proposed and implemented by considering a small web-application. By this method Cost, Effort and Duration of small and medium size projects can be calculated efficiently. The proposed algorithm overcomes the challenges in Agile estimation. In the future work the other factors which affect the estimation most can be added; thereby having the estimation more efficient.

References

[1] Cockburn, “Agile Software Development,” Pearson Education, Asia Low Price Edition, 2007.       

[2] S. Bhalereo,Maya Ingle,”Incorporating Vital Factors In Agile Estimation Through Algorithmic Methods”, International Journal of Computer Science and Applications, 2009 Vol. 6, No. 1, pp. 85 – 97.

[3] Rashmi Popli, Naresh Chauhan,” A Sprint-point Based Estimation in Scrum” IEEE International conference Information Systems and Computer networks 9^th-10^th march 2013 at GLA University

[4] Rashmi Popli, Naresh Chauhan,” Research Challenges of Agile Estimation” Journal of Intelligent Computing and Applications” (July-December2012)

[5]Mike Cohn, “Agile Estimating and Planning" Copyright 2005 Addison-Wesley.

[6] P. Abrahamsson, Koskela, J., "Extreme Programming: A Survey of Empirical Data from a Controlled Case Study", Proceedings of International Symposium on Empirical Software Engineering, pp. 73-82, 2004.

[7] Cohn, M. (2004). User stories applied: For agile software development. Boston, MA: Addison-Wesley

[8] F. Maurer and S. Martel, "Extreme Programming: Rapid Development for Web-Based Applications", IEEE Internet Computing, 6(1), pp. 86-91, Jan/Feb 2002.

[9] L. Williams, W. Krebs, L. Layman, A. Antón, and P. Abrahamsson, "Toward a Framework for Evaluating Extreme Programming", Proceedings of Empirical Assessment in Software Engineering. (EASE)2004,Edinburgh,Scot.,pp.11-20,2004.

[10] F. J. Heemstra, “Software cost estimation”, Information and Software Technology, vol. 34, no.10, 1992,pp.627-639.

[11] J. Hihn and H. Habib-Agahi, “Cost estimation of software intensive projects: a survey of current practices”, International Conference on Software Engineering, 1991, pp. 276-287.                                         

[12]G. R. Finnie, G. E. Wittig, “AI tools for software development effort estimation”, Software Engineering and Education and Practice Conference, IEEE Computer Society Press, pp. 346-353, 1996. [13]M. Jørgensen, " Estimation of Software Development Work Effort: Evidence on Expert Judgment and Formal Models," forthcoming in International Journal of Forecasting, 2006.