Evaluating Engineering Department’s Website Maintainability



A Comparative Evaluation of Maintainability:

A Study of Engineering Department’s Website Maintainability

|Nary Subramanian* |Richard Puerzer |Lawrence Chung |

|Dept. of Computer Science |Dept. of Engineering |Dept. of Computer Science |

|University of Texas at Tyler |Hofstra University |University of Texas at Dallas |

|Tyler, TX |Hempstead, N.Y. |Richardson, TX |

|koolnary@ |eggrjp@hofstra.edu |chung@utdallas.edu |

We have been maintaining the Engineering Department’s website at Hofstra University for the past five years and our experience has been a mixed one: on the one hand it usually takes a few days to make even minor modifications to the website while on the other, the chain of checks and balances ensures that the updates work and the accuracy of the new information is high. We thought it will be an interesting and useful exercise to analyze and qualitatively evaluate the website maintenance policy at Hofstra compared to other universities so that this information will help all concerned stakeholders to make modifications to the policy, if needed. We used the NFR Framework for the analysis and evaluation, and compared the closely related NFRs (or Non-Functional Requirements) of maintainability and cost (both, time and money costs), for the three different website maintenance policies used in the academia including that of Hofstra Engineering Department, and concluded that while there are tradeoffs involved, small changes in Hofstra’s policies could significantly increase the maintainability of the Engineering Department website.

Keywords: Website, maintainability, NFR Framework

1. Introduction

Website maintenance could involve a high investment of time and money for the institution that owns the website. Moreover, in order to have some degree of control on the maintenance activity most institutions have some form of policy for maintaining websites. While different policies have their plus and minus points, we feel that an evaluation of the maintainability afforded by a policy will be of great help to the policy makers by providing a qualitative measure of their policies. We have been maintaining the website for Hofstra Engineering Department (HED) at hofsta.edu/ engineering for the past five years and our experience with the procedure for updating the HED website has been mixed: we have been alternately frustrated and enjoyed the experience. This has been due to various factors: availability of correct data on time, ease with which we have been able to update the pages using the Hofstra technology, the time

______________________________________

* This work was done when this author was a member of the faculty of Department of Engineering, Hofstra University.

taken for the approval process through the pipeline, and so on. Therefore, while it usually takes a few days to make even minor modifications to the website while on the other, the chain of checks and balances ensures that the updates work and the accuracy of the new information is high, and since the maintenance is performed by faculty during their spare time, the cost to the department is kept to a minimum.

In this paper we present the qualitative evaluation of the maintainability of HED website using the NFR Framework [1, 2], where NFR stands for Non-Functional Requirement – the maintainability of the current process in HED is systematically evaluated and suggestions provided on how the maintainability may be improved. The maintainability process at HED is contrasted with other website processes used in the academia and this helps us to compare and contrast the HED process. While comparing maintenance models used in the academia we used the closely related NFRs of maintainability and cost – a complex maintenance process involves higher cost both in terms of time and money, while a less costly process may negatively affect maintainability of the website.

One approach to maintainability evaluation in the literature is based on metrics: thus [7] develops a set of metrics for estimating web engineering effort – these metrics are mostly based on time taken for a particular activity; mean-time-to-change (MTTC) is also used as a measure of maintainability of software in general [9] (see page 630 of [9]); and the IEEE Standard 982.1-1988 [10] suggests the usage of software maturity index to evaluate the maintainability (more appropriately, stability) of software. In [11] an accessibility framework is proposed that evaluates web accessibility using a set of accessibility factors and it has been suggested that maintainability may be similarly evaluated. The Software Engineering Institute has proposed a maintainability index (MI) that is based on a polynomial that includes Halstead volume, cyclomatic complexity, and lines of code [12]. Our work differs from the above in the following ways: we have made a comparative evaluation and analysis of the website maintenance models used in the academia using the NFR Framework.

Section 2 discusses some of the website maintenance models used in academia including the process at HED, Section 3 applies the NFR Framework for analyzing and evaluating the HED model, Section 4 presents the observations from this analysis, while Section 5 presents the conclusions and directions for future research. In this paper we have used the phrases maintenance process, maintenance policy, and maintenance model synonymously.

2. Website Maintenance Processes

The HED website maintenance process [3] is given in Figure 1. An engineering department faculty member serves

[pic]Figure 1. The Hofstra Engineering Department Website Maintenance Model

as the website developer and maintainer. His/her job is to create new web pages or modify existing pages – the requests come from faculty, from students to update a student organization webpage on the HED, and from the department itself for creating (e.g., a new lab page) or modifying existing pages. The developer/maintainer creates/updates the webpage by starting a task flow in the Hofstra web maintenance system (called WMX). Once the developer/maintainer creates/updates the webpage(s) using WMX, the task flow is modified to that of the review and assigned to the department reviewer, an activity performed by another department faculty member. The reviewer either approves the task in which case the reviewer assigns the task to the next step, or the reviewer may find errors in the task (such as non-conformance to department standards and regulations) and assigns it back to the developer with comments. The reviewer may assign the task (in case of acceptance) either to the University Web Editor (the usual case) or to the Engineering Department Chairman (in rare cases). In case the task is assigned to the University web editor, that person does a feasibility and conformance check on the page(s) – the page(s) should not have too much data such as video (so that bandwidth is not consumed) and the page should conform to University regulations. In case of non-acceptance by the University web editor the task is reassigned to the HED developer/maintainer; else the page is made live. In the case when the HED reviewer assigns the task to the HED Chairman the latter either confirms existing data or provides clarifications and reassigns the task to the reviewer in both acceptance and rejection cases. As can be seen that the task approval process is a (maximum) four step process that ensures that all errors are removed, and that all departmental and university regulations are met. While this process is systematic, this four-step process could take anywhere between a couple of days to a few months since all the work in the department is done by full time faculty members during their spare time, and sometimes not all data is available to begin with. Moreover, this four-step process will have to be followed for even minor modifications such as spelling/grammar errors, incorrect links, or wrong data. Therefore, maintainability does not seem high.

In order to compare and contrast HED process, we considered two other processes followed in the academia – one in which department has a full time staff to maintain web pages (which we call Full Time Staff Model or FTS Model) and the other in which a centralized University web team (which we call Centralized University Web Team Model or CUWT Model) does all the maintenance. Both these models are given in Figure 2 and Figure 3. In the FTS model each department has a team of full time staff to maintain departmental websites (for example, the University of Texas at Dallas, Computer Science department). In the CUWT model there exists a team of web maintenance professionals who help out all the departments in the academic institution (for example, [15], [16], [17]).

[pic][pic]

Figure 2. The FTS and CUTW Maintenance Models

3. Evaluating Website Maintenance Model

The various models of web maintenance were compared for their maintainability using the NFR Framework [1, 2, 4, 5]. The Softgoal Interdependency Graph that results from the application of the NFR Framework is shown in Figure 3 (details given in the original version of this paper). Using the label propagation rules of the NFR Framework it can be determined that while the root NFR softgoal Maintainability [Engineering Website] is denied, the root NFR softgoal Cost [Maintaining, Engineering Website] is satisficed. Therefore, the HED website maintenance model denies maintainability, that is, it is not highly maintainable, but it is cost effective.

By observing Figure 3 we can determine the reasons for poor maintainability of the HED model: several leaf NFR softgoals (including the critical softgoals) are denied. The maintainability of the HED model can be improved by any one (or more) of the following means:

[pic]

Figure 3. Complete SIG for HED Model

1. The NFR softgoals Time[Creating New Page] and Time[Approval, New Page] are both satisficed – then using the propagation rules, we can determine that the root NFR softgoal Maintainability[Engineering Website] will be satisficed. This means that the time to create a new page should be made favorable to the maintainers (e.g., by automation) and the time for approving a new page should be reduced (e.g., by reducing approval stages) – both of these measures will help improve HED website maintainability.

2. The NFR softgoals Time[Modifying Page] and Time[Approval, Modification] are both satisficed and this will satisfice the root NFR softgoal Maintainability[Engineering Website]. This means that if the time to modify a page (e.g., by automation) and the time taken to approve modifications (e.g., by reducing approval stages) are both reduced then the maintainability of HED website will improve.

We can see the potential of the NFR Framework in evaluating the maintainability of the HED model as well as in analyzing the reasons for the poor (or good) maintainability.

5. Observations

From the application of the NFR Framework, we determined that the HED website maintenance model is not easy to maintain but cost effective; and we also determined what changes to the policy need to be implemented to make the process more maintainable. How do the other maintenance models (Figure 2) fare with respect to maintainability and cost effectiveness? This can be answered from the SIG of Figure 4 (this is a partial SIG – for succinctness sake we have shown only the leaf NFR softgoals; the other NFR softgoals can be seen from Figure 3). By applying the propagation rules, we will be able to determine that the FTS model is highly maintainable but expensive to the department, while for the CUTW model the availability of data is a serious concern – if data is incomplete it usually takes a long time to get the data from the concerned entity (faculty, student or the department) – hence, the CUTW scores poorly on maintainability but is very good in terms of cost effectiveness. Based on our analysis of the maintainability of the HED website we suggest the modified process for HED in Figure 5 that will improve the maintainability without much effect on the cost and elements of this modified process used in Figure 4.

[pic] Figure 4. Partial SIGs for the FTS, CUTW, and Modified HED Models

5. Conclusions

Our experience with maintaining the Hofstra Engineering Department (HED) website convinced us that the maintainability of the process used to maintain web pages was low. However, we wished to analyze and evaluate the process and compare it with other maintenance processes used in the academia – the full-time staff model (FTS model) that uses a dedicated departmental staff to maintain the departmental website, and the centralized university web team model (CUWT model) that uses a central web maintenance team to maintain websites of all departments.

[pic]

Figure 5. Modified HED Maintenance Model

Using the NFR Framework [1, 2] we determined that the maintainability of the HED model was not good but cost effective. The FTS model has better maintainability but not as cost effective as the HED model, while the CUWT is cost effective but not as maintainable as the FTS process. We analyzed the reasons for the poor maintainability of the HED process and suggested modifications to the process to improve the maintainability – it turns out that our suggestions coincide with some of the improvements made to the HED model recently.

However, there are several areas for further study – for example, we have not compared the professionalism of web updates, the accuracy of the models, and the satisfaction of the users; we would like to extend the qualitative analysis to quantitative analysis on the lines of [18]; and we could extend this approach to evaluate the maintainability of the processes used in the industry [7, 17]. However, we believe that the process of maintainability analysis and evaluation adopted in this paper is practical and useful to stakeholders involved in the maintenance of web-based systems both in the industry and academia.

Acknowledgements

We would like to thank Ms. Lalitha Krishnan, Web Site Manager, Cold Spring Harbor Labs, New York, for reviewing the original version of this paper and providing valuable feedback. We would also like to thank the anonymous reviewers of the original version of this paper for providing excellent comments and suggestions.

References

1. L. Chung, B. A. Nixon, E. Yu, and J. Mylopoulos, Non-Functional Requirements in Software Engineering, Kluwer Academic Publishers, Boston, 2000.

2. J. Mylopoulos, L. Chung, and B. Nixon, “Representing and Using Nonfunctional Requirements: A Process-Oriented Approach”, IEEE Transactions on Software Engineering, Vol. 18, No. 6, June 1992, pp. 483-497.

3. WebTeam Monthly, February 2005, Volume 1, Issue 3, The Monthly Journal of Web Development at Hofstra University.

4. L. Chung and N. Subramanian, “Architecture-Based Semantic Evolution: A Study of Remotely Controlled Embedded Systems”, Proc. Int. Conf. on Software Maintenance, IEEE Computer Press, Florence, Italy, Nov., 2001, pp. 663-666.

5. L. Chung and N. Subramanian, “Architecture-based Semantic Evolution for Embedded Systems: A Study of Remotely Controlled Systems”, Journal of Software Maintenance and Evolution: Research and Practice, 15(3), May/June 2003, pp. 145 - 190.

6. E. Mendes, N. Mosley, and S. Counsell, “Estimating Design and Authoring Effort”, IEEE Multimedia, January-March 2001, pp. 50-57.

7. W. W. Vasconcelos and J. Cavalcanti, “Agent-Based Web Site Maintenance”, Technical Report AUCS/TR0401, Department of Computing Science, Univ. of Aberdeen, UK, Nov. 2003.

8. R. S. Pressman, Software Engineering: A Practitioner’s Approach, McGraw Hill, Sixth Edition, 2005.

9. Software Engineering Standards, 1994 Edition, IEEE, 1994.

10. C. Boldyreff, “Determination and Evaluation of Web Accessibility”, Proc. 11th Int. Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, IEEE Computer Press, Pittsburg, USA, June 2002, pp. 35 – 42.

11. “Maintainability Index Technique for Measuring Program Maintainability”, Software Engineering Institute, available at , March 2002.

12. G. Brajnik, “Automatic Web Usability Evaluation: What needs to be done?”, Sixth Conference on Human Factors and the Web, June 2000, available at /hfweb/brajnik/hfweb-brajnik.html.

13. utdallas.edu

14. swarthmore.edu

15. richlandcollege.edu

16.

17. M. Kajko-Mattsson, “Towards a Business Maintenance Model”, Proc. Int. Conf. on Software Maintenance, IEEE Computer Press, Florence, Italy, Nov., 2001, pp. 500-509.

18. L. Chung and N. Subramanian, “Process-Oriented Metrics for Software Architecture Adaptability”, Proc. Int. Symp. on Requirements Engineering, IEEE Computer Press, Aug-Sep. 2001, pp. 310-311.

19. N. Subramanian, Adaptable Software Architecture Generation Using The NFR Approach, Ph.D. Thesis, The University of Texas at Dallas, 2003.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download