- Department of Computer Science,
University College London,
Malet Place, London, WC1E 6BT, UK.
Mark Harman
University College London, Computer Science, Faculty Member
Abstract Side-effects are widely believed to impede program comprehension and have a detrimental effect upon software maintenance. This paper introduces an algorithm for side-effect removal which splits the side-effects into their pure... more
Abstract Side-effects are widely believed to impede program comprehension and have a detrimental effect upon software maintenance. This paper introduces an algorithm for side-effect removal which splits the side-effects into their pure expression meaning and their state-changing meaning. Symbolic execution is used to determine the expression meaning, while transformation is used to place the state-changing part in a suitable location in a transformed version of the program.
Abstract This paper introduces the concept of test suite latency. The more latent a test suite, the more it is possible to repeatedly select subsets that achieve a test goal (such as coverage) without re-applying test cases. Where a test... more
Abstract This paper introduces the concept of test suite latency. The more latent a test suite, the more it is possible to repeatedly select subsets that achieve a test goal (such as coverage) without re-applying test cases. Where a test case is re-applied it cannot reveal new information. The more a test suite is forced to re-apply already applied test cases in order to achieve the test goal, the more it has becomeworn out'. Test suite latency is the flipside of wear out; the more latent a test suite, the less prone it is to wear out.
It is said 90% of faults that survive manufacturer's testing procedures are complex. That is, the corresponding bug fix contains multiple changes. Higher order mutation testing is used to study defect interactions and their impact on... more
It is said 90% of faults that survive manufacturer's testing procedures are complex. That is, the corresponding bug fix contains multiple changes. Higher order mutation testing is used to study defect interactions and their impact on software testing for fault finding. We adopt a multi-objective Pareto optimal approach using Monte Carlo sampling, genetic algorithms and genetic programming to search for higher order mutants which are both hard-to-kill and realistic.
SUMMARY Service-oriented architecture (SOA) is gaining momentum as an emerging distributed system architecture for business-to-business collaborations. This momentum can be observed in both industry and academic research. SOA presents new... more
SUMMARY Service-oriented architecture (SOA) is gaining momentum as an emerging distributed system architecture for business-to-business collaborations. This momentum can be observed in both industry and academic research. SOA presents new challenges and opportunities for testing and verification, leading to an upsurge in research.
Abstract Requirements engineering for multiple customers, each of whom have competing and often conflicting priorities, raises issues of negotiation, mediation and conflict resolution. This paper uses a multi-objective optimisation... more
Abstract Requirements engineering for multiple customers, each of whom have competing and often conflicting priorities, raises issues of negotiation, mediation and conflict resolution. This paper uses a multi-objective optimisation approach to support investigation of the trade-offs in various notions of fairness between multiple customers. Results are presented to validate the approach using two real-world data sets and also using data sets created specifically to stress test the approach.
Abstract. The omnipresence of software graphs as useful intermediate representations means that the identification of near-match subgraphs (Error-Correcting Subgraph Isomorphism) has diverse and widespread applications in software... more
Abstract. The omnipresence of software graphs as useful intermediate representations means that the identification of near-match subgraphs (Error-Correcting Subgraph Isomorphism) has diverse and widespread applications in software engineering, such as querying, clone detection and model checking. Each software engineering subarea has developed specific tailored approaches to subgraph isomorphism, thereby reducing comparability and generality, and potentially yielding sub-optimal results.
ABSTRACT There has recently been a great deal of interest in search–based test data generation, with many local and global search algorithms being proposed. However, to date, there has been no investigation of the relationship between the... more
ABSTRACT There has recently been a great deal of interest in search–based test data generation, with many local and global search algorithms being proposed. However, to date, there has been no investigation of the relationship between the size of the input domain (the search space) and performance of the search–based algorithms. Static analysis can be used to remove irrelevant variables for a given test data generation problem, thereby reducing the search space size.
Abstract This paper addresses the question:" How can animated visualisation be used to express interesting properties of static analysis?" The particular focus is upon static dependence analysis, but the approach adopted in the paper is... more
Abstract This paper addresses the question:" How can animated visualisation be used to express interesting properties of static analysis?" The particular focus is upon static dependence analysis, but the approach adopted in the paper is applicable to other forms of static analysis. The challenge is twofold. First, there is the inherent difficultly of using animation, which is inherently dynamic, as a representation of static analysis, which is not. The paper shows one way in which this apparent contradiction can be overcome.
A program schema defines a class of programs, all of which have identical statement structures, but whose expressions may differ. We prove that given any two structured schemas which are conservative, linear and free, it is decidable... more
A program schema defines a class of programs, all of which have identical statement structures, but whose expressions may differ. We prove that given any two structured schemas which are conservative, linear and free, it is decidable whether they are equivalent.
Abstract Optimising programs for non-functional properties such as speed, size, throughput, power consumption and bandwidth can be demanding; pity the poor programmer who is asked to cater for them all at once! We set out an alternate... more
Abstract Optimising programs for non-functional properties such as speed, size, throughput, power consumption and bandwidth can be demanding; pity the poor programmer who is asked to cater for them all at once! We set out an alternate vision for a new kind of software development environment inspired by recent results from Search Based Software Engineering (SBSE).
Abstract This paper presents techniques to integrate boundary overlap into concept assignment using Plausible Reasoning. Heuristic search techniques such as Hill climbing and Genetic Algorithms are investigated. A new fitness measure... more
Abstract This paper presents techniques to integrate boundary overlap into concept assignment using Plausible Reasoning. Heuristic search techniques such as Hill climbing and Genetic Algorithms are investigated. A new fitness measure appropriate for overlapping concept assignment is introduced. The new algorithms are compared to randomly generated results and the Genetic Algorithm is shown to be the best of the proposed search algorithms in terms of the quality of concept binding, as measured by the fitness function.
ABSTRACT Companies such as Google tend to develop products from one continually evolving core of code. Software is neither shipped, nor released in the traditional sense. It is simply made available, with dramatically compressed release... more
ABSTRACT Companies such as Google tend to develop products from one continually evolving core of code. Software is neither shipped, nor released in the traditional sense. It is simply made available, with dramatically compressed release cycles regression testing. This large scale rapid release environment creates challenges for the application of regression test optimisation techniques.
This paper presents an approach to Search Based Software Project Management based on Cooperative Co-evolution. Our approach aims to optimize both developers' team staffing and work package scheduling through cooperative co-evolution to... more
This paper presents an approach to Search Based Software Project Management based on Cooperative Co-evolution. Our approach aims to optimize both developers' team staffing and work package scheduling through cooperative co-evolution to achieve early overall completion time. To evaluate our approach, we conducted an empirical study, using data from four real-world software projects. Results indicate that the Co-evolutionary approach significantly outperforms a single population evolutionary algorithm.
Abstract A dependence cluster is a set of program statements, all of which are mutually inter-dependent. This article reports a large scale empirical study of dependence clusters in C program source code. The study reveals that large... more
Abstract A dependence cluster is a set of program statements, all of which are mutually inter-dependent. This article reports a large scale empirical study of dependence clusters in C program source code. The study reveals that large dependence clusters are surprisingly commonplace. Most of the 45 programs studied have clusters of dependence that consume more than 10% of the whole program. Some even have clusters consuming 80% or more.
Abstract Pair-wise comparison has been successfully utilised in order to prioritise test cases by exploiting the rich, valuable and unique knowledge of the tester. However, the prohibitively large cost of the pair-wise comparison method... more
Abstract Pair-wise comparison has been successfully utilised in order to prioritise test cases by exploiting the rich, valuable and unique knowledge of the tester. However, the prohibitively large cost of the pair-wise comparison method prevents it from being applied to large test suites. In this paper, we introduce a cluster-based test case prioritisation technique. By clustering test cases, based on their dynamic runtime behaviour, we can reduce the required number of pair-wise comparisons significantly.
Abstract Generating realistic test data is a major problem for software testers. Realistic test data generation for certain input types is hard to automate and therefore laborious. We propose a novel automated solution to test data... more
Abstract Generating realistic test data is a major problem for software testers. Realistic test data generation for certain input types is hard to automate and therefore laborious. We propose a novel automated solution to test data generation that exploits existing web services as sources of realistic test data. Our approach is capable of generating realistic test data and also generating data based on tester-specified constraints.
Abstract A slice is constructed by deleting statements from a program whilst preserving some projection of its semantics. Since Mark Weiser introduced program slicing in 1979, a wide variety of slicing paradigms have been proposed, each... more
Abstract A slice is constructed by deleting statements from a program whilst preserving some projection of its semantics. Since Mark Weiser introduced program slicing in 1979, a wide variety of slicing paradigms have been proposed, each of which is based upon a new formulation of the slicing criterion, capturing the semantic projection to be preserved during the process of command deletion. This paper surveys these slicing criteria, attempting to establish a set of parameters which combine to form a slicing criterion.
Abstract Test case prioritisation techniques aim to maximise the chance of fault detection as early in testing as possible. This is most commonly achieved by prioritising the tests according to a surrogate measure that is thought to... more
Abstract Test case prioritisation techniques aim to maximise the chance of fault detection as early in testing as possible. This is most commonly achieved by prioritising the tests according to a surrogate measure that is thought to correspond to fault detection capabilities, such as code coverage. However, once the prioritised test suite indeed detects a fault, the original prioritisation may become obsolete.
Abstract This paper introduces an approach to web application regression testing, based upon repair of user session data. The approach is entirely automated. It consists of a white box examination of the structure of the changed web... more
Abstract This paper introduces an approach to web application regression testing, based upon repair of user session data. The approach is entirely automated. It consists of a white box examination of the structure of the changed web application to detect changes and a set of techniques to map these detected changes onto repair actions. The paper reports the results of experiments that explore both the performance and effectiveness of the approach.
Abstract Like other engineering disciplines, software engineering is typically concerned with near optimal solutions or those which fall within a specified applicable tolerance. More recently, search-based techniques have started to find... more
Abstract Like other engineering disciplines, software engineering is typically concerned with near optimal solutions or those which fall within a specified applicable tolerance. More recently, search-based techniques have started to find application in software engineering problem domains. This area of search-based software engineering has its origins in work on search-based testing, which began in the mid 1990s. Already, search-based solutions have been applied to software engineering problems right through the development life cycle.
This paper provides a motivation for the application of search based optimization to Software Engineering, an area that has come to be known as Search Based Software Engineering (SBSE). SBSE techniques have already been applied to many... more
This paper provides a motivation for the application of search based optimization to Software Engineering, an area that has come to be known as Search Based Software Engineering (SBSE). SBSE techniques have already been applied to many problems throughout the Software Engineering lifecycle, with new application domains emerging on a regular basis. The approach is very generic and therefore finds wide application in Software Engineering.
Abstract This paper uses a multi-objective optimisation approach to support investigation of the trade-offs in various notions of fairness between multiple customers. Results are presented to validate the approach using two real-world... more
Abstract This paper uses a multi-objective optimisation approach to support investigation of the trade-offs in various notions of fairness between multiple customers. Results are presented to validate the approach using two real-world data sets and also using data sets created specifically to stress test the approach. Simple graphical techniques are used to visualize the solution space.
Abstract—Search Based testing has proved effective at generating test data to cover targeted branches and has consequently received a great deal of attention from the automated software testing community. However, previous approaches to... more
Abstract—Search Based testing has proved effective at generating test data to cover targeted branches and has consequently received a great deal of attention from the automated software testing community. However, previous approaches to search based test data generation do not take account of oracle cost.
Abstract Companies such as Google tend to develop products from one continually evolving core of code. Software is neither shipped, nor released in the traditional sense. It is simply made available, with dramatically compressed release... more
Abstract Companies such as Google tend to develop products from one continually evolving core of code. Software is neither shipped, nor released in the traditional sense. It is simply made available, with dramatically compressed release cycles regression testing. This large scale rapid release environment creates challenges for the application of regression test optimisation techniques.
Of the music representations previously addressed by such work, patches written in visual languages such as Pure Data [1] and Max/MSP [2] are not usually considered, yet they represent a class of music representation where the generative... more
Of the music representations previously addressed by such work, patches written in visual languages such as Pure Data [1] and Max/MSP [2] are not usually considered, yet they represent a class of music representation where the generative elements can have a close and often deterministic relationship with the produced sound.
Test case prioritization techniques seek to maximise early fault detection. Fault localization seeks to use test cases already executed to help find the fault location. There is a natural interplay between the two techniques; once a fault... more
Test case prioritization techniques seek to maximise early fault detection. Fault localization seeks to use test cases already executed to help find the fault location. There is a natural interplay between the two techniques; once a fault is detected, we often switch focus to fault fixing, for which localization may be a first step. In this paper we introduce the Fault Localization Prioritization (FLP) problem, which combines prioritization and localization.
Abstract—Software Engineering and development is wellknown to suffer from unplanned overtime, which causes stress and illness in engineers and can lead to poor quality software with higher defects. In this paper, we introduce a... more
Abstract—Software Engineering and development is wellknown to suffer from unplanned overtime, which causes stress and illness in engineers and can lead to poor quality software with higher defects. In this paper, we introduce a multi-objective decision support approach to help balance project risks and duration against overtime, so that software engineers can better plan overtime.
Abstract Existing automated test data generation techniques tend to start from scratch, implicitly assuming that no pre-existing test data are available. However, this assumption may not always hold, and where it does not, there may be a... more
Abstract Existing automated test data generation techniques tend to start from scratch, implicitly assuming that no pre-existing test data are available. However, this assumption may not always hold, and where it does not, there may be a missed opportunity; perhaps the pre-existing test cases could be used to assist the automated generation of additional test cases.
Abstract We describe an approach which mixes testing, slicing, transformation and formal verification to investigate speculative hypotheses concerning a program, formulated during program comprehension activity. Our philosophy is that... more
Abstract We describe an approach which mixes testing, slicing, transformation and formal verification to investigate speculative hypotheses concerning a program, formulated during program comprehension activity. Our philosophy is that such hypotheses (which are typically undecidable) can, in some sense, beanswered'by a partly automated system which returns neithertrue'norfalse'but a program (thetest program') which computes the answer.
Abstract Program transformation is useful in a number of applications including program comprehension, reverse engineering and compiler optimization. In all these applications, transformation algorithms are constructed by hand for each... more
Abstract Program transformation is useful in a number of applications including program comprehension, reverse engineering and compiler optimization. In all these applications, transformation algorithms are constructed by hand for each different transformation goal. Loosely speaking, a transformation algorithm defines a sequence of transformation steps to apply to a given program. It is notoriously hard to find good transformation sequences automatically, and so much (costly) human intervention is required.
Abstract Automated web crawlers can be used to explore and exercise portions of a web application under test. However, the possibility to achieve full exploration of a web application through automated crawling is severely limited by the... more
Abstract Automated web crawlers can be used to explore and exercise portions of a web application under test. However, the possibility to achieve full exploration of a web application through automated crawling is severely limited by the choice of the input values submitted with forms. Depending on the crawler's capabilities, a larger or smaller portion of web application will be automatically explored.
Abstract. Stream X-machines have been used in order to specify a range of systems. One of the strengths of this approach is that, under certain well-defined conditions, it is possible to produce a finite test that is guaranteed to... more
Abstract. Stream X-machines have been used in order to specify a range of systems. One of the strengths of this approach is that, under certain well-defined conditions, it is possible to produce a finite test that is guaranteed to determine the correctness of the implementation under test (IUT). Initially only deterministic stream X-machines were considered in the literature. This is largely because the standard test algorithm relies on the stream X-machine being deterministic.
Abstract Program slicing is a source code extraction technique that can be used to support reverse engineering by automatically extracting executable subprograms that preserve some aspect of the original program's semantics. Although... more
Abstract Program slicing is a source code extraction technique that can be used to support reverse engineering by automatically extracting executable subprograms that preserve some aspect of the original program's semantics. Although minimal slices are not generally computable, safe approximate algorithms can be used to good effect. However, the precision of such slicing algorithms is a major factor in determining the value of slicing for reverse engineering.
ABSTRACT This paper introduces a messy-GA for transition coverage of Simulink/StateFlow models. We introduce a tool that implements our approach and evaluate it on three benchmark embedded system Simulink models. Our messy-GA is able to... more
ABSTRACT This paper introduces a messy-GA for transition coverage of Simulink/StateFlow models. We introduce a tool that implements our approach and evaluate it on three benchmark embedded system Simulink models. Our messy-GA is able to achieve statistically significantly better coverage when compared to both random search and to a commercial tool for Simulink/StateFlow model Testing.
Abstract Amorphous slicing is an automated source code extraction technique with applications in many areas of software engineering, including comprehension, reuse, testing and reverse engineering. Algorithms for syntax-preserving slicing... more
Abstract Amorphous slicing is an automated source code extraction technique with applications in many areas of software engineering, including comprehension, reuse, testing and reverse engineering. Algorithms for syntax-preserving slicing are well established, but amorphous slicing is harder because it requires arbitrary transformation; finding good general purpose amorphous slicing algorithms therefore remains as hard as general program transformation.
Abstract This paper introduces Model Projection. Finite state models such as Extended Finite State Machines are being used in an ever increasing number of software engineering activities. Model projection facilitates model development by... more
Abstract This paper introduces Model Projection. Finite state models such as Extended Finite State Machines are being used in an ever increasing number of software engineering activities. Model projection facilitates model development by specializing models for a specific operating environment. A projection is useful in many design-level applications including specification reuse and property verification.
In software development, testing and maintenance, as in other large scale engineering activities, effective project planning is essential. Failure to plan and/or poor planning can cause delays and costs that, given timing and budget... more
In software development, testing and maintenance, as in other large scale engineering activities, effective project planning is essential. Failure to plan and/or poor planning can cause delays and costs that, given timing and budget constraints, are often unacceptable, leading to business–critical failures. Traditional tools such as the Project Evaluation and Review Technique (PERT), the Critical Path Method (CPM), Gantt diagrams and Earned Value Analysis help to plan and track project milestones.
Page 1. A CLUSTERING GENETIC ALGORITHM FOR SOFTWARE MODULARISATION WITH A MULTIPLE HILL CLIMBING APPROACH By Kiarash Mahdavi SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY AT BRUNEL UNIVERSITY... more
Page 1. A CLUSTERING GENETIC ALGORITHM FOR SOFTWARE MODULARISATION WITH A MULTIPLE HILL CLIMBING APPROACH By Kiarash Mahdavi SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY AT BRUNEL UNIVERSITY WEST LONDON UXBRIDGE, MIDDLESEX, UB8 3PH, UK FEBRUARY 2005 c Copyright by Kiarash Mahdavi, 2005 Page 2.
Abstract The main goal of the seminar``Practical Software Testing: Tool Automation and Human Factors''was to bring together academics working on algorithms, methods, and techniques for practical software testing, with practitioners,... more
Abstract The main goal of the seminar``Practical Software Testing: Tool Automation and Human Factors''was to bring together academics working on algorithms, methods, and techniques for practical software testing, with practitioners, interested in developing more soundly-based and well-understood testing processes and practices. The seminar's purpose was to make researchers aware of industry's problems, and practitioners aware of research approaches.
Abstract Variable dependence is a source code analysis problem, related to slicing and chopping, in which the aim is to determine the set of variables that can affect the values of given program variables at specified points in the... more
Abstract Variable dependence is a source code analysis problem, related to slicing and chopping, in which the aim is to determine the set of variables that can affect the values of given program variables at specified points in the program. This paper describes an approach to solving the problem based on transforming program code into an intermediate language without preserving concrete semantics but keeping variable dependence relations intact.
Abstract—This paper1 makes a case for Source Code Analysis and Manipulation. The paper argues that it will not only remain important, but that its importance will continue to grow. This argument is partly based on the 'law'of tendency to... more
Abstract—This paper1 makes a case for Source Code Analysis and Manipulation. The paper argues that it will not only remain important, but that its importance will continue to grow. This argument is partly based on the 'law'of tendency to executability, which the paper introduces. The paper also makes a case for Source Code Analysis purely for the sake of analysis. Analysis for its own sake may not be merely indulgent introspection.
ABSTRACT SBSE techniques have been widely applied to requirements selection and prioritization problems in order to ascertain a suitable set of requirements for the next release of a system. Unfortunately, it has been widely observed that... more
ABSTRACT SBSE techniques have been widely applied to requirements selection and prioritization problems in order to ascertain a suitable set of requirements for the next release of a system. Unfortunately, it has been widely observed that requirements tend to be changed as the development process proceeds and what is suitable for today, may not serve well into the future.
A program schema defines a class of programs, all of which have identical statement structure, but whose expressions may differ. We define a class of syntactic similarity binary relations between linear structured schemas, which... more
A program schema defines a class of programs, all of which have identical statement structure, but whose expressions may differ. We define a class of syntactic similarity binary relations between linear structured schemas, which characterise schema equivalence for structured schemas that are linear, free and liberal. In this paper we report that similarity implies equivalence for linear schemas, and that a near-converse holds for schemas that are linear, free and liberal.
Abstract The procedure extraction problem is concerned with the meaning preserving formation of a procedure from a (not necessarily contiguous) selected set of statements. Previous approaches to the problem have used dependence analysis... more
Abstract The procedure extraction problem is concerned with the meaning preserving formation of a procedure from a (not necessarily contiguous) selected set of statements. Previous approaches to the problem have used dependence analysis to identify the non-selected statements which must be'promoted'(also selected) in order to preserve semantics. All previous approaches to the problem have been syntax preserving.
Abstract During regression testing, a modified system is often retested using an existing test suite. Since the size of the test suite may be very large, testers are interested in detecting faults in the modified system as early as... more
Abstract During regression testing, a modified system is often retested using an existing test suite. Since the size of the test suite may be very large, testers are interested in detecting faults in the modified system as early as possible during this retesting process. Test prioritization attempts to order tests for execution so that the chances of early detection of faults during retesting are increased. The existing prioritization methods are based on the source code of the system under test.
Abstract Generating realistic test data is a major problem for software testers. Realistic test data generation for certain input types is hard to automate and therefore laborious. We propose a novel automated solution to test data... more
Abstract Generating realistic test data is a major problem for software testers. Realistic test data generation for certain input types is hard to automate and therefore laborious. We propose a novel automated solution to test data generation that exploits existing web services as sources of realistic test data. Our approach is capable of generating realistic test data and also generating data based on tester-specified constraints.
SUMMARY Conditioned slicing is a source code extraction technique. The extraction is performed with respect to a slicing criterion which contains a set of variables and conditions of interest. Conditioned slicing removes the parts of the... more
SUMMARY Conditioned slicing is a source code extraction technique. The extraction is performed with respect to a slicing criterion which contains a set of variables and conditions of interest. Conditioned slicing removes the parts of the original program which cannot affect the variables at the point of interest, when the conditions are satisfied. This produces a conditioned slice, which preserves the behaviour of the original with respect to the slicing criterion.
Abstract Large clusters of mutual dependence can cause problems for comprehension, testing and maintenance. This paper introduces the concept of coherent dependence clusters, techniques for their efficient identification, visualizations... more
Abstract Large clusters of mutual dependence can cause problems for comprehension, testing and maintenance. This paper introduces the concept of coherent dependence clusters, techniques for their efficient identification, visualizations to better understand them, empirical results concerning their practical significance. As the paper will show, coherent dependence clusters facilitate a fine grained analysis of the subtle relationships between clusters of dependence.
Abstract Previous approaches to search based test data generation tend to focus on coverage, rather than oracle cost. While there may be an aspiration that systems should have models, checkable specifications and/or contract driven... more
Abstract Previous approaches to search based test data generation tend to focus on coverage, rather than oracle cost. While there may be an aspiration that systems should have models, checkable specifications and/or contract driven development, this sadly remains an aspiration; in many real cases, system behaviour must be checked by a human. This painstaking checking process forms a significant cost, the oracle cost, which previous work on automated test data generation tends to overlook.
