Given a data set and n random perturbations on this data, if the index is robust, all (or most) perturbations would yield the same general ranking. Therefore, in the context of conservation in an optimal situation, we would prefer areas that:
1. Have the same position in the ranking (original and re-sampled), no matter if we delete areas, species, or phylogenies
= same ranking or position, insensitive to changes in the item(s) deleted.
2. if not, at least must be the same position in the ranking but considering just a subgroup (e.g. be first or second, or first to third).
3. Have the same position in the ranking (original and re-sampled), no matter thedelete probability used (from 0.01 to 0.5).
= same ranking or position, insensitive to changes in the delete probability.
4. or, have the same position for most of the probabilities used, but not counting extreme situations as a delete probability of 0.5.
= not too sensitive to the probability values used.
In a real world, an scenario to meet the requirements of the first and third conditions is too strict and maybe impossible to fulfill. Therefore, my decision rules to select the best index and the best ranking are based in the second and fourth situations. The area must have the same position in the ranking considering just a subgroup, from the first to the third position in the ranking, no matter the type of item deleted, and for most of the probability values.
An alternative measure is to evaluate the behavior of an index and its success as the number of times that a replicate recovers part of the original ranking (e.g. 1st/2nd/3rd), but in any order. The researcher could consider only the first position in the ranking and evaluates the persistence of this area, or could consider the whole ordered ranking. These measures could be too strict and will be sensitive to the smallest perturbation to the data set, while the first to third position would be enough in terms of conservation planning.
Given any measure of success, the re-sampling approach in conservation have some possible applications as:
1. Which is the best index? that will answer also, what do we want to conserve/use to prioritize?
The best index would be defined as the most supported index, while the area used would be that found for most of the probabilities used.
2. How stable is the ranking (e.g. 1st/2nd/3rd position)?
This is a variation of the previous question, but focused in the ranking, as we prefer a supported ranking, we might evaluate the support for the original ranking.