[ Below is the TLD ScoreCards article that appeared in Issue 3 of the State-of-the-Domains Report by the DNA. The framework and ideas behind it are not set in stone, this is still very much work-in-progress. Comments or suggestions welcome..]
In the previous issue of the State-of-the-Domains Report, NameStat.org introduced an outline model for measuring the “success” of a domain name registry. The framework described a series of key metrics per domain name registry that covered differing elements of performance such as: average registration term, number of countries with a registration, and renewal rate. With domain name registries targeting differing markets and business models, a need for additional differentiation was needed, namely the idea of registry categories or Cohorts. Grouping registry operators into Cohorts allows like-for-like comparisons of key metrics across related or similarly situated registries, thereby providing a means to measure the relative “success” of a TLD (for one particular metric) in relation to its peers. More importantly, the measures set a baseline for the entire Cohort set so that industry growth and success can be measured.
Following on from this work we propose a further enhancement to the framework, combining and distilling certain metrics to produce a visual scorecard of a TLD within its Cohort.
For each TLD a number of core measures have been identified:
Each measure is composed of a number of sub-measures, which are derived from base data available from NameStat.org:
Revenue is calculated through a combination of:
Channel is calculated through a combination of:
Usage is calculated through a combination of:
Using Forward Revenue as an example, each base value (e.g., new registrations per month) is converted to a relative rank between 0 and 100. Combining this rank with ranks for average term and average retail produces a further ranked value, the figure for Forward Revenue. This is in turn combined with Repeat Revenue to generate the final overall value for Revenue.
The temptation here is to compare the scores and pick out winners and losers. While this type of comparison is valid, the real value here is reducing the complexity of the base numbers in order to create a useful interpretation of performance and the reasons for performance.
In addition, rather than selecting winners and losers from within each Cohort, this tool is more valuable in measuring the health of the industry. The combined scores of all the registry operators within a Cohort will set a baseline. As each of the registry operations becomes stable, we expect to see convergence on a set of steady-state measures that reflect a robust industry sector. As time goes on, these metrics can be used to report on the relative success of a registry vis-àvis the others in the Cohort. However, rather than compare the scores of one registry versus another, the scores can be used as a baseline for measuring the overall success of the registries in that Cohort and the new domain name program.
In the meantime, it is very interesting to look at some individual measures, understanding that the measures do not reflect success or lack thereof, but rather raise questions about individual operations.
For example, .coffee is clearly doing well; however, Channel Velocity (the trend in recent numbers of registrars and countries) is trailing. In contrast, Channel Size for .coffee (total countries ever sold into, total registrars ever sold through) is the best in its class. Does this suggest a slowing in marketing momentum and if so, why? The .coffee operator can determine if the slowing was an intentional part of the .coffee strategy or if renewed efforts are called for in order to maintain momentum.
Another example, .bar (the most expensive domain name in this class) has Forward Revenue similar to .pub (significantly cheaper but more registrations). However, Repeat Revenue for .pub (a combination of average term, total domains and renewal price) is lacking in comparison to its peers. The .bar and .pub operators can consider these metrics in determining their go-forward pricing strategy.
The Food & Drink domain name registry Cohort can be further subdivided into smaller groups such as Places (.bar, .pub, .rest and .restaurant), Production (.cooking & .catering) and Consumables (.beer, .coffee, .vodka & .pizza). Scorecards could be produced for each sub-Cohort for more refined analysis.
Note that this analysis separates out .kitchen, .menu and .recipes. The Revenue and Loyalty categories use Renewal Rates as one of the base values. At present only these three registries have entered renewal periods. As more domain name registries enter this phase, Revenue and Loyalty values will continue to become more meaningful as renewal rates settle.
Looking forward: the scorecard is designed to allow at-a-glance comparisons of relative merit for Revenue, Channel, Loyalty and (shortly) Usage. Further enhancements to the model such as Second-Level Domain characteristics are also planned. Currently, sub-categories (e.g., Repeat Revenue) are displayed in the table. Going forward, the base data used to calculate the sub-category measurements will also be available, for full transparency. Base data includes, for example, data available from real-world actions such as pricing, promotions and domain sales.
More details and discussion on the scorecard can be found at NameStat.org. The scorecard and model are still in development and feedback/suggestions are welcome.
The number crunching to arrive at the scorecard figures and layout may seem complex; however, the calculations involved were relatively straightforward.
The first step was overcoming the lack of “bounds” for base numbers such as the number of registrations. Just scoring these numbers would yield a simple first — 120k, second — 30k, third — 5k league. However, we can’t use the number position (first, second, etc.) in our calculation, as we have lost information. What happens when the secondplace TLD has a quarter of the first? Also, if we include the registration figures, because they have no upper limit and almost always increase, any score based on these will increase indefinitely. To solve this we “normalized” the figures to a common scale of 100, with the largest value as 100. Now the first place TLD would have a score of 100 and the second place 25, thus preserving relative differences.
Not all base figures were normalized; the registrar and country trends can be positive or negative. This was tricky until we noticed .vodka had a trend of 0 (i.e., no upward or downward increase over three months). This could be the mid or neutral point of the scale, a score of 50. We had already decided that Channel Velocity would modify the Channel Size in a positive or negative way, so a Channel Velocity of 50 now became the point at which the channel velocity did not affect the channel overall score.
Combining the resultant figures in most cases was straightforward: a simple product of the ingredient figures was taken. This resulted in large results for some of the columns but with further normalization the final figures were derived.
Visually it’s important to provide clues to the size and meaning of a score in an easy and memorable way without clutter or confusion. We settled on five bands with darker blue as notionally “excellent,” darker orange (not red, too negative) as notionally “poor” and grey as average or no change.
The above sounds precise and exact. However, this is new territory and the design of the scorecard is a work in progress and open to suggestion and feedback. Our aim is to produce a clear and transparent framework that is rational and easy to use.
Traditionally the goal of collecting and analyzing numbers such as those in the first table is an empirical base for taking data-based business decisions. When there are many different (and sometimes interdependent) numbers for analysis, the task of developing a set of meaningful data can become difficult.
The goal with the TLD scorecard is to provide a simple graphical representation of Registry metrics, easily understood to inform management actions. With any simplification of figures, information is almost always lost. The scorecard is a distillation of the base numbers, not an end in itself. It is meant as a guide as to where to look, and perhaps to generate additional questions. The base data can be re-examined in light of the questions arising from the scorecard. Custom metrics can be developed to answer specific inquiries.
In today’s crowded and developing TLD market, it is not enough just to analyze your own figures. Whether directly competing with another string or not, the advantage gained from analyzing how similarly-situated domain name registries are achieving their outcomes will always be useful.