Criteria catalog for sustainable software

1 Resource efficiency

To what extent are hardware capacities used, and therefore, to what extent are natural resources consumed indirectly, when a given function is performed?

This main criterion assumes that a given functionality can be fulfilled by a software product using different amounts of hardware capacities, which indirectly results in different amounts of natural resource consumption required for hardware provision, operation, and disposal.

The ideal is a software product that achieves a given functionality with minimum resource consumption, i.e., that maximizes resource efficiency (see glossary). Functionality is specified by standard usage scenarios (see glossary). The hardware capacities to be made available and those actually used as well as the energy consumed serve as approximations for estimating natural resource consumption.

1.1 Hardware efficiency

Which hardware capacities must be available for operating the software product and what is the degree of capacity utilization during operation?

Hardware capacities are measured in % of the corresponding capacity of a reference system1. They can be differentiated according to two dimensions (see table below). On one dimension, they are differentiated in local, network and remote capacities. Here, we further distinguish in recommended (1.1.1) and minimum (1.1.2) capacities as well as capacities required in idle mode (1.1.3) and during the execution of a standard usage scenario (1.1.4). On the other dimension, we differentiate according to the type of hardware capacity: processing power, working memory, permanent storage, bandwidth, and display resolution. The matrix is open to the addition of new columns in case new categories of hardware will become relevant in the future.

  Processing Power Working memory Permanent storage Bandwidth Display resolution
local 1.1.1 a)
1.1.2 a)
1.1.3 a)
1.1.4 a)
1.1.1 b)
1.1.2 b)
1.1.3 b)
1.1.4 b)
1.1.1 c)
1.1.2 c)
1.1.3 c)
1.1.4 c)
- 1.1.1 d)
1.1.2 d)
network - - - 1.1.1 e)
1.1.2 e)
1.1.3 d)
1.1.4 d)
-
remote 1.1.1 f)
1.1.2 f)
1.1.3 e)
1.1.4 e)
1.1.1 g)
1.1.2 g)
1.1.3 f)
1.1.4 f)
1.1.1 h)
1.1.2 h)
1.1.3 g)
1.1.4 g)
- -

Each cell of the matrix in the table above shows the associated criterion (e.g., 1.1.1) with the corresponding indicator (e.g. a)) for operationalization.  The criteria and indicators will be described in the following sections, which are numbered accordingly. Not all of the criteria 1.1.1 to 1.1.4 are applicable in all the matrix cells. For this reason, some of the cells remain empty. 

Criteria 1.1.5 and 1.1.6 are used for the assessment of hardware efficiency as well. They can be assessed in general terms; they do not require differentiation according to this matrix and do not show up in Table 1-1 for this reason. 

When these criteria are to be aggregated later, the principal problem arises that a trade-off between different hardware capacities (local vs. remote, processing power vs. working memory, processing power for data compression vs. bandwidth, etc.) must be made. If it were possible to evaluate the hardware capacities in the form of an ecological footprint, they could be weighted and aggregated in that regard. Assessing this footprint is not part of the work reported here; we refer the reader to existing life cycle inventories for ICT hardware and electric energy as a basis for aggregation.

Basic definitions
Basic definitions for the measurement of the criteria 1.1.3 and 1.1.4:
Identifier Name Definition Comment
FLi full load Upper limit of the capacity i in the reference system. For processing power, the FL is 100%, for working memory the sum of the installed RAM, for network bandwidth the maximum transmission speed, etc.
BLi base load Average load of the capacity i in the reference system when the software product under study is not installed.
ILi idle load Average load of the capacity i in the reference system when the software product under study is installed, but idle. Idle load includes base load.
NILi net idle load NILi = ILi - BLi  
t time Time needed to execute the standard usage scenario on the reference system. Begins with the start of the standard usage scenario and ends when all required actions are executed, including follow-up processes (such as releasing memory or deleting temporary files).
GLi gross load Load of the capacity i in the reference system while executing the standard usage scenario, measured as time-weighted average over t.  
NLi net load NLi = GLi – BLi  
AFi allocation factor AFi = NLi / (FLi - BLi) Allocation factor used to assign a share of the base load GA to the effective load EL (defined below).
AFIi allocation factor idle AFIi = NILi / (FLi - BLi Allocation factor used to assign a share of the base load GA to the effective load idle ELI (defined below).
ELi effective load ELi = NLi + AFi * BLi  
EILi effective load idle EILi = NILi + AFIi * BLi Used to calculate the indicators for hardware demand of criterion 1.1.3
HDi hardware demand HDi = ELi * t Used to calculate the indicators for hardware demand of criterion 1.1.4
1.1.1 Recommended system requirements and resulting hardware requirements (including peripheral devices)

Which system requirements does the manufacturer recommend for operating the software product?

Indicators:

a) Recommended local processing power as specified by the manufacturer in % of the processing power of the reference system

b) Recommended local working memory as specified by the manufacturer in % of the working memory of the reference system

c) Recommended local permanent storage as specified by the manufacturer in % of the permanent storage of the reference system

d) Recommended display resolution as specified by the manufacturer in % of the display resolution of the reference system

e) Recommended network bandwidth as specified by the manufacturer in % of the network bandwidth of the reference system

f) Recommended server processing power as specified by the manufacturer in % of the processing power of the reference system

g) Recommended server working memory as specified by the manufacturer in % of the server working memory of the reference system

h) Recommended server permanent storage as specified by the manufacturer in % of the server permanent storage of the reference system

1.1.2 Minimum system requirements and resulting hardware requirements (including peripheral devices)

What are the minimum system requirements for operating the software product?

Indicators:

a) Minimum local processing power as specified by the manufacturer in % of the processing power of the reference system

b) Minimum local working memory as specified by the manufacturer in % of the working memory of the reference system

c) Minimum local permanent storage as specified by the manufacturer in % of the permanent storage of the reference system

d) Minimum display resolution as specified by the manufacturer in % of the display resolution of the reference system

e) Minimum network bandwidth as specified by the manufacturer in % of the network bandwidth of the reference system

f) Minimum server processing power as specified by the manufacturer in % of the processing power of the reference system

g) Minimum server working memory as specified by the manufacturer in % of the server working memory of the reference system

h) Minimum server permanent storage as specified by the manufacturer in % of the server permanent storage of the reference system

 

1.1.3 Hardware utilization in idle mode assuming a standard configuration

What is the level of utilization of the available hardware capacities by the software product in idle mode?

Indicators:

a) Measurement of average processor utilization in idle mode under the standard configuration

b) Measurement of average working memory utilization in idle mode under the standard configuration

c) Measurement of average permanent storage utilization in idle mode under the standard configuration

d) Measurement of average bandwidth utilization for network access in idle mode under the standard configuration

e) Measurement of average server processor utilization in idle mode under the standard configuration

f) Measurement of average server working memory utilization in idle mode under the standard configuration

g) Measurement of average server permanent storage utilization in idle mode under the standard configuration

Average processor load (indicators a. and e.) and average working memory load (indicators b. and f.) are calculated as effective idle load EIL (see Table "Basic definitions").

 

1.1.4 Hardware utilization during normal use assuming a standard configuration and a standard usage scenario

What is the average utilization of the available hardware capacities during operation of the software product?*

It should be noted here that utilization of hardware capacities is understood as a variable integrated over time. If, for example, program A requires twice as much processing power, working memory, or bandwidth as program B to accomplish a given standard usage scenario, but makes the processor, memory, or bandwidth available again after half the period of time required by B, then according to this criterion, A is not less efficient than B. (This is not the case for criteria 1.1.1 to 1.1.3.) Thus, the use of acceleration technologies is not penalized by this criterion.

Indicators:

a) Measurement of average processor utilization when running the standard usage scenario under the standard configuration

b) Measurement of average working memory utilization when running the standard usage scenario under the standard configuration

c) Measurement of average permanent storage utilization when running the standard usage scenario under the standard configuration

d) Measurement of average bandwidth utilization for network access when running the standard usage scenario under the standard configuration

e) Measurement of average server processor utilization when running the standard usage scenario under the standard configuration

f) Measurement of average server working memory utilization when running the standard usage scenario under the standard configuration

g) Measurement of average server permanent memory utilization when running the standard usage scenario under the standard configuration

Hardware demand for processor load (indicators a. and e.) and average working memory load (indicators b. and f.) are calculated as defined in Table "Basic definitions".

* Average capacity utilization determines which free hardware capacities can be used by other software products during operation of the software product

 

1.1.5 Economical use of hardware through adaptability and support for users when adapting the software product

Does the software product use only those hardware capacities required for running the functions demanded by the individual user? Does the software product provide sufficient support when users adapt it to their needs?*

Indicators:

a) Does the software automatically minimize the required capacities and/or are there relevant options available during installation? (Scale: yes/no)

b) If users choose an option, can they change the decision for or against installation options at any later point in time? (Scale: yes/ no)

c) Black box test whether hardware-intensive modules can be disabled (Scale: can permanently be disabled/can temporarily be disabled/cannot be disabled)

d) Is it possible (without drawbacks) to disable peripheral devices that are not needed temporarily or permanently or to avoid providing them at all? (Scale: can be disabled temporarily and permanently/can be disabled only temporarily/cannot be disabled)

e) Will files used only for installing the product be deleted after installation?

* No utilization of capacities by functionality temporarily or permanently not demanded by the user.

 

1.1.6 Online delivery

Can the software product (including all programs, data, and documentation including manuals) be purchased, installed, and operated without transporting physical storage media (including paper) or other materials goods (including packaging)?

Indicators:

a) Can the software be delivered and updated online?

b) Is it supported that the user organization can store the software product and its updates on a local server, avoiding transferring them for every single user?

 

1.2 Energy efficiency

How much electricity does the hardware consume when the software product is used to execute a standard usage scenario? 

The consumption of electric energy is a consequence of the utilization of hardware capacities. How to measure hardware utilization has already been described in section 1.1.4 above. In parallel to those measurements, the electrical power demanded by the hardware should be measured (or estimated) as well, at least for the entirety of hardware used locally, for data transmission in the network or remotely, respectively.

Indicators:

a) Measurement of the energy consumed on the local device for running the standard usage scenario under the standard configuration

b) Estimation of the energy consumed in the network for the data traffic caused by running the standard usage scenario under the standard configuration (a current estimate of network energy intensity in kWh/GB from literature may be used, if necessary differentiated among types of access network)

c) Measurement of the energy consumed by servers for the remote processing and storage for running the standard usage scenario under the standard configuration (if measurement not possible, an estimate based on factors for average energy intensity of data center services from literature may be used)

 

The electric energy consumed is the integral of electric power over the time needed for the execution of the standard usage scenario. Departing from the specifications provided to measure hardware load (section 1.1.4), only net indicators will be used for the energy measurements (indicators a. and c.), i.e., only the quantity that exceeds the level of the electric base load. This is done to increase practicability (calculating an allocation factor for electricity may be difficult because a true upper limit for electric power is sometimes not known). It also adds to the clarity of the results of the energy measurements if base load energy is not included when comparing software products.

1.3 Resource management

To what extent does the software product contribute to efficient management of the resources it uses during operation?

Since the extent to which a given software product is used may vary, adaptive demand for hardware capacities that is supported by the software product contributes to resource conservation. Hardware capacities not in use can potentially be used by other processes or reduce their energy consumption. Both options contribute indirectly to natural resource conservation.

In contrast to criteria 1.1 and 1.2, this criterion refers to adapting the demand for hardware capacities at the program's runtime, in particular the transition to less energy-consuming modes, dependent on the current user requirements or the available hardware capacities or energy. In other words, while resource efficiency in the various modes was addressed by criteria 1.1 and 1.2, the focus here is on the ability to switch between modes depending on context.

 

1.3.1 Adaptation of hardware capacities used to current demand

Does the software product have the feature to release hardware capacities (and reduce energy consumption as a consequence) when it doesn’t temporarily use these capacities?

Indicators:

a) Does the software product have different modes which have a measurable effect on energy consumption?

b) Does the software product dynamically change to a more energy saving mode when possible (e.g. sleep mode)?

c) In case the user has to make energy-relevant settings, are these settings concentrated in one place and easily understandable for the user?*

* Examples: Background/sleep settings, animations, computing-intensive processes such as indexing etc., cache sizes, ability to select the time at which processes are executed to take advantage of ecologically more beneficial energy (demand shaping).

 

1.3.2 Adaptation of hardware capacities used to current supply

Is the software product able to dynamically adapt its demand for hardware capacities and energy when the supply is changing? (e.g., when the available bandwidth is decreasing or battery is low)

Indicators:

a) Does the software product switch to a more economical mode when less hardware capacity or energy is available, avoiding errors or loss of data? (no restrictions, slower execution, error during execution)

b) Is the full software functionality available in if the energy management of lower system layers or connected client systems is activated?*

* In particular server-based software should avoid that activating the energy management on client side hampers the functionality. For example, no session information should be lost if the client computer enters sleep mode.

 

1.3.3 Default settings supporting resource conservation

Are the default settings of the software product selected in such a way that they also take the goal of resource conservation into account?*

Indicators:

a) Reviewer's assessment whether the default settings of the software product are selected in such a way that they also take the goal of resource conservation into account

* Example: Default setting for printing: Double-sided printing if the printer has this capability?  

 

1.3.4 Feedback on use of hardware capacities and energy

Can the local and remote hardware capacities used by the software product and their resulting energy consumption be monitored, and are the displayed values correct?

Indicators:

a) Are the hardware capacities in use, data flow, and energy consumption displayed? (Scale: yes/to some extent/no)

b) Assessment by the reviewer whether the display is correct

 

Glossary
Term Description
Energy efficiency Generally, the amount of “useful work” divided by the amount of energy it requires. In the context of this document, “useful work” is operationalized as the successful execution of standard usage scenarios.
Hardware The material goods required to run programs or to store or transport data.
Hardware capacity Quantifiable characteristic of a hardware system which represents its performance limit on a given dimension of performance (e.g., working memory capacity, computing power, bandwidth).
Hardware system Delimitable unit of hardware that performs defined functions.
Indicator An empirically determinable quantity that provides insight into a matter that cannot be measured directly. The indicators proposed in this document have different levels of measurement. In some cases, researchers will have to settle for an ordinal scale (e.g., “insufficient”, “sufficient”, “good”, “very good”, or even merely “fulfilled”, “not fulfilled”) to avoid giving the false impression of non-existent precision arising from a cardinal scale.
Reference system A hardware system that is defined as generally customary in terms of its most important capacities (e.g., working memory, processor performance) during a defined period of time (e.g., one year). The purpose of the reference system is to be able to express indicators such as “minimum local memory” in relation to a reference value (currently “customary” memory).
Resource In the context of this document, a natural resource, in particular a raw material, a form of energy, or also the capacity of an environmental medium to absorb emissions. To differentiate natural resources from technical ones, especially hardware resources, the more precise term “hardware capacities” is used here for the latter. Since using hardware capacities always results in using natural resources, this distinction (which ultimately amounts to a definitionally difficult differentiation between the ecosphere and the technosphere) is not of decisive importance here.
Resource efficiency Generally, the amount of “useful work” divided by the amount of resources it requires. In the context of this document, “useful work” is operationalized as the successful execution of standard usage scenarios.
Software Programs and data in digital form.
Software product A delimitable unit of programs and data for which a license is available.
Standard configuration A set of conditions, defined as a reference, under which a given software product is run; it includes the parameter settings selected during installation or operation, the system software provided, potentially additional software products required for operation, as well as the reference system at the hardware level.
Standard usage scenario A usage scenario that is used for testing a software product and is supposed to be as representative as possible for the customary use case.
Usage pattern Abstracted form of a sequence of interactions with a given software product.
Usage scenario Description of a usage pattern which is generally machine executable.

[1] Application of the set of criteria requires that a reference system corresponding to current technical developments is determined periodically. The reference system serves to standardize indicators.

 

[2] Use of electricity is a consequence of the use of hardware capacities already discussed in section 1.1. This implies that this criterion is redundant. However, the redundancy is desired since energy can be measured separately and not all subcriteria of hardware efficiency (1.1) are operationalizable.

 

back-to-top nach oben