HEART 2022 Special Session on Efficiency-Driven Computing

Overview

The special session on Efficiency-Driven Computing at HEART2022 aims to showcase state-of-the-art software, hardware and architectural innovations that advance the research frontier in improving power- and energy-efficiency in computing systems. Since power-efficiency, by definition, is affected by both the performance and the power consumption of a system, it is possible that the most power-efficient system is not necessarily the fastest system at the same time. The goal of this session is therefore to highlight works that improve power-efficiency of a system even when the resulting system may not be faster, and in some cases, even slower.

This special session is designed to be inclusive in terms of targeted systems and applications. We are interested in contributions that target computing systems of all kinds: from the largest supercomputers to tiniest battery-operated systems, from CPU-only systems to accelerated systems using FPGA, GPU or CGRA. We are interested in software, hardware and/or architectural techniques that can demonstrate measurable improvement in power-efficiency of the system. Depending on the target application and architecture, either power consumption or energy consumption of the system may be used as a metric for measuring the efficiency of a system. The only technical requirement is that the submitted work must demonstrate measurable improvement in efficiency of the targeted system over a baseline system, and we are interested in works that can attribute the technical innovations that lead to such improvement.

Specifically, let A be the accelerated system that is compared with a baseline system B. Then authors must demonstrate that:

1. Eff_A > Eff_B

2. Perf_A > 0.8 Perf_B

where Perf and Eff are performance metric and power- or energy-efficiency metric that the authors must clearly define (see the description on Performance Evaluation Document below). Note Perf is defined as a metric that is better when the number is higher (e.g. throughput, or 1/elapsed time).

Example topics that fall under this very inclusive call for contribution includes, but not limited to:

• Use of accelerator (FPGA, GPU, other) vs pure CPU system

• Use of SIMD extension vs original CPU without SIMD extension

• Use of different connection topology for processor array

• Use of run-time reconfigurable heterogeneous architectures (e.g. big.LITTLE)

• Use of partial power shutdown function of CPU. (e.g. Fugaku’s eco-mode)

Note that the above two evaluation metrics are prerequisite for the submissions to be considered for this special session. Fulfilling both of them does not automatically imply acceptance of the paper. See Evaluation Criteria below.

Performance Evaluation Document

To facilitate meaningful discussion of power- and energy-efficiency improvements in a wide range of target scenarios, all submissions to this special session must include a Performance Evaluation Document together with the main technical contribution paper. In this document, the authors must provide information on all questions listed below concisely (within 1 page, or 800 words):

1. Describe your target system

2. Describe your baseline system

3. Describe your target benchmark or application.

4. Define your performance metric

5. Define your power or energy measurement metric

6. Describe the experimental procedure to measure performance as defined in Ans 4

7. Describe the experimental procedure to measure your power or energy consumption as defined in Ans 5

8. Performance result (your system)

9. Power or Energy Result (your system)

10. Efficiency of your system (Eff_A)

11. Performance result (baseline)

12. Power or Energy Result (baseline)

13. Efficiency of baseline system (Eff_B)

14. Efficiency Improvement (Eff_A / Eff_B)

Question 1: Describe your target system

Sample Answers:

• “The XYZ super computer [reference]”

• “An FPGA with embedded processor on Zynq [reference]”

• “A custom cluster of Jetson Xaviar [reference]”

Note:

Any computer system (even CPU-only systems) may be targeted. Improvement in power efficiency may come from software, hardware or even architectural innovations.

Question 2: Describe your baseline system

Sample Answers:

• “The same system as Ans 1 but with SIMD extension turned off”

• “Using only embedded processor on Zynq”

• “A similar design published in [reference]”

• “A similar embedded GPU system”

Note:

Describe the baseline system that you intend to compare against. You have to make a convincing argument why such a baseline system should be chosen for comparison. For example, if you want to demonstrate how an additional accelerator can improve your system’s power efficiency, you probably want to compare against the same host system without the accelerator. If you want to demonstrate a new compilation technique that improves power-efficiency of your system, then you may want to compare against the case when this new compiler optimization is turned off.

Question 3: Describe your target benchmark or application.

Sample Answers:

• “Image classification”

• “Linpack”

• “SPEC”

• “Video filtering”

• “Custom designed microbenchmark”

Note:

Describe your target application or the benchmark. For this special session, we accept any application or benchmark as the target as long as it is a reasonable task that can benefit from power-efficient computing. Describe the application scenarios or context to make an argument on why such a target is important from a power-efficiency or energy-efficiency perspective.

Question 4: Define your performance metric

Sample Answers:

• “Elapsed time”

• “Throughput”

• “Images per second”

• “Base pair comparison per second”

• “Time needed to classify an image”

• “Latency needed to detect anomaly”

Note:

Every application is different so the notion of performance can be different. In a throughput-oriented system, the number of tasks completed per second is probably more relevant than the elapsed time of a single task. But on a real-time system, latency or system response time can be more important. You should clearly describe your performance metric here, and make an argument on why it is an appropriate measurement

Question 5. Define your power or energy measurement metric

• • Sample Answers:

  • “Chip energy consumption per image classification”

  • “Chip power to perform image classification”

  • “System energy consumption per image classification”

  • “System power to perform image classification”

Note:

Either power or energy consumption can be used colloquially for evaluating “efficiency” for the purpose of this special session. In high performance computers or datacenter operations, system power consumption may be more relevant, while total system energy consumption is more important as a metric of performance for battery-operated IoT systems. For this special session, either of them can be used to evaluate the overall “efficiency” of the proposed system.

Question 6: Describe the experimental procedure to measure performance as defined in Ans 4

Sample Answers:

• “We measured time T to process 1000 images and reported elapsed time per image as T/1000. We performed the experiment 10 times and the average result was reported.”

• “We measured time T to process 1000 images and reported throughput as 1000/T”

Note:

Explain the experimental procedure employed to produce the performance measurement defined in 4 above. The procedure should be designed to reflect objective comparison between your system and the target baseline system. If you cannot reproduce the results from the baseline, you should ensure you measure the same performance as reported for your baseline system.

Question 7: Describe the experimental procedure to measure your power or energy consumption as defined in Ans 5

Sample Answers:

• “We ran classification on 1000 images and measured the average power consumption during that time. Power was measured at the external system power supply.”

• “We performed the target breadth-first-search algorithm on a graph with 1000 nodes and recorded the instantaneous current and voltage supplied to the target chip during the process. Energy consumption was reported by integrated the values”

Note:

Explain the experimental procedure employed to produce the power or energy consumption as defined in 5 above. The procedure should be designed to reflect objective comparison between your system and the target baseline. For power measurement, be specific on what is the average window. For energy consumption please ensure the measurement covers the entire duration of the target experiment.

Question 8: Performance result (your system) (Perf_A)

Sample Answers:

• “100 images per second”

• “1 / (23.7 milliseconds)”

Note:

Perf is defined as 1/(elapsed time) for this special session.

Question 9: Power or Energy Result (your system)

Sample Answers:

• “1.23 W”

• “0.89 mJ”

Question 10: Efficiency (Your system) (Eff_A)

Sample Answers:

• “100 / 1.23”

• “1/(23.7 x 0.89)”

Note:

Ans 8 / Ans 9

Question 11: Performance result (baseline) (Perf_B)

Sample Answers:

• “50 images per second”

• “1 /(24 ms)”

Question 12: Power or Energy Result (baseline)

Sample Answers:

• “1.1 W”

• “1.08 mJ”

Question 13: Efficiency (baseline) (Eff_B)

Sample Answers:

• “50 / 1.1”

• “1 / (24 x 1.08)”

Note:

Ans 11 / Ans 12

Question 14: Efficiency Improvement(Eff_A/Eff_B)

Sample Answer:

• “1.78”

• “1.29”

Note:

Ans 10 / Ans 13

Submission & Evaluation Criteria

Submission to this special session should include 2 parts:

1. An 8-page technical paper

2. The Performance Evaluation Document

The 8-page technical paper should present original research with technical contributions to the design of power- or energy-efficient computing systems. This paper may be an extension to previously published works. In this case, the submission should include at least 30% new technical content and previously unpublished results.

The technical paper will undergo a single-blind peer-review process (the main research track review is double-blinded). The identity of the authors may be revealed to the reviewers but the reviewers’ identities are hidden. This way, you can clearly explain the context of this performance evaluation and where the technical contributions are. The technical paper will be evaluated on its novelty and technical contributions with respect to improvements in power- or energy-efficiency of a computing system. The Performance Evaluation Document will be evaluated based on its objectiveness and scientifically soundness of the metrics used and the comparison methodology. A best paper award is planned for the best submission to this special session.

Submit to this special session through the main HEART 2022 conference submission system. You should combine the 8-page technical paper and the Performance Evaluation Document into a single PDF for submission.