{"id":410,"date":"2020-04-07T21:22:33","date_gmt":"2020-04-07T20:22:33","guid":{"rendered":"http:\/\/www.smartrebirth.org\/?p=410"},"modified":"2020-04-08T17:37:38","modified_gmt":"2020-04-08T16:37:38","slug":"bench","status":"publish","type":"post","link":"http:\/\/www.smartrebirth.org\/index.php\/2020\/04\/07\/bench\/","title":{"rendered":"Benchmarks analysis"},"content":{"rendered":"[fullwidth_text alt_background=&#8221;none&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;]\n<p>Following the first benchmarks last week, here is a deeper analysis of performance and power consumption.<\/p>\n<p style=\"text-align: left;\"><span style=\"color: #ff0000;\">The main purpose is not to measure the exact performance of the ARM64 \/ Snapdragon platform, it is to compare the ARM64 chipset embedded in a phone vs x86 \/ x86_64b setups in different tests.<\/span><\/p>\n<p>Here is the link to the full set of results on: <a href=\"https:\/\/openbenchmarking.org\/result\/2004070-VE-2004069VE13&amp;obr_sor=y\" target=\"_blank\" rel=\"noopener noreferrer\">openbenchmarking.org<\/a>.<\/p>\n<p>&nbsp;<\/p>\n[\/fullwidth_text] [spb_text_block title=&#8221;Test platforms&#8221; pb_margin_bottom=&#8221;no&#8221; pb_border_bottom=&#8221;no&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;]\n<ul>\n<li><strong>Qualcomm SDM660<\/strong> : Xiaomi Redmi Note 7 (4GB RAM + 64GB Flash), 8 (4+4) cores up to 2.2 GHz, Debian 10 (buster) for Arm64 running from internal storage (not SD card)<\/li>\n<li><strong>Intel i5-4690K<\/strong> (Everyday SW build platform) : 4 cores &#8211; 3.5\/3.9GHz (No overclocking), 16 GB of DDR3, NVMe PCIe SSD 500GB (Crucial P1), Ubuntu 18.04 LTS, No external GPU for power consumption measurement<\/li>\n<li><strong>Intel i7-6700K<\/strong> (Gaming platform) : 4 cores \/ 8 threads &#8211; 4.0\/4.2GHz (No overclocking), 16 GB of DDR4, NVMe PCIe SSD 500GB (Crucial P1), Ubuntu 18.04 LTS, No external GPU for power consumption measurement<\/li>\n<li><strong>VPS<\/strong> : The Virtual Private Server hosting this website, provided by <a href=\"https:\/\/www.ovh.com\/world\/vps\/\">OVH<\/a>. Single x86 CPU core with 2GB of RAM. Debian 8. Actual hardware platform is unknown. (No power measurement available)<\/li>\n<li><strong>Raspberry PI 4<\/strong> : 4GB RAM version, with cooling fan running Ubuntu Server 18.04 LTS from SD Card (Sandisk Extreme PRO 64Gb UHS-I SDXC)<\/li>\n<\/ul>\n<p><em>Note : NVMe SSDs on PC platforms might increase the performance gap where benches can be influenced by I\/O performance. Phone runs on internal storage, and Raspberry Pi uses (fast) SD Card.<\/em><\/p>\n<p>The power measurement was performed at the outlet for PC platforms and using a DC power supply for the phone and raspberry Pi (see end of the post)<\/p>\n<p>&nbsp;<\/p>\n[\/spb_text_block] [spb_text_block title=&#8221;Test results&#8221; pb_margin_bottom=&#8221;no&#8221; pb_border_bottom=&#8221;no&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;]\n<p><img loading=\"lazy\" class=\"aligncenter size-full wp-image-463\" src=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/SDM660-results.png\" alt=\"\" width=\"838\" height=\"274\" srcset=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/SDM660-results.png 838w, http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/SDM660-results-300x98.png 300w, http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/SDM660-results-768x251.png 768w\" sizes=\"(max-width: 838px) 100vw, 838px\" \/><\/p>\n<p style=\"text-align: center;\"><a href=\"https:\/\/drive.google.com\/file\/d\/1ZGGfzWghbabpw2im3xCMS6crX6O2xz21\/view?usp=sharing\" target=\"_blank\" rel=\"noopener noreferrer\">Link to the full spreadsheet<\/a><\/p>\n<p style=\"text-align: center;\"><em>Note : the PrimeSieve test is by far the most power demanding test on all platforms. Without air cooling on the phone, there is CPU throttling, stabilizing at 8.5Watts after about 5 minutes. The 10.4 Watts is the value with lateral air flow from a fan (not directed to the front of the phone), as it is in the <a href=\"http:\/\/www.smartrebirth.org\/index.php\/2020\/03\/30\/rack-printed-and-assembled\/\">demo rack<\/a>.<\/em><\/p>\n<p>There are two scores per test:<\/p>\n<ul>\n<li><strong>Performance score<\/strong>, usually reflecting the time to execute the tests or the number of computations per seconds. Highest platforms will perform the test in a smaller amount of time or execute more computations per second.<\/li>\n<li><strong>Energy Score<\/strong>. This score is the amount of energy needed to complete the test or a number of computations. This score will be used to sort efficiency of platforms.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n[\/spb_text_block] [spb_text_block title=&#8221;Platform Comparison&#8221; pb_margin_bottom=&#8221;no&#8221; pb_border_bottom=&#8221;no&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;]\n<p>The first graphs show the performance of platforms:<\/p>\n<p><img loading=\"lazy\" class=\"aligncenter size-full wp-image-413\" src=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Performance-index-Avg-normalized.png\" alt=\"\" width=\"764\" height=\"208\" srcset=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Performance-index-Avg-normalized.png 764w, http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Performance-index-Avg-normalized-300x82.png 300w\" sizes=\"(max-width: 764px) 100vw, 764px\" \/><\/p>\n<p style=\"text-align: center;\"><em>Values are average of all tests, normalized to 1 for SDM660, the Arm64 SoC of the Xiaomi phone.<\/em><\/p>\n<p>We can read this graph as : i7-6700K performs tests 3.68 times faster than the phone.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" class=\"aligncenter size-full wp-image-412\" src=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Energy-Avg-normalized.png\" alt=\"\" width=\"763\" height=\"170\" srcset=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Energy-Avg-normalized.png 763w, http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Energy-Avg-normalized-300x67.png 300w\" sizes=\"(max-width: 763px) 100vw, 763px\" \/><\/p>\n<p style=\"text-align: center;\"><em>Values are average of all tests, normalized to 1 for SDM660, the Arm64 SoC of the Xiaomi phone.<\/em><\/p>\n<p>We can read this graph as : i7-6700K platform needs 5.34 times more energy than the phone to complete the test.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" class=\"aligncenter size-full wp-image-411\" src=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Efficiency-Avg-normalized.png\" alt=\"\" width=\"763\" height=\"185\" srcset=\"http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Efficiency-Avg-normalized.png 763w, http:\/\/www.smartrebirth.org\/wp-content\/uploads\/2020\/04\/Efficiency-Avg-normalized-300x73.png 300w\" sizes=\"(max-width: 763px) 100vw, 763px\" \/><\/p>\n<p style=\"text-align: center;\"><em>Values are average of all tests, normalized to 1 for SDM660, the Arm64 SoC of the Xiaomi phone.<\/em><\/p>\n<p>The value presented here is an efficiency indicator : ratio between time of complete the test divided by the needed energy.<\/p>\n<p>We can read this graph as : i7-6700K has 69% of efficiency compared the the phone to perform a test with a given amount of energy.<\/p>\n<p><strong>Idle Power:<\/strong><\/p>\n<ul>\n<li>SDM660 : 1.6 Watts -&gt; 14 kWh per Year<\/li>\n<li>i5-4690K : 24.9 Watts -&gt; 218 kWh per year<\/li>\n<li>i7-6700K : 33 Watts -&gt; 289 kWh per year<\/li>\n<li>RPi4 : 3.48 Watts -&gt; 30.5 kWh per year<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n[\/spb_text_block] [fullwidth_text title=&#8221;Analysis&#8221; alt_background=&#8221;none&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;]\n<p>The benchmarks confirm that CPUs in modern mid-range phones can compete with x86 \/ x86_64 cores for some applications.<\/p>\n<p>For high-end applications when very fast response is needed, high end x86_64 CPUs are more than 5 times quicker, but requiring more energy.<\/p>\n<p>For all the other use cases, phone based server can compete, since they are more efficient to perform the same computing.<\/p>\n<p>About VPS : We can see that a phone-based server can easily compete with VPS for website hosting, in terms of CPU and probably in terms of efficiency, with a ridiculous 1.6 Watts of power in idle mode for the phone. Of course the VPS tested here has only one CPU core and the phone SoC has 8 cores, so it will be interesting to test how many &#8216;virtual&#8217; environments can be run on a phone and how it can compete with VPS.<\/p>\n<p>About Raspberry Pi 4 : First, we can confirm that RPi4 can compete with low-end server for basic tasks in terms of performance. The problem with this new version of the famous board is it&#8217;s power management, as mentionned by everyone using this board. The tests were performed with a cooling fan and heat sinks on chips, but for many tests we experienced CPU throttling. The comparison with the Phone is very interesting : RPi4 is about half the computing power of the Phone, but it&#8217;s energy is about twice the phone ! So RPi4 is a decent computing platform, but actually not as efficient as phone SoCs and large x86_64 systems.<\/p>\n<p class=\"impact-text\" style=\"text-align: center;\">In conclusion, we can say that servers based on mid-range phones are really faster than low-end internet servers, and can compete in terms of efficiency with high-end ones when fast performance and response is not needed.<\/p>\n<p>In the next steps of the project, we will build larger systems with phones and test performance for scientific applications, website hosting and other use cases.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n[\/fullwidth_text] [spb_column width=&#8221;1\/3&#8243; el_position=&#8221;first&#8221;] [spb_single_image image=&#8221;471&#8243; image_size=&#8221;large&#8221; frame=&#8221;noframe&#8221; intro_animation=&#8221;none&#8221; full_width=&#8221;no&#8221; lightbox=&#8221;yes&#8221; link_target=&#8221;_self&#8221; caption=&#8221;Idle Power&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;] [\/spb_column] [spb_column width=&#8221;1\/3&#8243;] [spb_single_image image=&#8221;472&#8243; image_size=&#8221;large&#8221; frame=&#8221;noframe&#8221; intro_animation=&#8221;none&#8221; full_width=&#8221;no&#8221; lightbox=&#8221;yes&#8221; link_target=&#8221;_self&#8221; caption=&#8221;Rust Prime bench running&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;] [\/spb_column] [spb_column width=&#8221;1\/3&#8243; el_position=&#8221;last&#8221;] [spb_single_image image=&#8221;474&#8243; image_size=&#8221;large&#8221; frame=&#8221;noframe&#8221; intro_animation=&#8221;none&#8221; full_width=&#8221;no&#8221; lightbox=&#8221;yes&#8221; link_target=&#8221;_self&#8221; caption=&#8221;i5-4690K Power measurement&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;] [\/spb_column]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>[fullwidth_text alt_background=&#8221;none&#8221; width=&#8221;1\/1&#8243; el_position=&#8221;first last&#8221;] Following the first benchmarks last week, here is a deeper analysis of performance and power consumption. The main purpose is not to measure the exact performance of the ARM64 \/ Snapdragon platform, it is to compare the ARM64 chipset embedded in a phone vs x86 \/ x86_64b setups in different tests. Here is the link [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[7],"tags":[],"_links":{"self":[{"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/posts\/410"}],"collection":[{"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/comments?post=410"}],"version-history":[{"count":29,"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/posts\/410\/revisions"}],"predecessor-version":[{"id":493,"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/posts\/410\/revisions\/493"}],"wp:attachment":[{"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/media?parent=410"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/categories?post=410"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.smartrebirth.org\/index.php\/wp-json\/wp\/v2\/tags?post=410"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}