137 lines
12 KiB
HTML
137 lines
12 KiB
HTML
|
<!DOCTYPE html>
|
||
|
|
<html lang="en">
|
||
|
|
<head>
|
||
|
|
<meta charset="UTF-8" />
|
||
|
|
<title>Cookbook » Profile Taskflow Programs | Taskflow QuickStart</title>
|
||
|
|
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600" />
|
||
|
|
<link rel="stylesheet" href="m-dark+documentation.compiled.css" />
|
||
|
|
<link rel="icon" href="favicon.ico" type="image/vnd.microsoft.icon" />
|
||
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
|
|
<meta name="theme-color" content="#22272e" />
|
||
|
|
</head>
|
||
|
|
<body>
|
||
|
|
<header><nav id="navigation">
|
||
|
|
<div class="m-container">
|
||
|
|
<div class="m-row">
|
||
|
|
<span id="m-navbar-brand" class="m-col-t-8 m-col-m-none m-left-m">
|
||
|
|
<a href="https://taskflow.github.io"><img src="taskflow_logo.png" alt="" />Taskflow</a> <span class="m-breadcrumb">|</span> <a href="index.html" class="m-thin">QuickStart</a>
|
||
|
|
</span>
|
||
|
|
<div class="m-col-t-4 m-hide-m m-text-right m-nopadr">
|
||
|
|
<a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
|
||
|
|
<path id="m-doc-search-icon-path" d="m6 0c-3.31 0-6 2.69-6 6 0 3.31 2.69 6 6 6 1.49 0 2.85-0.541 3.89-1.44-0.0164 0.338 0.147 0.759 0.5 1.15l3.22 3.79c0.552 0.614 1.45 0.665 2 0.115 0.55-0.55 0.499-1.45-0.115-2l-3.79-3.22c-0.392-0.353-0.812-0.515-1.15-0.5 0.895-1.05 1.44-2.41 1.44-3.89 0-3.31-2.69-6-6-6zm0 1.56a4.44 4.44 0 0 1 4.44 4.44 4.44 4.44 0 0 1-4.44 4.44 4.44 4.44 0 0 1-4.44-4.44 4.44 4.44 0 0 1 4.44-4.44z"/>
|
||
|
|
</svg></a>
|
||
|
|
<a id="m-navbar-show" href="#navigation" title="Show navigation"></a>
|
||
|
|
<a id="m-navbar-hide" href="#" title="Hide navigation"></a>
|
||
|
|
</div>
|
||
|
|
<div id="m-navbar-collapse" class="m-col-t-12 m-show-m m-col-m-none m-right-m">
|
||
|
|
<div class="m-row">
|
||
|
|
<ol class="m-col-t-6 m-col-m-none">
|
||
|
|
<li><a href="pages.html">Handbook</a></li>
|
||
|
|
<li><a href="namespaces.html">Namespaces</a></li>
|
||
|
|
</ol>
|
||
|
|
<ol class="m-col-t-6 m-col-m-none" start="3">
|
||
|
|
<li><a href="annotated.html">Classes</a></li>
|
||
|
|
<li><a href="files.html">Files</a></li>
|
||
|
|
<li class="m-show-m"><a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
|
||
|
|
<use href="#m-doc-search-icon-path" />
|
||
|
|
</svg></a></li>
|
||
|
|
</ol>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</nav></header>
|
||
|
|
<main><article>
|
||
|
|
<div class="m-container m-container-inflatable">
|
||
|
|
<div class="m-row">
|
||
|
|
<div class="m-col-l-10 m-push-l-1">
|
||
|
|
<h1>
|
||
|
|
<span class="m-breadcrumb"><a href="Cookbook.html">Cookbook</a> »</span>
|
||
|
|
Profile Taskflow Programs
|
||
|
|
</h1>
|
||
|
|
<nav class="m-block m-default">
|
||
|
|
<h3>Contents</h3>
|
||
|
|
<ul>
|
||
|
|
<li><a href="#ProfilerEnableTFProf">Enable Taskflow Profiler</a></li>
|
||
|
|
<li><a href="#ProfilerEnableTFProfServer">Enable Taskflow Profiler on a HTTP Server</a></li>
|
||
|
|
<li><a href="#ProfilerDisplayProfileSummary">Display Profile Summary</a></li>
|
||
|
|
</ul>
|
||
|
|
</nav>
|
||
|
|
<p>Taskflow comes with a built-in profiler, <em>TFProf</em>, for you to profile and visualize taskflow programs.</p><img class="m-image" src="tfprof.png" alt="Image" style="width: 100%;" /><section id="ProfilerEnableTFProf"><h2><a href="#ProfilerEnableTFProf">Enable Taskflow Profiler</a></h2><p>All taskflow programs come with a lightweight profiling module to observer worker activities in every executor. To enable the profiler, set the environment variable <code>TF_ENABLE_PROFILER</code> to a file name in which the profiling result will be stored.</p><pre class="m-console"><span class="go">~$ TF_ENABLE_PROFILER=result.json ./my_taskflow</span>
|
||
|
|
<span class="go">~$ cat result.json</span>
|
||
|
|
<span class="go">[</span>
|
||
|
|
<span class="go">{"executor":"0","data":[{"worker":12,"level":0,"data":[{"span":[72,117],"name":"12_0","type":"static"},{"span":[121,123],"name":"12_1","type":"static"},{"span":[123,125],"name":"12_2","type":"static"},{"span":[125,127],"name":"12_3","type":"static"}]}]}</span>
|
||
|
|
<span class="go">]</span></pre><p>When the program finishes, it generates and saves the profiling data to <code>result.json</code> in JavaScript Object Notation (JSON) format. You can then paste the JSON data to our web-based interface, <a href="https://taskflow.github.io/tfprof/">Taskflow Profiler</a>, to visualize the execution timelines of tasks and workers. The web interface supports the following features:</p><ul><li>zoom into a selected window</li><li>double click to zoom back to the previously selected window</li><li>filter workers</li><li>mouse over to show the tooltip of the task</li><li>rank tasks in decreasing order of criticality (i.e., execution time)</li></ul><p>TFProf implements a clustering-based algorithm to efficiently visualize tasks and their execution timelines in a browser. Without losing much visual accuracy, each <em>clustered</em> task indicates a group of adjacent tasks clustered by the algorithm, and you can zoom in to see these tasks.</p></section><section id="ProfilerEnableTFProfServer"><h2><a href="#ProfilerEnableTFProfServer">Enable Taskflow Profiler on a HTTP Server</a></h2><p>When profiling large taskflow programs, the method in the previous section may not work because of the limitation of processing large JSON files. For example, a taskflow program of a million tasks can produce several GBs of profiling data, and the profile may respond to your requests very slowly. To solve this problem, we have implemented a C++-based http server optimized for our profiling data. To compile the server, enable the cmake option <code>TF_BUILD_PROFILER</code>. You may visit <a href="install.html" class="m-doc">Building and Installing</a> to understand Taskflow's build environment.</p><pre class="m-console"><span class="gp"># </span>under the build directory
|
||
|
|
<span class="go">~$ cmake ../ -DTF_BUILD_PROFILER=ON</span>
|
||
|
|
<span class="go">~$ make</span></pre><p>After successfully compiling the server, you can find the executable at <code>tfprof/server/tfprof</code>. Now, generate profiling data from running a taskflow program but specify the output file with extension .tfp.</p><pre class="m-console"><span class="go">~$ TF_ENABLE_PROFILER=my_taskflow.tfp ./my_taskflow</span>
|
||
|
|
<span class="go">~$ ls</span>
|
||
|
|
<span class="go">my_taskflow.tfp # my_taskflow.tfp is of binary format</span></pre><p>Launch the server program <code>tfprof/server/tfprof</code> and pass (1) the directory of <code>index.html</code> (default at <code>tfprof/</code>) via the option <code>–mount</code> and (2) the <code>my_taskflow.tfp</code> via the option <code>–input</code>.</p><pre class="m-console"><span class="gp"># </span>under the build/ directory
|
||
|
|
<span class="go">~$ ./tfprof/server/tfprof --mount ../tfprof/ --input my_taskflow.tfp</span></pre><p>Now, open your favorite browser at <code>localhost:8080</code> to visualize and profile your <code>my_taskflow</code> program.</p><img class="m-image" src="tfprof-local.png" alt="Image" /><p>The compiled profiler is a more powerful version than the pure JavaScript-based interface and it is able to more efficiently handle large profiling data under different queries. We currently support the following two view types:</p><ul><li>Cluster: visualize the profiling data using a clustering algorithm with a limit</li><li>Criticality: visualize the top-limit tasks in decreasing order of their execution times</li></ul></section><section id="ProfilerDisplayProfileSummary"><h2><a href="#ProfilerDisplayProfileSummary">Display Profile Summary</a></h2><p>You can display a profile summary by specifying only the environment variable <code>TF_ENABLE_PROFILER</code> without any value. The Taskflow will generate a separate summary report of tasks and workers for each executor created by the program.</p><pre class="m-console"><span class="gp"># </span><span class="nb">enable</span> the environment variable without any value
|
||
|
|
<span class="go">~$ TF_ENABLE_PROFILER= ./my_taskflow_program </span>
|
||
|
|
|
||
|
|
<span class="gp"># </span>your program output
|
||
|
|
<span class="go">...</span>
|
||
|
|
<span class="go">... </span>
|
||
|
|
<span class="go">...</span>
|
||
|
|
<span class="gp"># </span>Taskflow profile summary
|
||
|
|
<span class="go">==Observer 0: 1 workers completed 18 tasks in 28 us</span>
|
||
|
|
<span class="go"> -Task- Count Time (us) Avg (us) Min (us) Max (us)</span>
|
||
|
|
<span class="go"> static 7 5 0.714286 0 4</span>
|
||
|
|
<span class="go"> condition 11 0 0.000000 0 0</span>
|
||
|
|
|
||
|
|
<span class="go"> -Worker- Level Task Count Time (us) Avg (us) Min (us) Max (us)</span>
|
||
|
|
<span class="go"> 14 0 static 7 5 0.714286 0 4</span>
|
||
|
|
<span class="go"> condition 11 0 0.000000 0 0</span>
|
||
|
|
<span class="go"> 18 5 0.277778 0 4</span></pre><p>The report consists of two sections, task summary and worker summary. In the first section, the summary reports for each task type the number of executions (<code>Count</code>), the total execution time (<code>Time</code>), average execution time per task (<code>Avg</code>), and the minimum (<code>Min</code>) and the maximum (<code>Max</code>) execution time among all tasks. Similarly in the second section, the summary reports for each worker the task execution statistics.</p></section>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</article></main>
|
||
|
|
<div class="m-doc-search" id="search">
|
||
|
|
<a href="#!" onclick="return hideSearch()"></a>
|
||
|
|
<div class="m-container">
|
||
|
|
<div class="m-row">
|
||
|
|
<div class="m-col-m-8 m-push-m-2">
|
||
|
|
<div class="m-doc-search-header m-text m-small">
|
||
|
|
<div><span class="m-label m-default">Tab</span> / <span class="m-label m-default">T</span> to search, <span class="m-label m-default">Esc</span> to close</div>
|
||
|
|
<div id="search-symbolcount">…</div>
|
||
|
|
</div>
|
||
|
|
<div class="m-doc-search-content">
|
||
|
|
<form>
|
||
|
|
<input type="search" name="q" id="search-input" placeholder="Loading …" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
|
||
|
|
</form>
|
||
|
|
<noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
|
||
|
|
<div id="search-help" class="m-text m-dim m-text-center">
|
||
|
|
<p class="m-noindent">Search for symbols, directories, files, pages or
|
||
|
|
modules. You can omit any prefix from the symbol or file path; adding a
|
||
|
|
<code>:</code> or <code>/</code> suffix lists all members of given symbol or
|
||
|
|
directory.</p>
|
||
|
|
<p class="m-noindent">Use <span class="m-label m-dim">↓</span>
|
||
|
|
/ <span class="m-label m-dim">↑</span> to navigate through the list,
|
||
|
|
<span class="m-label m-dim">Enter</span> to go.
|
||
|
|
<span class="m-label m-dim">Tab</span> autocompletes common prefix, you can
|
||
|
|
copy a link to the result using <span class="m-label m-dim">⌘</span>
|
||
|
|
<span class="m-label m-dim">L</span> while <span class="m-label m-dim">⌘</span>
|
||
|
|
<span class="m-label m-dim">M</span> produces a Markdown link.</p>
|
||
|
|
</div>
|
||
|
|
<div id="search-notfound" class="m-text m-warning m-text-center">Sorry, nothing was found.</div>
|
||
|
|
<ul id="search-results"></ul>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
<script src="search-v2.js"></script>
|
||
|
|
<script src="searchdata-v2.js" async="async"></script>
|
||
|
|
<footer><nav>
|
||
|
|
<div class="m-container">
|
||
|
|
<div class="m-row">
|
||
|
|
<div class="m-col-l-10 m-push-l-1">
|
||
|
|
<p>Taskflow handbook is part of the <a href="https://taskflow.github.io">Taskflow project</a>, copyright © <a href="https://tsung-wei-huang.github.io/">Dr. Tsung-Wei Huang</a>, 2018–2024.<br />Generated by <a href="https://doxygen.org/">Doxygen</a> 1.9.1 and <a href="https://mcss.mosra.cz/">m.css</a>.</p>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
</nav></footer>
|
||
|
|
</body>
|
||
|
|
</html>
|