index.html (23968B)
1 <!DOCTYPE html> 2 3 <html lang="en"> 4 5 <head> 6 <meta charset="utf-8"> 7 <meta http-equiv="content-type" content="text/html; charset=utf-8" /> 8 9 <link rel="start" href="https://vincent.demeester.fr" /> 10 11 <title>Vincent Demeester</title> 12 <link rel="canonical" href="https://vincent.demeester.fr/posts/2018-09-18-gotest-tools-icmd/"> 13 <link href="https://vincent.demeester.fr/index.xml" rel="alternate" type="application/rss+xml" title="Vincent Demeester" /> 14 15 <link rel="openid.server" href="https://indieauth.com/openid" /> 16 <link rel="openid.delegate" href="http://vincent.demeester.fr/" /> 17 <link rel="shortcut icon" type="image/x-icon" href="favicon.ico"> 18 19 <link rel="stylesheet" href="/css/screen.css" type="text/css" /> 20 <link rel="stylesheet" href="/css/sbrain.css" type="text/css" /> 21 <link rel="stylesheet" href="/css/syntax.css" type="text/css" /> 22 23 </head> 24 25 <body lang=""/> 26 27 28 29 30 31 32 <div id="main-container"> 33 <div id="page"> 34 <article class="post"> 35 <header> 36 <h1 class="emphnext">Golang testing — gotest.tools icmd</h1><a href='https://vincent.demeester.fr/posts/2018-09-18-gotest-tools-icmd/'></a> 37 <address class="signature"> 38 <span class="date">Tue, 18 September, 2018</span> 39 <span class="words">(1100 Words)</span> 40 </address> 41 <ul class="tag_box inline"> 42 43 <li class="category"><a href="/categories/#developement">developement</a></li> 44 45 46 47 48 49 <li class="tag tag-testing"><a href="/tags/#testing">testing<span>11</span></a></li> 50 51 52 <li class="tag tag-golang"><a href="/tags/#golang">golang<span>12</span></a></li> 53 54 55 <li class="tag tag-exec"><a href="/tags/#exec">exec<span>1</span></a></li> 56 57 58 <li class="tag tag-command"><a href="/tags/#command">command<span>1</span></a></li> 59 60 <br/> 61 62 </ul> 63 </header> 64 65 66 67 68 69 <p>Let’s continue the <a href="https://gotest.tools"><code>gotest.tools</code></a> serie, this time with the <code>icmd</code> package.</p> 70 71 <blockquote> 72 <p>Package icmd executes binaries and provides convenient assertions for testing the results.</p> 73 </blockquote> 74 75 <p>After file-system operations (seen in <a href="/posts/2018-09-14-gotest-tools-fs/"><code>fs</code></a>), another common use-case in tests is to 76 <strong>execute a command</strong>. The reasons can be you’re testing the <code>cli</code> you’re currently writing 77 or you need to setup something using a command line. A classic execution in a test might 78 lookup like the following.</p> 79 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="nx">cmd</span> <span class="o">:=</span> <span class="nx">exec</span><span class="p">.</span><span class="nf">Command</span><span class="p">(</span><span class="s">"echo"</span><span class="p">,</span> <span class="s">"foo"</span><span class="p">)</span> 80 <span class="nx">cmd</span><span class="p">.</span><span class="nx">Stout</span> <span class="p">=</span> <span class="o">&</span><span class="nx">stdout</span> 81 <span class="nx">cmd</span><span class="p">.</span><span class="nx">Env</span> <span class="p">=</span> <span class="nx">env</span> 82 <span class="k">if</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">cmd</span><span class="p">.</span><span class="nf">Run</span><span class="p">();</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> 83 <span class="nx">t</span><span class="p">.</span><span class="nf">Fatal</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span> 84 <span class="p">}</span> 85 <span class="k">if</span> <span class="nb">string</span><span class="p">(</span><span class="nx">stdout</span><span class="p">)</span> <span class="o">!=</span> <span class="s">"foo"</span> <span class="p">{</span> 86 <span class="nx">t</span><span class="p">.</span><span class="nf">Fatalf</span><span class="p">(</span><span class="s">"expected: foo, got %s"</span><span class="p">,</span> <span class="nb">string</span><span class="p">(</span><span class="nx">stdout</span><span class="p">))</span> 87 <span class="p">}</span></code></pre></div> 88 <p>The package <code>icmd</code> is there to ease your pain (as usual 😉) — we used <em>the name <code>icmd</code></em> 89 instead of <code>cmd</code> because it’s a pretty common identifier in Go source code, thus would be 90 really easy to <em>shadow</em> — and have some really weird problems going on.</p> 91 92 <p>The usual <code>icmd</code> workflow is the following:</p> 93 94 <ol> 95 <li>Describe the command you want to execute using : type <code>Cmd</code>, function <code>Command</code> and 96 <code>CmdOp</code> operators.</li> 97 <li>Run it using : function <code>RunCmd</code> or <code>RunCommand</code> (that does 1. for you). You can also 98 use <code>StartCmd</code> and <code>WaitOnCmd</code> if you want more control on the execution workflow.</li> 99 <li>Check the result using the <code>Assert</code>, <code>Equal</code> or <code>Compare</code> methods attached to the 100 <code>Result</code> struct that the command execution return.</li> 101 </ol> 102 103 <h2 id="create-and-run-a-command">Create and run a command</h2> 104 105 <p>Let’s first dig how to create commands. In this part, the assumption here is that the 106 command is successful, so we’ll have <code>.Assert(t, icmd.Success)</code> for now — we’ll learn more 107 about <code>Assert</code> in the next section 👼.</p> 108 109 <p>The simplest way to create and run a command is using <code>RunCommand</code>, it has the same 110 signature as <code>os/exec.Command</code>. A simple command execution goes as below.</p> 111 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCommand</span><span class="p">(</span><span class="s">"echo"</span><span class="p">,</span> <span class="s">"foo"</span><span class="p">).</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">icmd</span><span class="p">.</span><span class="nx">Sucess</span><span class="p">)</span></code></pre></div> 112 <p>Sometimes, you need to customize the command a bit more, like adding some environment 113 variable. In those case, you are going to use <code>RunCmd</code>, it takes a <code>Cmd</code> and operators. 114 Let’s look at those functions.</p> 115 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="kd">func</span> <span class="nf">RunCmd</span><span class="p">(</span><span class="nx">cmd</span> <span class="nx">Cmd</span><span class="p">,</span> <span class="nx">cmdOperators</span> <span class="o">...</span><span class="nx">CmdOp</span><span class="p">)</span> <span class="o">*</span><span class="nx">Result</span> 116 117 <span class="kd">func</span> <span class="nf">Command</span><span class="p">(</span><span class="nx">command</span> <span class="kt">string</span><span class="p">,</span> <span class="nx">args</span> <span class="o">...</span><span class="kt">string</span><span class="p">)</span> <span class="nx">Cmd</span> 118 119 <span class="kd">type</span> <span class="nx">Cmd</span> <span class="kd">struct</span> <span class="p">{</span> 120 <span class="nx">Command</span> <span class="p">[]</span><span class="kt">string</span> 121 <span class="nx">Timeout</span> <span class="nx">time</span><span class="p">.</span><span class="nx">Duration</span> 122 <span class="nx">Stdin</span> <span class="nx">io</span><span class="p">.</span><span class="nx">Reader</span> 123 <span class="nx">Stdout</span> <span class="nx">io</span><span class="p">.</span><span class="nx">Writer</span> 124 <span class="nx">Dir</span> <span class="kt">string</span> 125 <span class="nx">Env</span> <span class="p">[]</span><span class="kt">string</span> 126 <span class="p">}</span></code></pre></div> 127 <p>As we’ve seen <a href="/posts/2017-01-01-go-testing-functionnal-builders/">multiple</a> <a href="/posts/2018-08-16-gotest-tools-assertions/">times</a> <a href="/posts/2018-09-14-gotest-tools-fs/">before</a>, it uses the <em>powerful</em> functional arguments. At the 128 time I wrote this post, the <code>icmd</code> package doesn’t contains too much <code>CmdOp</code> <sup class="footnote-ref" id="fnref:fn-1"><a href="#fn:fn-1">1</a></sup>, so I’ll 129 propose two version for each example : one with <code>CmdOpt</code> present in <a href="https://github.com/gotestyourself/gotest.tools/pull/122">this PR</a> and one 130 without them.</p> 131 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="c1">// With 132 </span><span class="c1"></span><span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCmd</span><span class="p">(</span><span class="nx">icmd</span><span class="p">.</span><span class="nf">Command</span><span class="p">(</span><span class="s">"sh"</span><span class="p">,</span> <span class="s">"-c"</span><span class="p">,</span> <span class="s">"echo $FOO"</span><span class="p">),</span> 133 <span class="nx">icmd</span><span class="p">.</span><span class="nf">WithEnv</span><span class="p">(</span><span class="s">"FOO=bar"</span><span class="p">,</span> <span class="s">"BAR=baz"</span><span class="p">),</span> <span class="nx">icmd</span><span class="p">.</span><span class="nf">Dir</span><span class="p">(</span><span class="s">"/tmp"</span><span class="p">),</span> 134 <span class="nx">icmd</span><span class="p">.</span><span class="nf">WithTimeout</span><span class="p">(</span><span class="mi">10</span><span class="o">*</span><span class="nx">time</span><span class="p">.</span><span class="nx">Second</span><span class="p">),</span> 135 <span class="p">).</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">icmd</span><span class="p">.</span><span class="nx">Success</span><span class="p">)</span> 136 137 <span class="c1">// Without 138 </span><span class="c1"></span><span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCmd</span><span class="p">(</span><span class="nx">icmd</span><span class="p">.</span><span class="nx">Cmd</span><span class="p">{</span> 139 <span class="nx">Command</span><span class="p">:</span> <span class="p">[]</span><span class="kt">string</span><span class="p">{</span><span class="s">"sh"</span><span class="p">,</span> <span class="s">"-c"</span><span class="p">,</span> <span class="s">"echo $FOO"</span><span class="p">},</span> 140 <span class="nx">Env</span><span class="p">:</span> <span class="p">[]</span><span class="kt">string</span><span class="p">{</span><span class="s">"FOO=bar"</span><span class="p">,</span> <span class="s">"BAR=baz"</span><span class="p">},</span> 141 <span class="nx">Dir</span><span class="p">:</span> <span class="s">"/tmp"</span><span class="p">,</span> 142 <span class="nx">Timeout</span><span class="p">:</span> <span class="mi">10</span><span class="o">*</span><span class="nx">time</span><span class="p">.</span><span class="nx">Second</span><span class="p">,</span> 143 <span class="p">}).</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">icmd</span><span class="p">.</span><span class="nx">Success</span><span class="p">)</span></code></pre></div> 144 <p>As usual, the intent is clear, it’s simple to read and composable (with <code>CmdOp</code>’s).</p> 145 146 <h2 id="assertions">Assertions</h2> 147 148 <p>Let’s dig into the assertion part of <code>icmd</code>. Running a command returns a struct 149 <code>Result</code>. It has the following methods :</p> 150 151 <ul> 152 <li><code>Assert</code> compares the Result against the Expected struct, and fails the test if any of 153 the expectations are not met.</li> 154 <li><code>Compare</code> compares the result to Expected and return an error if they do not match.</li> 155 <li><code>Equal</code> compares the result to Expected. If the result doesn’t match expected 156 returns a formatted failure message with the command, stdout, stderr, exit code, and any 157 failed expectations. It returns an <code>assert.Comparison</code> struct, that can be used by other 158 <code>gotest.tools</code>.</li> 159 <li><code>Combined</code> returns the stdout and stderr combined into a single string.</li> 160 <li><code>Stderr</code> returns the stderr of the process as a string.</li> 161 <li><code>Stdout</code> returns the stdout of the process as a string.</li> 162 </ul> 163 164 <p>When you have a result, you, most likely want to do two things :</p> 165 166 <ul> 167 <li><em>assert</em> that the command succeed or failed with some specific values (exit code, 168 stderr, stdout)</li> 169 <li>use the output — most likely <code>stdout</code> but maybe <code>stderr</code> — in the rest of the test.</li> 170 </ul> 171 172 <p>As seen above, <em>asserting</em> the command result is using the <code>Expected</code> struct.</p> 173 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="kd">type</span> <span class="nx">Expected</span> <span class="kd">struct</span> <span class="p">{</span> 174 <span class="nx">ExitCode</span> <span class="kt">int</span> <span class="c1">// the exit code the command returned 175 </span><span class="c1"></span> <span class="nx">Timeout</span> <span class="kt">bool</span> <span class="c1">// did it timeout ? 176 </span><span class="c1"></span> <span class="nx">Error</span> <span class="kt">string</span> <span class="c1">// error returned by the execution (os/exe) 177 </span><span class="c1"></span> <span class="nx">Out</span> <span class="kt">string</span> <span class="c1">// content of stdout 178 </span><span class="c1"></span> <span class="nx">Err</span> <span class="kt">string</span> <span class="c1">// content of stderr 179 </span><span class="c1"></span><span class="p">}</span></code></pre></div> 180 <p><code>Success</code> is a constant that defines a success — it’s an exit code of <code>0</code>, didn’t timeout, 181 no error. There is also the <code>None</code> constant, that should be used for <code>Out</code> or <code>Err</code>, to 182 specify that we don’t want any content for those standard outputs.</p> 183 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCmd</span><span class="p">(</span><span class="nx">icmd</span><span class="p">.</span><span class="nf">Command</span><span class="p">(</span><span class="s">"cat"</span><span class="p">,</span> <span class="s">"/does/not/exist"</span><span class="p">)).</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">icmd</span><span class="p">.</span><span class="nx">Expected</span><span class="p">{</span> 184 <span class="nx">ExitCode</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> 185 <span class="nx">Err</span><span class="p">:</span> <span class="s">"cat: /does/not/exist: No such file or directory"</span><span class="p">,</span> 186 <span class="p">})</span> 187 188 <span class="c1">// In case of success, we may want to do something with the result 189 </span><span class="c1"></span><span class="nx">result</span> <span class="o">:=</span> <span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCommand</span><span class="p">(</span><span class="s">"cat"</span><span class="p">,</span> <span class="s">"/does/exist"</span><span class="p">)</span> 190 <span class="nx">result</span><span class="p">.</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">icmd</span><span class="p">.</span><span class="nx">Success</span><span class="p">)</span> 191 <span class="c1">// Read the output line by line 192 </span><span class="c1"></span><span class="nx">scanner</span> <span class="o">:=</span> <span class="nx">bufio</span><span class="p">.</span><span class="nf">NewScanner</span><span class="p">(</span><span class="nx">strings</span><span class="p">.</span><span class="nf">NewReader</span><span class="p">(</span><span class="nx">result</span><span class="p">.</span><span class="nf">Stdout</span><span class="p">()))</span> 193 <span class="k">for</span> <span class="nx">scanner</span><span class="p">.</span><span class="nf">Scan</span><span class="p">()</span> <span class="p">{</span> 194 <span class="c1">// Do something with it 195 </span><span class="c1"></span><span class="p">}</span></code></pre></div> 196 <p>If the <code>Result</code> doesn’t map the <code>Expected</code>, a test failure will happen with a useful 197 message that will contains the executed command and what differs between the result and 198 the expectation.</p> 199 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="nx">result</span> <span class="o">:=</span> <span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCommand</span><span class="p">(</span><span class="err">…</span><span class="p">)</span> 200 <span class="nx">result</span><span class="p">.</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">icmd</span><span class="p">.</span><span class="nx">Expected</span><span class="p">{</span> 201 <span class="nx">ExitCode</span><span class="p">:</span> <span class="mi">101</span><span class="p">,</span> 202 <span class="nx">Out</span><span class="p">:</span> <span class="s">"Something else"</span><span class="p">,</span> 203 <span class="nx">Err</span><span class="p">:</span> <span class="nx">None</span><span class="p">,</span> 204 <span class="p">})</span> 205 <span class="c1">// Command: binary arg1 206 </span><span class="c1">// ExitCode: 99 (timeout) 207 </span><span class="c1">// Error: exit code 99 208 </span><span class="c1">// Stdout: the output 209 </span><span class="c1">// Stderr: the stderr 210 </span><span class="c1">// 211 </span><span class="c1">// Failures: 212 </span><span class="c1">// ExitCode was 99 expected 101 213 </span><span class="c1">// Expected command to finish, but it hit the timeout 214 </span><span class="c1">// Expected stdout to contain "Something else" 215 </span><span class="c1">// Expected stderr to contain "[NOTHING]" 216 </span><span class="c1"></span><span class="err">…</span></code></pre></div> 217 <p>Finally, we listed <code>Equal</code> above, that returns a <code>Comparison</code> struct. This means we can 218 use it easily with the <code>assert</code> package. As written in a <a href="/posts/2018-08-16-gotest-tools-assertions/">previous post (about the <code>assert</code> 219 package)</a>, I tend prefer to use <code>cmp.Comparison</code>. Let’s convert the above examples using 220 <code>assert</code>.</p> 221 <div class="highlight"><pre class="chroma"><code class="language-go" data-lang="go"><span class="nx">result</span> <span class="o">:=</span> <span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCmd</span><span class="p">(</span><span class="nx">icmd</span><span class="p">.</span><span class="nf">Command</span><span class="p">(</span><span class="s">"cat"</span><span class="p">,</span> <span class="s">"/does/not/exist"</span><span class="p">))</span> 222 <span class="nx">assert</span><span class="p">.</span><span class="nf">Check</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">result</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="nx">icmd</span><span class="p">.</span><span class="nx">Expected</span><span class="p">{</span> 223 <span class="nx">ExitCode</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> 224 <span class="nx">Err</span><span class="p">:</span> <span class="s">"cat: /does/not/exist: No such file or directory"</span><span class="p">,</span> 225 <span class="p">}))</span> 226 227 <span class="c1">// In case of success, we may want to do something with the result 228 </span><span class="c1"></span><span class="nx">result</span> <span class="o">:=</span> <span class="nx">icmd</span><span class="p">.</span><span class="nf">RunCommand</span><span class="p">(</span><span class="s">"cat"</span><span class="p">,</span> <span class="s">"/does/exist"</span><span class="p">)</span> 229 <span class="nx">assert</span><span class="p">.</span><span class="nf">Assert</span><span class="p">(</span><span class="nx">t</span><span class="p">,</span> <span class="nx">result</span><span class="p">.</span><span class="nf">Equal</span><span class="p">(</span><span class="nx">icmd</span><span class="p">.</span><span class="nx">Success</span><span class="p">))</span> 230 <span class="c1">// Read the output line by line 231 </span><span class="c1"></span><span class="nx">scanner</span> <span class="o">:=</span> <span class="nx">bufio</span><span class="p">.</span><span class="nf">NewScanner</span><span class="p">(</span><span class="nx">strings</span><span class="p">.</span><span class="nf">NewReader</span><span class="p">(</span><span class="nx">result</span><span class="p">.</span><span class="nf">Stdout</span><span class="p">()))</span> 232 <span class="k">for</span> <span class="nx">scanner</span><span class="p">.</span><span class="nf">Scan</span><span class="p">()</span> <span class="p">{</span> 233 <span class="c1">// Do something with it 234 </span><span class="c1"></span><span class="p">}</span></code></pre></div> 235 <h2 id="conclusion">Conclusion…</h2> 236 237 <p>… that’s a wrap. The <code>icmd</code> package allows to easily run command and describe what result 238 are expected of the execution, with the least noise possible. We <strong>use this package heavily</strong> 239 on several <code>docker/*</code> projects (the engine, the cli)…</p> 240 <div class="footnotes"> 241 242 <hr /> 243 244 <ol> 245 <li id="fn:fn-1">The <code>icmd</code> package is one of the oldest <code>gotest.tools</code> package, that comes from the <a href="https://github.com/docker/docker"><code>docker/docker</code></a> initially. We introduced these <code>CmdOp</code> but implementations were in <code>docker/docker</code> at first and we never really updated them. 246 <a class="footnote-return" href="#fnref:fn-1"><sup>[return]</sup></a></li> 247 </ol> 248 </div> 249 250 251 </article> 252 <hr /> 253 <div class="prev-next"> 254 255 <a class="paging-link prev" href="/posts/2018-12-16-link/" title="Go testing style guide">← Previous post</a> 256 257 258 259 <a class="paging-link next" href="/posts/2018-09-14-gotest-tools-fs/" title="Golang testing — gotest.tools fs">Next post →</a> 260 261 </div> 262 263 </div> 264 </div> 265 266 <footer> 267 <nav> 268 269 <a href="/">home</a> 270 <span class="text-muted"> | </span> 271 272 <a href="/about">about</a> 273 <span class="text-muted"> | </span> 274 275 <a href="/archive">archive</a> 276 <span class="text-muted"> | </span> 277 278 <a href="/categories">categories</a> 279 <span class="text-muted"> | </span> 280 281 <a href="/tags">tags</a> 282 <span class="text-muted"> | </span> 283 284 <a href="https://twitter.com/vdemeest">twitter</a> 285 <span class="text-muted"> | </span> 286 287 <a href="https://github.com/vdemeester">github</a> 288 <span class="text-muted"> | </span> 289 290 <a href="https://vincent.demeester.fr/index.xml">rss</a> 291 </nav> 292 <br/> 293 <address> 294 <span class="copyright"> 295 Content and design by Vincent Demeester 296 (<a rel="licence" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Some rights reserved</a>) 297 </span><br /> 298 <span class="engine"> 299 Powered by <a href="https://gohugo.io/">Hugo</a> and <a href="https://github.com/kaushalmodi/ox-hugo/">ox-hugo</a> 300 </span> 301 </address> 302 </footer> 303 </body> 304