home

Unnamed repository; edit this file 'description' to name the repository.
Log | Files | Refs | README | LICENSE

index.html (21925B)


      1 <!DOCTYPE html>
      2 
      3 <html lang="en">
      4   
      5   <head>
      6     <meta charset="utf-8">
      7     <meta http-equiv="content-type" content="text/html; charset=utf-8" />
      8 
      9     <link rel="start" href="https://vincent.demeester.fr" />
     10 
     11     <title>Vincent Demeester</title>
     12     <link rel="canonical" href="https://vincent.demeester.fr/posts/2017-04-22-golang-testing-golden-file/">
     13     <link href="https://vincent.demeester.fr/index.xml" rel="alternate" type="application/rss+xml" title="Vincent Demeester" />
     14 
     15     <link rel="openid.server" href="https://indieauth.com/openid" />
     16     <link rel="openid.delegate" href="http://vincent.demeester.fr/" />
     17     <link rel="shortcut icon" type="image/x-icon" href="favicon.ico">
     18 
     19     <link rel="stylesheet" href="/css/screen.css" type="text/css" />
     20     <link rel="stylesheet" href="/css/sbrain.css" type="text/css" />
     21     <link rel="stylesheet" href="/css/syntax.css" type="text/css" />
     22 
     23   </head>
     24   
     25   <body lang="en"/>
     26   
     27 
     28 
     29 
     30 
     31 
     32 <div id="main-container">
     33   <div id="page">
     34     <article class="post">
     35       <header>
     36         <h1 class="emphnext">Golang testing — golden file</h1><a href='https://vincent.demeester.fr/posts/2017-04-22-golang-testing-golden-file/'></a>
     37         <address class="signature">
     38           <span class="date">Sat, 22 April, 2017</span>
     39           <span class="words">(900 Words)</span>
     40         </address>
     41 	<ul class="tag_box inline">
     42 	  
     43 	  <li class="category"><a href="/categories/#developement">developement</a></li>
     44 	  
     45 	  
     46 	  
     47 	  
     48 	  
     49 	  <li class="tag tag-golang"><a href="/tags/#golang">golang<span>12</span></a></li>
     50 	  
     51 	  
     52 	  <li class="tag tag-testing"><a href="/tags/#testing">testing<span>11</span></a></li>
     53 	  
     54 	  
     55 	  <li class="tag tag-golden"><a href="/tags/#golden">golden<span>1</span></a></li>
     56 	  
     57 	  
     58 	  <li class="tag tag-file"><a href="/tags/#file">file<span>1</span></a></li>
     59 	  
     60 	  
     61 	  <li class="tag tag-functionnal"><a href="/tags/#functionnal">functionnal<span>2</span></a></li>
     62 	  
     63 	  
     64 	  <li class="tag tag-java"><a href="/tags/#java">java<span>4</span></a></li>
     65 	  
     66 	  <br/>
     67 	  
     68 	</ul>
     69       </header>
     70       
     71       
     72       
     73       <p>
     74 Tests are all about <b>maintainability</b> and <b>readability</b>. You want the
     75 least visual noise possible and it should not be a hassle to
     76 maintain. When testing functions that output a long of string, in case
     77 of a command line output testing, readability and maintainance tend to
     78 be tricky to achieve.
     79 </p>
     80 
     81 
     82 <div id="outline-container-org2f1fc39" class="outline-2">
     83 <h2 id="org2f1fc39">The problem</h2>
     84 <div class="outline-text-2" id="text-org2f1fc39">
     85 <p>
     86 As an example let's say we want to test out the output of a command
     87 that displays a list as a table. The output would look like the
     88 following:
     89 </p>
     90 
     91 <div class="org-src-container">
     92 <pre class="src src-sh"><span class="org-rainbow-identifiers-identifier-3">ID:</span>                     <span class="org-rainbow-identifiers-identifier-15">nodeID</span>
     93 <span class="org-rainbow-identifiers-identifier-4">Name:</span>                   <span class="org-rainbow-identifiers-identifier-8">defaultNodeName</span>
     94 <span class="org-rainbow-identifiers-identifier-10">Hostname:</span>               <span class="org-rainbow-identifiers-identifier-6">defaultNodeHostname</span>
     95 <span class="org-rainbow-identifiers-identifier-4">Joined</span> <span class="org-rainbow-identifiers-identifier-1">at:</span>              <span class="org-rainbow-identifiers-identifier-11">2009-11-10</span> <span class="org-rainbow-identifiers-identifier-11">23:00:00</span> <span class="org-rainbow-identifiers-identifier-13">+0000</span> <span class="org-rainbow-identifiers-identifier-15">utc</span>
     96 <span class="org-rainbow-identifiers-identifier-3">Status:</span>
     97  <span class="org-rainbow-identifiers-identifier-4">State:</span>                 <span class="org-rainbow-identifiers-identifier-10">Ready</span>
     98  <span class="org-rainbow-identifiers-identifier-11">Availability:</span>          <span class="org-rainbow-identifiers-identifier-8">Active</span>
     99  <span class="org-rainbow-identifiers-identifier-11">Address:</span>               <span class="org-rainbow-identifiers-identifier-13">127.0.0.1</span>
    100 <span class="org-rainbow-identifiers-identifier-9">Manager</span> <span class="org-rainbow-identifiers-identifier-3">Status:</span>
    101  <span class="org-rainbow-identifiers-identifier-11">Address:</span>               <span class="org-rainbow-identifiers-identifier-13">127.0.0.1</span>
    102  <span class="org-rainbow-identifiers-identifier-3">Raft</span> <span class="org-rainbow-identifiers-identifier-3">Status:</span>           <span class="org-rainbow-identifiers-identifier-8">Reachable</span>
    103  <span class="org-rainbow-identifiers-identifier-8">Leader:</span>                <span class="org-rainbow-identifiers-identifier-15">No</span>
    104 <span class="org-rainbow-identifiers-identifier-3">Platform:</span>
    105  <span class="org-rainbow-identifiers-identifier-15">Operating</span> <span class="org-rainbow-identifiers-identifier-13">System:</span>      <span class="org-rainbow-identifiers-identifier-12">linux</span>
    106  <span class="org-rainbow-identifiers-identifier-10">Architecture:</span>          <span class="org-rainbow-identifiers-identifier-2">x86_64</span>
    107 <span class="org-rainbow-identifiers-identifier-8">Resources:</span>
    108  <span class="org-rainbow-identifiers-identifier-8">CPUs:</span>                  <span class="org-rainbow-identifiers-identifier-12">0</span>
    109  <span class="org-rainbow-identifiers-identifier-9">Memory:</span>                <span class="org-rainbow-identifiers-identifier-15">20</span> <span class="org-rainbow-identifiers-identifier-5">MiB</span>
    110 <span class="org-rainbow-identifiers-identifier-14">Plugins:</span>
    111   <span class="org-rainbow-identifiers-identifier-5">Network:</span>              <span class="org-rainbow-identifiers-identifier-3">bridge,</span> <span class="org-rainbow-identifiers-identifier-12">overlay</span>
    112   <span class="org-rainbow-identifiers-identifier-9">Volume:</span>               <span class="org-rainbow-identifiers-identifier-12">local</span>
    113 <span class="org-rainbow-identifiers-identifier-10">Engine</span> <span class="org-rainbow-identifiers-identifier-14">Version:</span>         <span class="org-rainbow-identifiers-identifier-15">1.13.0</span>
    114 <span class="org-rainbow-identifiers-identifier-10">Engine</span> <span class="org-rainbow-identifiers-identifier-12">Labels:</span>
    115  <span class="org-rainbow-identifiers-identifier-11">-</span> <span class="org-variable-name">engine</span> = <span class="org-rainbow-identifiers-identifier-12">label</span>
    116 </pre>
    117 </div>
    118 
    119 <p>
    120 Let's see how we would test that output, naively.
    121 </p>
    122 
    123 <div class="org-src-container">
    124 <pre class="src src-go"><span class="org-keyword">func</span> <span class="org-function-name">TestNodeInspectPretty</span>(<span class="org-rainbow-identifiers-identifier-8">t</span> *<span class="org-type">testing.T</span>) {
    125         <span class="org-rainbow-identifiers-identifier-14">expected</span> := <span class="org-string">```</span>
    126 <span class="org-string">ID:                     nodeID</span>
    127 <span class="org-string">Name:                   defaultNodeName</span>
    128 <span class="org-string">Hostname:               defaultNodeHostname</span>
    129 <span class="org-string">Joined at:              2009-11-10 23:00:00 +0000 utc</span>
    130 <span class="org-string">Status:</span>
    131 <span class="org-string"> State:                 Ready</span>
    132 <span class="org-string"> Availability:          Active</span>
    133 <span class="org-string"> Address:               127.0.0.1</span>
    134 <span class="org-string">Manager Status:</span>
    135 <span class="org-string"> Address:               127.0.0.1</span>
    136 <span class="org-string"> Raft Status:           Reachable</span>
    137 <span class="org-string"> Leader:                No</span>
    138 <span class="org-string">Platform:</span>
    139 <span class="org-string"> Operating System:      linux</span>
    140 <span class="org-string"> Architecture:          x86_64</span>
    141 <span class="org-string">Resources:</span>
    142 <span class="org-string"> CPUs:                  0</span>
    143 <span class="org-string"> Memory:                20 MiB</span>
    144 <span class="org-string">Plugins:</span>
    145 <span class="org-string">  Network:              bridge, overlay</span>
    146 <span class="org-string">  Volume:               local</span>
    147 <span class="org-string">Engine Version:         1.13.0</span>
    148 <span class="org-string">Engine Labels:</span>
    149 <span class="org-string"> - engine = label</span>
    150 <span class="org-string">```</span>
    151         <span class="org-rainbow-identifiers-identifier-3">buf</span> := <span class="org-builtin">new</span>(<span class="org-type">bytes.Buffer</span>)
    152         <span class="org-rainbow-identifiers-identifier-3">cmd</span> := <span class="org-function-name">newInspectCommand</span>(
    153                 <span class="org-rainbow-identifiers-identifier-5">test</span>.<span class="org-function-name">NewFakeCli</span>(&amp;<span class="org-type">fakeClient</span>{
    154                         <span class="org-constant">nodeInspectFunc</span>: <span class="org-keyword">func</span>() (<span class="org-type">swarm.Node</span>, []<span class="org-type">byte</span>, <span class="org-type">error</span>) {
    155                                 <span class="org-keyword">return</span> *<span class="org-function-name">Node</span>(<span class="org-function-name">Manager</span>()), []<span class="org-type">byte</span>{}, <span class="org-constant">nil</span>
    156                         },
    157                 }, <span class="org-rainbow-identifiers-identifier-3">buf</span>))
    158         <span class="org-rainbow-identifiers-identifier-3">cmd</span>.<span class="org-function-name">SetArgs</span>([]<span class="org-type">string</span>{<span class="org-string">"nodeID"</span>})
    159         <span class="org-rainbow-identifiers-identifier-3">cmd</span>.<span class="org-function-name">Flags</span>().<span class="org-function-name">Set</span>(<span class="org-string">"pretty"</span>, <span class="org-string">"true"</span>)
    160         <span class="org-rainbow-identifiers-identifier-11">assert</span>.<span class="org-function-name">NilError</span>(<span class="org-rainbow-identifiers-identifier-8">t</span>, <span class="org-rainbow-identifiers-identifier-3">cmd</span>.<span class="org-function-name">Execute</span>())
    161         <span class="org-rainbow-identifiers-identifier-15">actual</span> := <span class="org-rainbow-identifiers-identifier-3">buf</span>.<span class="org-function-name">String</span>()
    162         <span class="org-rainbow-identifiers-identifier-11">assert</span>.<span class="org-function-name">EqualNormalizedString</span>(<span class="org-rainbow-identifiers-identifier-8">t</span>, <span class="org-rainbow-identifiers-identifier-11">assert</span>.<span class="org-rainbow-identifiers-identifier-2">RemoveSpace</span>, <span class="org-rainbow-identifiers-identifier-15">actual</span>, <span class="org-function-name">string</span>(<span class="org-rainbow-identifiers-identifier-14">expected</span>))
    163 }
    164 </pre>
    165 </div>
    166 
    167 <p>
    168 This might look ok as is, but a few problem are present:
    169 </p>
    170 
    171 <ul class="org-ul">
    172 <li>The output is quite large and adds some noise to the test</li>
    173 <li><p>
    174 Lot's of value in the <code>expected</code> string comes from default values
    175 of our <code>Node</code> builder.
    176 </p>
    177 
    178 <p>
    179 This means any time we change our builder default values, we would
    180 need to update this test, <i>painful</i>.
    181 </p></li>
    182 <li>If the output changes for a good reason (add a field, fix a typo,
    183 …), this test has to be updated too.</li>
    184 </ul>
    185 </div>
    186 </div>
    187 
    188 <div id="outline-container-org0519f3e" class="outline-2">
    189 <h2 id="org0519f3e">Golden files to the rescue</h2>
    190 <div class="outline-text-2" id="text-org0519f3e">
    191 <p>
    192 First, let's get back at what is our test about and what we actually
    193 want to test.
    194 </p>
    195 
    196 <ul class="org-ul">
    197 <li>We want to ensure, the output of the function does not change by
    198 mistake (i.e. change that wasn't supposed to change the output)</li>
    199 <li>We want to have an update version of the output if that was the
    200 purpose of our change. And we want this update to be the least
    201 painful possible.</li>
    202 <li>We don't really care about the final outputs as long as it stays
    203 the same for the same inputs (i.e. we don't test for any number of
    204 space, or that word are valid English, or …).</li>
    205 </ul>
    206 
    207 <p>
    208 This is where the concept of <b>golden file</b> is useful. In a
    209 nutshell, a golden file is a file where we store the output and that
    210 will be used by the test as the expected output. This file should be
    211 updated any time the output changes for good reason. That's that
    212 simple 👼.
    213 </p>
    214 
    215 <p>
    216 Once again, the way go <code>testing</code> works, introducing and using golden
    217 files in our tests is pretty straightforward and easy to use.
    218 </p>
    219 
    220 <p>
    221 Let's write a small <i>golden file helper</i> so that our test has no
    222 visual noise, in a <code>golden</code> package.
    223 </p>
    224 
    225 <div class="org-src-container">
    226 <pre class="src src-go"><span class="org-keyword">var</span> <span class="org-rainbow-identifiers-identifier-3">update</span> = <span class="org-rainbow-identifiers-identifier-15">flag</span>.<span class="org-function-name">Bool</span>(<span class="org-string">"test.update"</span>, <span class="org-constant">false</span>, <span class="org-string">"update golden file"</span>)
    227 
    228 <span class="org-comment-delimiter">// </span><span class="org-comment">Get returns the golden file content. If the `test.update` is specified, it updates the</span>
    229 <span class="org-comment-delimiter">// </span><span class="org-comment">file with the current output and returns it.</span>
    230 <span class="org-keyword">func</span> <span class="org-function-name">Get</span>(<span class="org-rainbow-identifiers-identifier-8">t</span> *<span class="org-type">testing.T</span>, <span class="org-rainbow-identifiers-identifier-15">actual</span> []<span class="org-type">byte</span>, <span class="org-rainbow-identifiers-identifier-5">filename</span> <span class="org-type">string</span>) []<span class="org-type">byte</span> {
    231         <span class="org-rainbow-identifiers-identifier-1">golden</span> := <span class="org-rainbow-identifiers-identifier-9">filepath</span>.<span class="org-function-name">Join</span>(<span class="org-string">"testdata"</span>, <span class="org-rainbow-identifiers-identifier-5">filename</span>)
    232         <span class="org-keyword">if</span> *<span class="org-rainbow-identifiers-identifier-3">update</span> {
    233                 <span class="org-keyword">if</span> <span class="org-rainbow-identifiers-identifier-10">err</span> := <span class="org-rainbow-identifiers-identifier-1">ioutil</span>.<span class="org-function-name">WriteFile</span>(<span class="org-rainbow-identifiers-identifier-1">golden</span>, <span class="org-rainbow-identifiers-identifier-15">actual</span>, <span class="org-rainbow-identifiers-identifier-4">0644</span>); <span class="org-rainbow-identifiers-identifier-10">err</span> != <span class="org-constant">nil</span> {
    234                         <span class="org-rainbow-identifiers-identifier-8">t</span>.<span class="org-function-name">Fatal</span>(<span class="org-rainbow-identifiers-identifier-10">err</span>)
    235                 }
    236         }
    237         <span class="org-rainbow-identifiers-identifier-14">expected</span>, <span class="org-rainbow-identifiers-identifier-10">err</span> := <span class="org-rainbow-identifiers-identifier-1">ioutil</span>.<span class="org-function-name">ReadFile</span>(<span class="org-rainbow-identifiers-identifier-1">golden</span>)
    238         <span class="org-keyword">if</span> <span class="org-rainbow-identifiers-identifier-10">err</span> != <span class="org-constant">nil</span> {
    239                 <span class="org-rainbow-identifiers-identifier-8">t</span>.<span class="org-function-name">Fatal</span>(<span class="org-rainbow-identifiers-identifier-10">err</span>)
    240         }
    241         <span class="org-keyword">return</span> <span class="org-rainbow-identifiers-identifier-14">expected</span>
    242 }
    243 </pre>
    244 </div>
    245 
    246 <ul class="org-ul">
    247 <li>We define a <i>global</i> flag, <code>-test.update</code> that will <i>enhance</i> the
    248 <code>go test</code> command with it (as soon as we import this <code>golden</code> package).</li>
    249 <li><p>
    250 We define a <code>golden.Get</code> function that takes the current output
    251 and the path of the golden file. It also takes <code>testing.T</code> so any
    252 failure happening here (like reading file, …) will make the test
    253 fail (one less thing to write in the test calling this function).
    254 </p>
    255 
    256 <p>
    257 If the flag is present when running the test, it will update the
    258 file with the actual content.
    259 </p></li>
    260 </ul>
    261 
    262 <p>
    263 The initial test becomes.
    264 </p>
    265 
    266 <div class="org-src-container">
    267 <pre class="src src-go"><span class="org-keyword">func</span> <span class="org-function-name">TestNodeInspectPretty</span>(<span class="org-rainbow-identifiers-identifier-8">t</span> *<span class="org-type">testing.T</span>) {
    268         <span class="org-rainbow-identifiers-identifier-3">buf</span> := <span class="org-builtin">new</span>(<span class="org-type">bytes.Buffer</span>)
    269         <span class="org-rainbow-identifiers-identifier-3">cmd</span> := <span class="org-function-name">newInspectCommand</span>(
    270                 <span class="org-rainbow-identifiers-identifier-5">test</span>.<span class="org-function-name">NewFakeCli</span>(&amp;<span class="org-type">fakeClient</span>{
    271                         <span class="org-constant">nodeInspectFunc</span>: <span class="org-keyword">func</span>() (<span class="org-type">swarm.Node</span>, []<span class="org-type">byte</span>, <span class="org-type">error</span>) {
    272                                 <span class="org-keyword">return</span> *<span class="org-function-name">Node</span>(<span class="org-function-name">Manager</span>()), []<span class="org-type">byte</span>{}, <span class="org-constant">nil</span>
    273                         },
    274                 }, <span class="org-rainbow-identifiers-identifier-3">buf</span>))
    275         <span class="org-rainbow-identifiers-identifier-3">cmd</span>.<span class="org-function-name">SetArgs</span>([]<span class="org-type">string</span>{<span class="org-string">"nodeID"</span>})
    276         <span class="org-rainbow-identifiers-identifier-3">cmd</span>.<span class="org-function-name">Flags</span>().<span class="org-function-name">Set</span>(<span class="org-string">"pretty"</span>, <span class="org-string">"true"</span>)
    277         <span class="org-rainbow-identifiers-identifier-11">assert</span>.<span class="org-function-name">NilError</span>(<span class="org-rainbow-identifiers-identifier-8">t</span>, <span class="org-rainbow-identifiers-identifier-3">cmd</span>.<span class="org-function-name">Execute</span>())
    278         <span class="org-rainbow-identifiers-identifier-15">actual</span> := <span class="org-rainbow-identifiers-identifier-3">buf</span>.<span class="org-function-name">String</span>()
    279         <span class="org-rainbow-identifiers-identifier-14">expected</span> := <span class="org-rainbow-identifiers-identifier-1">golden</span>.<span class="org-function-name">Get</span>(<span class="org-rainbow-identifiers-identifier-8">t</span>, []<span class="org-function-name">byte</span>(<span class="org-rainbow-identifiers-identifier-15">actual</span>), <span class="org-string">"myfile.golden"</span>)
    280         <span class="org-rainbow-identifiers-identifier-11">assert</span>.<span class="org-function-name">EqualNormalizedString</span>(<span class="org-rainbow-identifiers-identifier-8">t</span>, <span class="org-rainbow-identifiers-identifier-11">assert</span>.<span class="org-rainbow-identifiers-identifier-2">RemoveSpace</span>, <span class="org-rainbow-identifiers-identifier-15">actual</span>, <span class="org-function-name">string</span>(<span class="org-rainbow-identifiers-identifier-14">expected</span>))
    281 }
    282 </pre>
    283 </div>
    284 
    285 <p>
    286 If we change the output, the workflow becomes the following :
    287 </p>
    288 
    289 <ul class="org-ul">
    290 <li>run <code>go test</code> and make sure it's failing,</li>
    291 <li>Validate that the current output is correct,</li>
    292 <li>run <code>go test -test.update</code> to update the golden file(s),</li>
    293 <li>re-run <code>go test</code> to make sure it's now green,</li>
    294 <li>you are done 👼.</li>
    295 </ul>
    296 
    297 <p>
    298 With this simple trick, our test now <b>contains less noise</b> and is
    299 way more <b>maintainable</b> (you just have a command to run to update
    300 the expected content).
    301 </p>
    302 </div>
    303 </div>
    304 
    305       
    306     </article>
    307     <hr />
    308     <div class="prev-next">
    309       
    310       <a class="paging-link prev" href="/posts/2018-07-28-gotest-tools-intro/" title="Golang testing — gotest.tools introduction">← Previous post</a>
    311       
    312 
    313       
    314       <a class="paging-link next" href="/posts/2017-01-01-go-testing-functionnal-builders/" title="Golang testing — functional arguments for wonderful builders">Next post →</a>
    315       
    316     </div>
    317 
    318   </div>
    319 </div>
    320 
    321 <footer>
    322   <nav>
    323     
    324     <a href="/">home</a>
    325     <span class="text-muted"> | </span>
    326     
    327     <a href="/about">about</a>
    328     <span class="text-muted"> | </span>
    329     
    330     <a href="/archive">archive</a>
    331     <span class="text-muted"> | </span>
    332     
    333     <a href="/categories">categories</a>
    334     <span class="text-muted"> | </span>
    335     
    336     <a href="/tags">tags</a>
    337     <span class="text-muted"> | </span>
    338     
    339     <a href="https://twitter.com/vdemeest">twitter</a>
    340     <span class="text-muted"> | </span>
    341     
    342     <a href="https://github.com/vdemeester">github</a>
    343     <span class="text-muted"> | </span>
    344     
    345     <a href="https://vincent.demeester.fr/index.xml">rss</a>
    346   </nav>
    347   <br/>
    348   <address>
    349     <span class="copyright">
    350       Content and design by Vincent Demeester
    351       (<a rel="licence" href="http://creativecommons.org/licenses/by-nc-sa/3.0/">Some rights reserved</a>)
    352     </span><br />
    353     <span class="engine">
    354       Powered by <a href="https://gohugo.io/">Hugo</a> and <a href="https://github.com/kaushalmodi/ox-hugo/">ox-hugo</a>
    355     </span>
    356   </address>
    357 </footer>
    358 </body>
    359