www

My personal website(s)
Log | Files | Refs

openshift_on_vm_bare_metal.html (13770B)


      1 <!DOCTYPE html>
      2 <html lang="en">
      3 <head>
      4 <!-- Sep 03, 2024 -->
      5 <meta charset="utf-8" />
      6 <meta name="viewport" content="width=device-width, initial-scale=1" />
      7 <title>OpenShift on VM Bare metal</title>
      8 <meta name="author" content="Vincent Demeester" />
      9 <meta name="generator" content="Org Mode" />
     10 <link rel='icon' type='image/x-icon' href='/images/favicon.ico'/>
     11 <meta name='viewport' content='width=device-width, initial-scale=1'>
     12 <link rel='stylesheet' href='/css/new.css' type='text/css'/>
     13 <link rel='stylesheet' href='/css/syntax.css' type='text/css'/>
     14 <link href='/index.xml' rel='alternate' type='application/rss+xml' title='Vincent Demeester' />
     15 </head>
     16 <body>
     17 <main id="content" class="content">
     18 <header>
     19 <h1 class="title">OpenShift on VM Bare metal</h1>
     20 </header><p>
     21 Let&rsquo;s try to install OpenShift 4 on bare metal, where bare metal is virtual machine
     22 managed outside of OpenShift reach. We are trying to follow the <a href="https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-bare-metal.html">OpenShift Install on Bare
     23 metal</a> official documentation as much as possible.
     24 </p>
     25 
     26 <nav id="table-of-contents" role="doc-toc">
     27 <h2>Table of Contents</h2>
     28 <div id="text-table-of-contents" role="doc-toc">
     29 <ul>
     30 <li><a href="#Prerequisites">Prerequisites</a></li>
     31 <li><a href="#Provisionning%20Machines">Provisionning Machines</a>
     32 <ul>
     33 <li><a href="#Bootstraper">Bootstraper</a></li>
     34 <li><a href="#Load%20balancer">Load balancer</a></li>
     35 <li><a href="#Master%20and%20worker%20nodes">Master and worker nodes</a></li>
     36 </ul>
     37 </li>
     38 <li><a href="#Links">Links</a></li>
     39 </ul>
     40 </div>
     41 </nav>
     42 <section id="outline-container-Prerequisites" class="outline-2">
     43 <h2 id="Prerequisites">Prerequisites</h2>
     44 <div class="outline-text-2" id="text-Prerequisites">
     45 <p>
     46 In my <a href="infrastructure.html">infrastructure</a> I do have at my disposal 2 machines that both have libvirt (qemu+kvm)
     47 enabled (with nested virtualization support even 😝).
     48 </p>
     49 
     50 <ul class="org-ul">
     51 <li>okinawa: 8 cores, 32GiB memory</li>
     52 <li>wakasu: 8 cores, 64GiB memory</li>
     53 </ul>
     54 
     55 <p>
     56 Because we are going to install the machine <i>bare metal</i> on virtual machine, some
     57 assumptions are made (see <a href="infrastructure.html">Infrastructure</a> on those).
     58 </p>
     59 
     60 <ul class="org-ul">
     61 <li>VMs are created using a pre-defined Mac address, and the DHCP server on the network
     62 assign them a static IP (based on the mac address).</li>
     63 <li>This allows to create <span class="underline">ahead</span> of time DNS entries on the network so that the bootstrap
     64 and the cluster can find its own by domain name.</li>
     65 </ul>
     66 
     67 <p>
     68 My <i>home</i> network is using <code>home</code> as domain name tld. So VMs will be <code>vm1.home</code>, … and for
     69 the ocp cluster, <i>aliased</i> with <code>ocp.home</code>. We will have 3 master and 5 workers 🙃. The
     70 DNS entry looks more or less like
     71 </p>
     72 
     73 <div class="org-src-container">
     74 <pre class="src src-text">;; Load balancer
     75 vm0.home. IN A 192.168.1.120
     76 api.ocp.home. IN A 192.168.1.120
     77 api-int.ocp.home. IN A 192.168.1.120
     78 *.apps.ocp.home. IN A 192.168.1.120
     79 
     80 ;; Masters
     81 vm1.home. IN A 192.168.1.121
     82 master1.ocp.home. IN A 192.168.1.121
     83 vm2.home. IN A 192.168.1.122
     84 master2.ocp.home. IN A 192.168.1.122
     85 vm3.home. IN A 192.168.1.123
     86 master3.ocp.home. IN A 192.168.1.123
     87 
     88 ;; Workers
     89 vm4.home. IN A 192.168.1.124
     90 worker1.ocp.home. IN A 192.168.1.124
     91 vm5.home. IN A 192.168.1.125
     92 worker2.ocp.home. IN A 192.168.1.125
     93 vm6.home. IN A 192.168.1.126
     94 worker3.ocp.home. IN A 192.168.1.126
     95 vm7.home. IN A 192.168.1.127
     96 worker4.ocp.home. IN A 192.168.1.127
     97 vm8.home. IN A 192.168.1.128
     98 worker5.ocp.home. IN A 192.168.1.128
     99 
    100 ;; Bootstrap machine
    101 vm9.home. IN A 192.168.1.129
    102 bootstrap.ocp.home. IN A 192.168.1.129
    103 
    104 ;; etcd records
    105 etcd-0.ocp.home. IN A 192.168.1.121
    106 etcd-1.ocp.home. IN A 192.168.1.122
    107 etcd-2.ocp.home. IN A 192.168.1.123
    108 _etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-0.ocp.home.
    109 _etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-1.ocp.home.
    110 _etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-2.ocp.haome.
    111 </pre>
    112 </div>
    113 
    114 <p>
    115 As you can see, aside from the 3 masters and the 5 workers, we will have a load-balancer
    116 VM as documented in the official documentation.
    117 </p>
    118 
    119 <p>
    120 For reference, VMs Mac address compared to their IP.
    121 </p>
    122 
    123 <table>
    124 
    125 
    126 <colgroup>
    127 <col  class="org-left">
    128 
    129 <col  class="org-right">
    130 
    131 <col  class="org-left">
    132 </colgroup>
    133 <tbody>
    134 <tr>
    135 <td class="org-left">MAC address</td>
    136 <td class="org-right">IP</td>
    137 <td class="org-left">Name</td>
    138 </tr>
    139 
    140 <tr>
    141 <td class="org-left">52:54:00:dd:a3:20</td>
    142 <td class="org-right">192.168.1.120</td>
    143 <td class="org-left">VM0</td>
    144 </tr>
    145 
    146 <tr>
    147 <td class="org-left">52:54:00:dd:a3:21</td>
    148 <td class="org-right">192.168.1.121</td>
    149 <td class="org-left">VM1</td>
    150 </tr>
    151 
    152 <tr>
    153 <td class="org-left">52:54:00:dd:a3:22</td>
    154 <td class="org-right">192.168.1.122</td>
    155 <td class="org-left">VM2</td>
    156 </tr>
    157 
    158 <tr>
    159 <td class="org-left">52:54:00:dd:a3:23</td>
    160 <td class="org-right">192.168.1.123</td>
    161 <td class="org-left">VM3</td>
    162 </tr>
    163 
    164 <tr>
    165 <td class="org-left">52:54:00:dd:a3:24</td>
    166 <td class="org-right">192.168.1.124</td>
    167 <td class="org-left">VM4</td>
    168 </tr>
    169 
    170 <tr>
    171 <td class="org-left">52:54:00:dd:a3:25</td>
    172 <td class="org-right">192.168.1.125</td>
    173 <td class="org-left">VM5</td>
    174 </tr>
    175 
    176 <tr>
    177 <td class="org-left">52:54:00:dd:a3:26</td>
    178 <td class="org-right">192.168.1.126</td>
    179 <td class="org-left">VM6</td>
    180 </tr>
    181 
    182 <tr>
    183 <td class="org-left">52:54:00:dd:a3:27</td>
    184 <td class="org-right">192.168.1.127</td>
    185 <td class="org-left">VM7</td>
    186 </tr>
    187 
    188 <tr>
    189 <td class="org-left">52:54:00:dd:a3:28</td>
    190 <td class="org-right">192.168.1.128</td>
    191 <td class="org-left">VM8</td>
    192 </tr>
    193 
    194 <tr>
    195 <td class="org-left">52:54:00:dd:a3:29</td>
    196 <td class="org-right">192.168.1.129</td>
    197 <td class="org-left">VM9</td>
    198 </tr>
    199 </tbody>
    200 </table>
    201 
    202 <p>
    203 Although I am tempted to use <a href="https://github.com/RedHatOfficial/ocp4-helpernode">ocp4-helpernode</a>, I am going to try to do as much as I can
    204 with my bare hands ✋.
    205 </p>
    206 </div>
    207 </section>
    208 <section id="outline-container-Provisionning%20Machines" class="outline-2">
    209 <h2 id="Provisionning%20Machines">Provisionning Machines</h2>
    210 <div class="outline-text-2" id="text-Provisionning%20Machines">
    211 <p>
    212 We will need to provision some machines, using <code>virsh</code>, <code>virt-install</code> or whatever works
    213 the best to be honest. This is also gonna be <b>heavily</b> based on
    214 <a href="https://github.com/RedHatOfficial/ocp4-helpernode/blob/master/docs/quickstart-static.md">ocp4-helpernode/quickstart-static.md</a>.
    215 </p>
    216 </div>
    217 <div id="outline-container-Bootstraper" class="outline-3">
    218 <h3 id="Bootstraper">Bootstraper</h3>
    219 <div class="outline-text-3" id="text-Bootstraper">
    220 <p>
    221 For the bootstrap VM (<code>vm9</code>), we can go with any system but, let&rsquo;s follow closely the
    222 quistart and use Centos EL8. The boostraper machine will be on <code>wakasu</code> so we will need to
    223 target the libvirt daemon from there.
    224 </p>
    225 
    226 <div class="org-src-container">
    227 <pre class="src src-shell">export QEMU_URI=qemu+ssh://vincent@wakasu.home/system
    228 virt-install --connect=${QEMU_URI} \
    229              --name="ocp4-bootstrap" --vcpus=4 --ram=8192 \
    230              --disk path=/var/lib/libvirt/images/ocp-bootstrap.qcow2,bus=virtio,size=120 \
    231              --boot menu=on --print-xml &gt; ocp4-bootstrap.xml
    232 virsh --connect=${QEMU_URI} \
    233       define --file ocp4-bootstrap.xml
    234 </pre>
    235 </div>
    236 </div>
    237 </div>
    238 <div id="outline-container-Load%20balancer" class="outline-3">
    239 <h3 id="Load%20balancer">Load balancer</h3>
    240 </div>
    241 
    242 <div id="outline-container-Master%20and%20worker%20nodes" class="outline-3">
    243 <h3 id="Master%20and%20worker%20nodes">Master and worker nodes</h3>
    244 <div class="outline-text-3" id="text-Master%20and%20worker%20nodes">
    245 <p>
    246 Let&rsquo;s group machines between wakasu and okinawa. There will be 2 masters on wakasu and one
    247 on okinawa, 3 workers on wakasu, and 2 on okinawa.
    248 </p>
    249 
    250 <ul class="org-ul">
    251 <li><p>
    252 <code>wakasu</code>
    253 </p>
    254 <div class="org-src-container">
    255 <pre class="src src-bash">export QEMU_URI=qemu+ssh://vincent@wakasu.home/system
    256 virt-install --connect=${QEMU_URI} \
    257              --name="ocp4-master1" --vcpus=4 --ram=12288 \
    258              --disk path=/var/lib/libvirt/images/ocp4-master1.qcow2,bus=virtio,size=120 \
    259              --os-variant rhel8.0 \
    260              --network bridge=br1,mac.address=52:54:00:dd:a3:21 \
    261              --boot menu=on --print-xml &gt; ocp4-wakasu-master1.xml
    262 virt-install --connect=${QEMU_URI} \
    263              --name="ocp4-master2" --vcpus=4 --ram=12288 \
    264              --disk path=/var/lib/libvirt/images/ocp4-master2.qcow2,bus=virtio,size=120 \
    265              --os-variant rhel8.0 \
    266              --network bridge=br1,mac.address=52:54:00:dd:a3:22 \
    267              --boot menu=on --print-xml &gt; ocp4-wakasu-master2.xml
    268 virt-install --connect=${QEMU_URI} \
    269              --name="ocp4-worker1" --vcpus=4 --ram=8192 \
    270              --disk path=/var/lib/libvirt/images/ocp4-worker1.qcow2,bus=virtio,size=120 \
    271              --os-variant rhel8.0 \
    272              --network bridge=br1,mac.address=52:54:00:dd:a3:24 \
    273              --boot menu=on --print-xml &gt; ocp4-wakasu-worker1.xml
    274 virt-install --connect=${QEMU_URI} \
    275              --name="ocp4-worker2" --vcpus=4 --ram=8192 \
    276              --disk path=/var/lib/libvirt/images/ocp4-worker2.qcow2,bus=virtio,size=120 \
    277              --os-variant rhel8.0 \
    278              --network bridge=br1,mac.address=52:54:00:dd:a3:25 \
    279              --boot menu=on --print-xml &gt; ocp4-wakasu-worker2.xml
    280 virt-install --connect=${QEMU_URI} \
    281              --name="ocp4-worker3" --vcpus=4 --ram=8192 \
    282              --disk path=/var/lib/libvirt/images/ocp4-worker3.qcow2,bus=virtio,size=120 \
    283              --os-variant rhel8.0 \
    284              --network bridge=br1,mac.address=52:54:00:dd:a3:26 \
    285              --boot menu=on --print-xml &gt; ocp4-wakasu-worker3.xml
    286 for f in ocp4-wakasu-*.xml; do
    287     virsh --connect=${QEMU_URI} define \
    288           --file $f
    289 done
    290 </pre>
    291 </div>
    292 
    293 <p>
    294 Domain ocp4-master1 defined from ocp4-wakasu-master1.xml
    295 </p>
    296 
    297 <p>
    298 Domain ocp4-master2 defined from ocp4-wakasu-master2.xml
    299 </p>
    300 
    301 <p>
    302 Domain ocp4-worker1 defined from ocp4-wakasu-worker1.xml
    303 </p>
    304 
    305 <p>
    306 Domain ocp4-worker2 defined from ocp4-wakasu-worker2.xml
    307 </p>
    308 
    309 <p>
    310 Domain ocp4-worker3 defined from ocp4-wakasu-worker3.xml
    311 </p></li>
    312 </ul>
    313 
    314 
    315 
    316 <ul class="org-ul">
    317 <li><p>
    318 <code>okinawa</code>
    319 </p>
    320 <div class="org-src-container">
    321 <pre class="src src-bash">export QEMU_URI=qemu+ssh://vincent@okinawa.home/system
    322 virt-install --connect=${QEMU_URI} \
    323              --name="ocp4-master3" --vcpus=4 --ram=12288 \
    324              --disk path=/var/lib/libvirt/images/ocp4-master3.qcow2,bus=virtio,size=120 \
    325              --os-variant rhel8.0 \
    326              --network bridge=br1,mac.address=52:54:00:dd:a3:23 \
    327              --boot menu=on --print-xml &gt; ocp4-okinawa-master1.xml
    328 virt-install --connect=${QEMU_URI} \
    329              --name="ocp4-worker4" --vcpus=4 --ram=8192 \
    330              --disk path=/var/lib/libvirt/images/ocp4-worker4.qcow2,bus=virtio,size=120 \
    331              --os-variant rhel8.0 \
    332              --network bridge=br1,mac.address=52:54:00:dd:a3:27 \
    333              --boot menu=on --print-xml &gt; ocp4-okinawa-worker1.xml
    334 virt-install --connect=${QEMU_URI} \
    335              --name="ocp4-worker5" --vcpus=4 --ram=8192 \
    336              --disk path=/var/lib/libvirt/images/ocp4-worker5.qcow2,bus=virtio,size=120 \
    337              --os-variant rhel8.0 \
    338              --network bridge=br1,mac.address=52:54:00:dd:a3:28 \
    339              --boot menu=on --print-xml &gt; ocp4-okinawa-worker2.xml
    340 for f in ocp4-okinawa-*.xml; do
    341     virsh --connect=${QEMU_URI} define \
    342           --file $f
    343 done
    344 
    345 </pre>
    346 </div>
    347 
    348 <table>
    349 
    350 
    351 <colgroup>
    352 <col  class="org-left">
    353 
    354 <col  class="org-left">
    355 
    356 <col  class="org-left">
    357 
    358 <col  class="org-left">
    359 
    360 <col  class="org-left">
    361 </colgroup>
    362 <tbody>
    363 <tr>
    364 <td class="org-left">Domain</td>
    365 <td class="org-left">ocp4-master3</td>
    366 <td class="org-left">defined</td>
    367 <td class="org-left">from</td>
    368 <td class="org-left">ocp4-okinawa-master1.xml</td>
    369 </tr>
    370 
    371 <tr>
    372 <td class="org-left">Domain</td>
    373 <td class="org-left">ocp4-worker4</td>
    374 <td class="org-left">defined</td>
    375 <td class="org-left">from</td>
    376 <td class="org-left">ocp4-okinawa-worker1.xml</td>
    377 </tr>
    378 
    379 <tr>
    380 <td class="org-left">Domain</td>
    381 <td class="org-left">ocp4-worker5</td>
    382 <td class="org-left">defined</td>
    383 <td class="org-left">from</td>
    384 <td class="org-left">ocp4-okinawa-worker2.xml</td>
    385 </tr>
    386 </tbody>
    387 </table></li>
    388 </ul>
    389 </div>
    390 </div>
    391 </section>
    392 <section id="outline-container-Links" class="outline-2">
    393 <h2 id="Links">Links</h2>
    394 <div class="outline-text-2" id="text-Links">
    395 <ul class="org-ul">
    396 <li><a href="https://github.com/RedHatOfficial/ocp4-helpernode">ocp4-helpernode</a></li>
    397 <li><a href="https://www.openshift.com/blog/openshift-4-bare-metal-install-quickstart">OpenShift 4 Bare metal quickstart</a> blog post</li>
    398 <li><a href="https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-bare-metal.html">OpenShift Install on Bare metal</a> official doc</li>
    399 </ul>
    400 </div>
    401 </section>
    402 </main>
    403 <footer id="postamble" class="status">
    404 <footer>
    405      <small><a href="/" rel="history">Index</a> • <a href="/sitemap.html">Sitemap</a> • <a href="https://dl.sbr.pm/">Files</a></small><br/>
    406      <small class='questions'>Questions, comments ? Please use my <a href="https://lists.sr.ht/~vdemeester/public-inbox">public inbox</a> by sending a plain-text email to <a href="mailto:~vdemeester/public-inbox@lists.sr.ht">~vdemeester/public-inbox@lists.sr.ht</a>.</small><br/>
    407      <small class='copyright'>
    408       Content and design by Vincent Demeester
    409       (<a rel='licence' href='http://creativecommons.org/licenses/by-nc-sa/3.0/'>Some rights reserved</a>)
    410     </small><br />
    411 </footer>
    412 </footer>
    413 </body>
    414 </html>