openshift_on_vm_bare_metal.html (13770B)
1 <!DOCTYPE html> 2 <html lang="en"> 3 <head> 4 <!-- Sep 03, 2024 --> 5 <meta charset="utf-8" /> 6 <meta name="viewport" content="width=device-width, initial-scale=1" /> 7 <title>OpenShift on VM Bare metal</title> 8 <meta name="author" content="Vincent Demeester" /> 9 <meta name="generator" content="Org Mode" /> 10 <link rel='icon' type='image/x-icon' href='/images/favicon.ico'/> 11 <meta name='viewport' content='width=device-width, initial-scale=1'> 12 <link rel='stylesheet' href='/css/new.css' type='text/css'/> 13 <link rel='stylesheet' href='/css/syntax.css' type='text/css'/> 14 <link href='/index.xml' rel='alternate' type='application/rss+xml' title='Vincent Demeester' /> 15 </head> 16 <body> 17 <main id="content" class="content"> 18 <header> 19 <h1 class="title">OpenShift on VM Bare metal</h1> 20 </header><p> 21 Let’s try to install OpenShift 4 on bare metal, where bare metal is virtual machine 22 managed outside of OpenShift reach. We are trying to follow the <a href="https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-bare-metal.html">OpenShift Install on Bare 23 metal</a> official documentation as much as possible. 24 </p> 25 26 <nav id="table-of-contents" role="doc-toc"> 27 <h2>Table of Contents</h2> 28 <div id="text-table-of-contents" role="doc-toc"> 29 <ul> 30 <li><a href="#Prerequisites">Prerequisites</a></li> 31 <li><a href="#Provisionning%20Machines">Provisionning Machines</a> 32 <ul> 33 <li><a href="#Bootstraper">Bootstraper</a></li> 34 <li><a href="#Load%20balancer">Load balancer</a></li> 35 <li><a href="#Master%20and%20worker%20nodes">Master and worker nodes</a></li> 36 </ul> 37 </li> 38 <li><a href="#Links">Links</a></li> 39 </ul> 40 </div> 41 </nav> 42 <section id="outline-container-Prerequisites" class="outline-2"> 43 <h2 id="Prerequisites">Prerequisites</h2> 44 <div class="outline-text-2" id="text-Prerequisites"> 45 <p> 46 In my <a href="infrastructure.html">infrastructure</a> I do have at my disposal 2 machines that both have libvirt (qemu+kvm) 47 enabled (with nested virtualization support even 😝). 48 </p> 49 50 <ul class="org-ul"> 51 <li>okinawa: 8 cores, 32GiB memory</li> 52 <li>wakasu: 8 cores, 64GiB memory</li> 53 </ul> 54 55 <p> 56 Because we are going to install the machine <i>bare metal</i> on virtual machine, some 57 assumptions are made (see <a href="infrastructure.html">Infrastructure</a> on those). 58 </p> 59 60 <ul class="org-ul"> 61 <li>VMs are created using a pre-defined Mac address, and the DHCP server on the network 62 assign them a static IP (based on the mac address).</li> 63 <li>This allows to create <span class="underline">ahead</span> of time DNS entries on the network so that the bootstrap 64 and the cluster can find its own by domain name.</li> 65 </ul> 66 67 <p> 68 My <i>home</i> network is using <code>home</code> as domain name tld. So VMs will be <code>vm1.home</code>, … and for 69 the ocp cluster, <i>aliased</i> with <code>ocp.home</code>. We will have 3 master and 5 workers 🙃. The 70 DNS entry looks more or less like 71 </p> 72 73 <div class="org-src-container"> 74 <pre class="src src-text">;; Load balancer 75 vm0.home. IN A 192.168.1.120 76 api.ocp.home. IN A 192.168.1.120 77 api-int.ocp.home. IN A 192.168.1.120 78 *.apps.ocp.home. IN A 192.168.1.120 79 80 ;; Masters 81 vm1.home. IN A 192.168.1.121 82 master1.ocp.home. IN A 192.168.1.121 83 vm2.home. IN A 192.168.1.122 84 master2.ocp.home. IN A 192.168.1.122 85 vm3.home. IN A 192.168.1.123 86 master3.ocp.home. IN A 192.168.1.123 87 88 ;; Workers 89 vm4.home. IN A 192.168.1.124 90 worker1.ocp.home. IN A 192.168.1.124 91 vm5.home. IN A 192.168.1.125 92 worker2.ocp.home. IN A 192.168.1.125 93 vm6.home. IN A 192.168.1.126 94 worker3.ocp.home. IN A 192.168.1.126 95 vm7.home. IN A 192.168.1.127 96 worker4.ocp.home. IN A 192.168.1.127 97 vm8.home. IN A 192.168.1.128 98 worker5.ocp.home. IN A 192.168.1.128 99 100 ;; Bootstrap machine 101 vm9.home. IN A 192.168.1.129 102 bootstrap.ocp.home. IN A 192.168.1.129 103 104 ;; etcd records 105 etcd-0.ocp.home. IN A 192.168.1.121 106 etcd-1.ocp.home. IN A 192.168.1.122 107 etcd-2.ocp.home. IN A 192.168.1.123 108 _etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-0.ocp.home. 109 _etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-1.ocp.home. 110 _etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-2.ocp.haome. 111 </pre> 112 </div> 113 114 <p> 115 As you can see, aside from the 3 masters and the 5 workers, we will have a load-balancer 116 VM as documented in the official documentation. 117 </p> 118 119 <p> 120 For reference, VMs Mac address compared to their IP. 121 </p> 122 123 <table> 124 125 126 <colgroup> 127 <col class="org-left"> 128 129 <col class="org-right"> 130 131 <col class="org-left"> 132 </colgroup> 133 <tbody> 134 <tr> 135 <td class="org-left">MAC address</td> 136 <td class="org-right">IP</td> 137 <td class="org-left">Name</td> 138 </tr> 139 140 <tr> 141 <td class="org-left">52:54:00:dd:a3:20</td> 142 <td class="org-right">192.168.1.120</td> 143 <td class="org-left">VM0</td> 144 </tr> 145 146 <tr> 147 <td class="org-left">52:54:00:dd:a3:21</td> 148 <td class="org-right">192.168.1.121</td> 149 <td class="org-left">VM1</td> 150 </tr> 151 152 <tr> 153 <td class="org-left">52:54:00:dd:a3:22</td> 154 <td class="org-right">192.168.1.122</td> 155 <td class="org-left">VM2</td> 156 </tr> 157 158 <tr> 159 <td class="org-left">52:54:00:dd:a3:23</td> 160 <td class="org-right">192.168.1.123</td> 161 <td class="org-left">VM3</td> 162 </tr> 163 164 <tr> 165 <td class="org-left">52:54:00:dd:a3:24</td> 166 <td class="org-right">192.168.1.124</td> 167 <td class="org-left">VM4</td> 168 </tr> 169 170 <tr> 171 <td class="org-left">52:54:00:dd:a3:25</td> 172 <td class="org-right">192.168.1.125</td> 173 <td class="org-left">VM5</td> 174 </tr> 175 176 <tr> 177 <td class="org-left">52:54:00:dd:a3:26</td> 178 <td class="org-right">192.168.1.126</td> 179 <td class="org-left">VM6</td> 180 </tr> 181 182 <tr> 183 <td class="org-left">52:54:00:dd:a3:27</td> 184 <td class="org-right">192.168.1.127</td> 185 <td class="org-left">VM7</td> 186 </tr> 187 188 <tr> 189 <td class="org-left">52:54:00:dd:a3:28</td> 190 <td class="org-right">192.168.1.128</td> 191 <td class="org-left">VM8</td> 192 </tr> 193 194 <tr> 195 <td class="org-left">52:54:00:dd:a3:29</td> 196 <td class="org-right">192.168.1.129</td> 197 <td class="org-left">VM9</td> 198 </tr> 199 </tbody> 200 </table> 201 202 <p> 203 Although I am tempted to use <a href="https://github.com/RedHatOfficial/ocp4-helpernode">ocp4-helpernode</a>, I am going to try to do as much as I can 204 with my bare hands ✋. 205 </p> 206 </div> 207 </section> 208 <section id="outline-container-Provisionning%20Machines" class="outline-2"> 209 <h2 id="Provisionning%20Machines">Provisionning Machines</h2> 210 <div class="outline-text-2" id="text-Provisionning%20Machines"> 211 <p> 212 We will need to provision some machines, using <code>virsh</code>, <code>virt-install</code> or whatever works 213 the best to be honest. This is also gonna be <b>heavily</b> based on 214 <a href="https://github.com/RedHatOfficial/ocp4-helpernode/blob/master/docs/quickstart-static.md">ocp4-helpernode/quickstart-static.md</a>. 215 </p> 216 </div> 217 <div id="outline-container-Bootstraper" class="outline-3"> 218 <h3 id="Bootstraper">Bootstraper</h3> 219 <div class="outline-text-3" id="text-Bootstraper"> 220 <p> 221 For the bootstrap VM (<code>vm9</code>), we can go with any system but, let’s follow closely the 222 quistart and use Centos EL8. The boostraper machine will be on <code>wakasu</code> so we will need to 223 target the libvirt daemon from there. 224 </p> 225 226 <div class="org-src-container"> 227 <pre class="src src-shell">export QEMU_URI=qemu+ssh://vincent@wakasu.home/system 228 virt-install --connect=${QEMU_URI} \ 229 --name="ocp4-bootstrap" --vcpus=4 --ram=8192 \ 230 --disk path=/var/lib/libvirt/images/ocp-bootstrap.qcow2,bus=virtio,size=120 \ 231 --boot menu=on --print-xml > ocp4-bootstrap.xml 232 virsh --connect=${QEMU_URI} \ 233 define --file ocp4-bootstrap.xml 234 </pre> 235 </div> 236 </div> 237 </div> 238 <div id="outline-container-Load%20balancer" class="outline-3"> 239 <h3 id="Load%20balancer">Load balancer</h3> 240 </div> 241 242 <div id="outline-container-Master%20and%20worker%20nodes" class="outline-3"> 243 <h3 id="Master%20and%20worker%20nodes">Master and worker nodes</h3> 244 <div class="outline-text-3" id="text-Master%20and%20worker%20nodes"> 245 <p> 246 Let’s group machines between wakasu and okinawa. There will be 2 masters on wakasu and one 247 on okinawa, 3 workers on wakasu, and 2 on okinawa. 248 </p> 249 250 <ul class="org-ul"> 251 <li><p> 252 <code>wakasu</code> 253 </p> 254 <div class="org-src-container"> 255 <pre class="src src-bash">export QEMU_URI=qemu+ssh://vincent@wakasu.home/system 256 virt-install --connect=${QEMU_URI} \ 257 --name="ocp4-master1" --vcpus=4 --ram=12288 \ 258 --disk path=/var/lib/libvirt/images/ocp4-master1.qcow2,bus=virtio,size=120 \ 259 --os-variant rhel8.0 \ 260 --network bridge=br1,mac.address=52:54:00:dd:a3:21 \ 261 --boot menu=on --print-xml > ocp4-wakasu-master1.xml 262 virt-install --connect=${QEMU_URI} \ 263 --name="ocp4-master2" --vcpus=4 --ram=12288 \ 264 --disk path=/var/lib/libvirt/images/ocp4-master2.qcow2,bus=virtio,size=120 \ 265 --os-variant rhel8.0 \ 266 --network bridge=br1,mac.address=52:54:00:dd:a3:22 \ 267 --boot menu=on --print-xml > ocp4-wakasu-master2.xml 268 virt-install --connect=${QEMU_URI} \ 269 --name="ocp4-worker1" --vcpus=4 --ram=8192 \ 270 --disk path=/var/lib/libvirt/images/ocp4-worker1.qcow2,bus=virtio,size=120 \ 271 --os-variant rhel8.0 \ 272 --network bridge=br1,mac.address=52:54:00:dd:a3:24 \ 273 --boot menu=on --print-xml > ocp4-wakasu-worker1.xml 274 virt-install --connect=${QEMU_URI} \ 275 --name="ocp4-worker2" --vcpus=4 --ram=8192 \ 276 --disk path=/var/lib/libvirt/images/ocp4-worker2.qcow2,bus=virtio,size=120 \ 277 --os-variant rhel8.0 \ 278 --network bridge=br1,mac.address=52:54:00:dd:a3:25 \ 279 --boot menu=on --print-xml > ocp4-wakasu-worker2.xml 280 virt-install --connect=${QEMU_URI} \ 281 --name="ocp4-worker3" --vcpus=4 --ram=8192 \ 282 --disk path=/var/lib/libvirt/images/ocp4-worker3.qcow2,bus=virtio,size=120 \ 283 --os-variant rhel8.0 \ 284 --network bridge=br1,mac.address=52:54:00:dd:a3:26 \ 285 --boot menu=on --print-xml > ocp4-wakasu-worker3.xml 286 for f in ocp4-wakasu-*.xml; do 287 virsh --connect=${QEMU_URI} define \ 288 --file $f 289 done 290 </pre> 291 </div> 292 293 <p> 294 Domain ocp4-master1 defined from ocp4-wakasu-master1.xml 295 </p> 296 297 <p> 298 Domain ocp4-master2 defined from ocp4-wakasu-master2.xml 299 </p> 300 301 <p> 302 Domain ocp4-worker1 defined from ocp4-wakasu-worker1.xml 303 </p> 304 305 <p> 306 Domain ocp4-worker2 defined from ocp4-wakasu-worker2.xml 307 </p> 308 309 <p> 310 Domain ocp4-worker3 defined from ocp4-wakasu-worker3.xml 311 </p></li> 312 </ul> 313 314 315 316 <ul class="org-ul"> 317 <li><p> 318 <code>okinawa</code> 319 </p> 320 <div class="org-src-container"> 321 <pre class="src src-bash">export QEMU_URI=qemu+ssh://vincent@okinawa.home/system 322 virt-install --connect=${QEMU_URI} \ 323 --name="ocp4-master3" --vcpus=4 --ram=12288 \ 324 --disk path=/var/lib/libvirt/images/ocp4-master3.qcow2,bus=virtio,size=120 \ 325 --os-variant rhel8.0 \ 326 --network bridge=br1,mac.address=52:54:00:dd:a3:23 \ 327 --boot menu=on --print-xml > ocp4-okinawa-master1.xml 328 virt-install --connect=${QEMU_URI} \ 329 --name="ocp4-worker4" --vcpus=4 --ram=8192 \ 330 --disk path=/var/lib/libvirt/images/ocp4-worker4.qcow2,bus=virtio,size=120 \ 331 --os-variant rhel8.0 \ 332 --network bridge=br1,mac.address=52:54:00:dd:a3:27 \ 333 --boot menu=on --print-xml > ocp4-okinawa-worker1.xml 334 virt-install --connect=${QEMU_URI} \ 335 --name="ocp4-worker5" --vcpus=4 --ram=8192 \ 336 --disk path=/var/lib/libvirt/images/ocp4-worker5.qcow2,bus=virtio,size=120 \ 337 --os-variant rhel8.0 \ 338 --network bridge=br1,mac.address=52:54:00:dd:a3:28 \ 339 --boot menu=on --print-xml > ocp4-okinawa-worker2.xml 340 for f in ocp4-okinawa-*.xml; do 341 virsh --connect=${QEMU_URI} define \ 342 --file $f 343 done 344 345 </pre> 346 </div> 347 348 <table> 349 350 351 <colgroup> 352 <col class="org-left"> 353 354 <col class="org-left"> 355 356 <col class="org-left"> 357 358 <col class="org-left"> 359 360 <col class="org-left"> 361 </colgroup> 362 <tbody> 363 <tr> 364 <td class="org-left">Domain</td> 365 <td class="org-left">ocp4-master3</td> 366 <td class="org-left">defined</td> 367 <td class="org-left">from</td> 368 <td class="org-left">ocp4-okinawa-master1.xml</td> 369 </tr> 370 371 <tr> 372 <td class="org-left">Domain</td> 373 <td class="org-left">ocp4-worker4</td> 374 <td class="org-left">defined</td> 375 <td class="org-left">from</td> 376 <td class="org-left">ocp4-okinawa-worker1.xml</td> 377 </tr> 378 379 <tr> 380 <td class="org-left">Domain</td> 381 <td class="org-left">ocp4-worker5</td> 382 <td class="org-left">defined</td> 383 <td class="org-left">from</td> 384 <td class="org-left">ocp4-okinawa-worker2.xml</td> 385 </tr> 386 </tbody> 387 </table></li> 388 </ul> 389 </div> 390 </div> 391 </section> 392 <section id="outline-container-Links" class="outline-2"> 393 <h2 id="Links">Links</h2> 394 <div class="outline-text-2" id="text-Links"> 395 <ul class="org-ul"> 396 <li><a href="https://github.com/RedHatOfficial/ocp4-helpernode">ocp4-helpernode</a></li> 397 <li><a href="https://www.openshift.com/blog/openshift-4-bare-metal-install-quickstart">OpenShift 4 Bare metal quickstart</a> blog post</li> 398 <li><a href="https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-bare-metal.html">OpenShift Install on Bare metal</a> official doc</li> 399 </ul> 400 </div> 401 </section> 402 </main> 403 <footer id="postamble" class="status"> 404 <footer> 405 <small><a href="/" rel="history">Index</a> • <a href="/sitemap.html">Sitemap</a> • <a href="https://dl.sbr.pm/">Files</a></small><br/> 406 <small class='questions'>Questions, comments ? Please use my <a href="https://lists.sr.ht/~vdemeester/public-inbox">public inbox</a> by sending a plain-text email to <a href="mailto:~vdemeester/public-inbox@lists.sr.ht">~vdemeester/public-inbox@lists.sr.ht</a>.</small><br/> 407 <small class='copyright'> 408 Content and design by Vincent Demeester 409 (<a rel='licence' href='http://creativecommons.org/licenses/by-nc-sa/3.0/'>Some rights reserved</a>) 410 </small><br /> 411 </footer> 412 </footer> 413 </body> 414 </html>